Comparison of small open source LLMs (8b parameters or less)
General | Tool Usage | Chat Room | Logic | Vision | Vision OCR | Code Generation | More
All testing done with Ollama
| Key | |
|---|---|
| ✅ | Pass |
| ☑️ | Partial Pass |
| ❌ | Fail |
| 🕒 | Timeout |
| 😵💫 | Hallucinations |
| 🚫 | Rejected prompt / unsafe content |
| 💔 | Malformed Response / Tool call |
Tested with attogram/ollama-multirun
| Small Model | Hovercraft | Pirate | Who |
|---|---|---|---|
| bakllava:7b | ❌ | ✅ | |
| codellama:7b | ✅ | ✅ | ✅ |
| cogito:3b | ✅ | ✅ | |
| cogito:bb | ✅ | ✅ | |
| deepcoder:1.5b | ✅ | ☑️ | ✅ |
| deepseek-r1:1.5b | ✅ | ☑️ | ✅ |
| deepseek-r1:8b | ☑️ | ✅ | ✅ |
| dolphin-mistral:7b | ☑️ | ✅ | ✅ |
| dolphin3:8b | ✅ | ✅ | ✅ |
| gemma3:1b | ☑️ | ✅ | ✅ |
| gemma3:4b | ☑️ | ✅ | |
| gemma:2b | ☑️ | ✅ | |
| granite3.2-vision:2b | ✅ | ✅ | |
| granite3.3:2b | ☑️ | ✅ | |
| huihui_ai/baronllm-abliterated:8b | ☑️ | ✅ | |
| llama3-groq-tool-use:8b | ☑️ | ✅ | |
| llama3.2:1b | ✅ | ☑️ | |
| llava-llama3:8b | ☑️ | ✅ | |
| llava-phi3:3.8b | ✅ | ✅ | |
| llava:7b | ✅ | ✅ | |
| minicpm-v:8b | ❌ | ✅ | |
| mistral:7b | ☑️ | ✅ | |
| qwen2.5-coder:7b | ✅ | ✅ | |
| qwen2.5vl:3b | ✅ | ✅ | |
| qwen2.5vl:7b | ☑️ | ✅ | |
| qwen3:1.7b | ✅ | 🕒 | ✅ |
| qwen3:4b | ✅ | 🕒 | ✅ |
| qwen3:8b | ☑️ | ✅ | ✅ |
| smollm2:135m | ✅ | ✅ | |
| smollm2:360m | ☑️ | ✅ | |
| smollm2:1.7b | ✅ | ✅ | |
| stable-code:3b | ☑️ | ☑️ | ✅ |
| starcoder:7b | ❌ | ❌ | ❌ |
| Small Model | Hovercraft | Pirate | Who |
deepseek-r1:7b
llama3.1:8b
llama3.3:3b
llama3:8b
phi:2.7b
phi3:3.8b
tinyllama:1.1b
qwen3:0.6b
qwen3:4b
Tested with attogram/ollama-bash-toolshed
| Passing | Math | Ollama | Time | Web |
|---|---|---|---|---|
| cogito:3b | ✅ | ✅ | ✅ | ✅ |
| cogito:8b | ✅ | ✅ | ✅ | ✅ |
| granite3.3:2b | ✅ | ✅ | ✅ | ✅ |
| hermes3:8b | ✅ | ✅ | ✅ | ✅ |
| llama3.2:1b | ✅ | ✅ | ✅ | ✅ |
| llama3.2:3b | ✅ | ✅ | ✅ | ✅ |
| qwen3:1.7b | ✅ | ✅ | ✅ | ✅ |
| qwen3:4b | ✅ | ✅ | ✅ | ✅ |
| qwen3:8b | ✅ | ✅ | ✅ | ✅ |
| Failing | Math | Ollama | Time | Web |
|---|---|---|---|---|
| command-r7b:7b | ❌ | ❌ | ❌ | ❌ |
| granite3.2-vision:2b | ❌ | ❌ | ❌ | ❌ |
| granite3.3:8b | 💔 | 💔 | 💔 | 💔 |
| llama3-groq-tool-use:8b | ✅ | ✅ | ❌ | ✅ |
| mistral:7b | 💔 | 💔 | 💔 | 💔 |
| nemotron-mini:4b | ✅ | ❌ | ✅ | ☑️ |
| qwen2.5-coder:7b | 💔 | 💔 | 💔 | 💔 |
| qwen3:0.6b | ✅ | ☑️ | ❌ | ✅ |
| smollm2:1.7b | 💔 | 💔 | ✅ | 😵💫 |
aya-expanse:8b
command-r7b-arabic:7b
granite3-dense:2b
granite3-dense:8b
granite3-moe:1b
granite3-moe:3b
granite3.1-dense:2b
granite3.1-dense:8b
granite3.1-moe:1b
granite3.1-moe:3b
granite3.2:2b
granite3.2:8b
hermes3:3b
llama3.1:8b
mixtral:8x7b
phi4-mini:3.8b
qwen2.5-coder:0.5b
qwen2.5-coder:1.5b
qwen2.5-coder:3b
qwen2.5:0.5b
qwen2.5:1.5b
qwen2.5:3b
qwen2.5:7b
qwen2:0.5b
qwen2:1.5b
qwen2:7b
Hituzip/gemma3-tools:4b
PetrosStav/gemma3-tools:4b
hhao/qwen2.5-coder-tools:0.5b
hhao/qwen2.5-coder-tools:1.5b
hhao/qwen2.5-coder-tools:3b
hhao/qwen2.5-coder-tools:7b
ishumilin/deepseek-r1-coder-tools-tuned:1.5b
ishumilin/deepseek-r1-coder-tools-tuned:7b
ishumilin/deepseek-r1-coder-tools-tuned:8b
ishumilin/deepseek-r1-coder-tools:1.5b
ishumilin/deepseek-r1-coder-tools:7b
ishumilin/deepseek-r1-coder-tools:8b
maryasov/qwen2.5-coder-cline:7b
orieg/gemma3-tools:1b
orieg/gemma3-tools:4b
tom_himanen/deepseek-r1-roo-cline-tools:1.5b
tom_himanen/deepseek-r1-roo-cline-tools:7b
tom_himanen/deepseek-r1-roo-cline-tools:8b
Tested with attogram/llm-council
| Small Model | Peace | Quit | Topic |
|---|---|---|---|
| bakllava:7b | ❌ | ||
| codellama:7b | ✅ | ||
| deepcoder:1.5b | ❌ | ||
| deepseek-r1:1.5b | ✅ | ❌ | |
| deepseek-r1:8b | ✅ | ||
| dolphin-mistral:7b | ✅ | ||
| dolphin3:8b | ✅ | ✅ | |
| gemma3:1b | ✅ | ||
| gemma3:4b | ✅ | ✅ | ✅ |
| gemma:2b | ✅ | ||
| granite3.2-vision:2b | ❌ | ||
| granite3.3:2b | ✅ | ✅ | ❌ |
| huihui_ai/baronllm-abliterated:8b | ✅ | ||
| llama3-groq-tool-use:8b | ✅ | ||
| llama3.2:1b | ❌ | ||
| llava-llama3:8b | ✅ | ||
| llava-phi3:3.8b | ✅ | ||
| llava:7b | ✅ | ||
| minicpm-v:8b | ✅ | ✅ | |
| mistral:7b | ✅ | ✅ | |
| qwen2.5-coder:7b | ✅ | ||
| qwen2.5vl:3b | ✅ | ||
| qwen2.5vl:7b | ✅ | ||
| qwen3:1.7b | ✅ | ||
| qwen3:8b | ✅ | ✅ | |
| smollm2:135m | ❌ | ||
| smollm2:360m | ❌ | ||
| smollm2:1.7b | ❌ | ||
| stable-code:3b | ❌ | ||
| starcoder:7b | ❌ | ||
| Small Model | Peace | Quit | Topic |
Tested with attogram/ollama-multirun
| Small Model | Socrates | Strawberry |
|---|---|---|
| bakllava:7b | ✅ | ❌ |
| ❌ | ❌ | |
| deepcoder:1.5b | ✅ | ✅ |
| deepseek-r1:1.5b | ✅ | ✅ |
| deepseek-r1:8b | ✅ | 🕒 |
| dolphin-mistral:7b | ✅ | ❌ |
| dolphin3:8b | ✅ | ❌ |
| gemma3:1b | ✅ | ✅ |
| gemma3:4b | ✅ | ✅ |
| gemma:2b | ✅ | ✅ |
| granite3.2-vision:2b | ✅ | ✅ |
| granite3.3:2b | ✅ | ✅ |
| huihui_ai/baronllm-abliterated:8b | ✅ | ❌ |
| llama3-groq-tool-use:8b | ✅ | ✅ |
| ❌ | ❌ | |
| llava-llama3:8b | ✅ | ❌ |
| llava-phi3:3.8b | ✅ | ❌ |
| llava:7b | ✅ | ❌ |
| minicpm-v:8b | ✅ | ❌ |
| mistral:7b | ✅ | ❌ |
| qwen2.5-coder:7b | ✅ | ❌ |
| qwen2.5vl:3b | ✅ | ✅ |
| qwen2.5vl:7b | ✅ | ❌ |
| qwen3:1.7b | ✅ | ✅ |
| qwen3:8b | ✅ | ✅ |
| smollm2:135m | ☑️ | ❌ |
| smollm2:360m | ✅ | ❌ |
| smollm2:1.7b | ✅ | ❌ |
| ❌ | ❌ | |
| ❌ | ❌ | |
| Small Model | Socrates | Strawberry |
Tested with attogram/ollama-multirun
| Passing | Describe | Keywords | Faces |
|---|---|---|---|
| bakllava:7b | ✅ | ❌ | ❌ |
| granite3.2-vision:2b | ✅ | ✅ | ❌ |
| minicpm-v:8b | ✅ | ✅ | ☑️ |
| qwen2.5vl:3b | ✅ | ✅ | ❌ |
| qwen2.5vl:7b | ✅ | ✅ | ✅ |
| gemma3:4b | ✅ | ✅ | ✅ |
| llava:7b | ✅ | ✅ | ✅ |
| llava-llama3:8b | ✅ | ☑️ | ☑️ |
| llava-phi3:3.8b | ✅ | ☑️ | ☑️ |
| moondream:1.8b | ✅ | ❌ | ❌ |
Tested with attogram/ollama-multirun
| Passing | OCR |
|---|---|
| granite3.2-vision:2b | ☑️ |
| minicpm-v:8b | ☑️ |
| qwen2.5vl:3b | ✅ |
| qwen2.5vl:7b | ✅ |
| Failing | OCR |
|---|---|
| ❌ | |
| gemma3:4b | 🕒 |
| ❌ | |
| ❌ | |
| ❌ | |
| ❌ |
| Small Model | bash-1 | js-1 | php-1 | python-1 |
|---|---|---|---|---|
| bakllava:7b | ❌ | ☑️ | ||
| codellama:7b | ✅ ️ | ✅ | ||
| deepcoder:1.5b | ❌ | ✅ | ||
| deepseek-r1:1.5b | ❌ | ✅ | ||
| deepseek-r1:8b | 🕒 | 🕒 | ||
| dolphin-mistral:7b | ✅ | ✅ | ||
| dolphin3:8b | ✅ | ✅ | ||
| gemma3:1b | ☑️ | ☑️ | ||
| gemma3:4b | ☑️ | ✅ | ||
| gemma:2b | ✅ | ❌ | ||
| granite3.2-vision:2b | ✅ | ✅ | ||
| granite3.3:2b | ☑️ | ✅ | ||
| huihui_ai/baronllm-abliterated:8b | ✅ | ✅ | ||
| llama3-groq-tool-use:8b | ✅ | ✅ | ||
| llama3.2:1b | ☑️ | ☑️ | ||
| llava-llama3:8b | ☑️ | ✅ | ||
| llava-phi3:3.8b | ✅ | ✅ | ||
| llava:7b | ✅ | ✅ | ||
| minicpm-v:8b | ❌ | ✅ | ||
| mistral:7b | ☑️ | ✅ | ||
| qwen2.5-coder:7b | ✅ | ✅ | ||
| qwen2.5vl:3b | ☑️ | ✅ | ||
| qwen2.5vl:7b | ✅ | ✅ | ||
| qwen3:1.7b | ✅ | ✅ | ||
| qwen3:8b | ✅ | ✅ | ||
| ❌ | ❌ | |||
| smollm2:360m | ☑️ | ☑️ | ||
| smollm2:1.7b | ✅ | ☑️ | ||
| stable-code:3b | ☑️ | ✅ | ||
| starcoder:7b | ☑️ | ❌ | ||
| Small Model | bash-1 | js-1 | php-1 | python-1 |
| Project | About |
|---|---|
| Attogram Project Discord Channel |
Join the Attogram Project Discord Channel for: - Announcements - Technical Support - General Chat about Attogram Projects |
| Ollama Multirun | Run a prompt against all, or some, of your models running on Ollama. - Creates web pages with the output, performance statistics and model info. - All in a single Bash shell script. |
| Ollama Bash Lib | A Bash Library to interact with Ollama |
| Ollama Bash Toolshed | Chat with tool calling models. - Sample tools included. - Add new tools to your shed with ease. - Runs on Ollama. - All via Bash shell scripts. |
| LLM Council | Start a chat room between all, or some, of your models running on Ollama. - All in a single Bash shell script. |
| Small Models | Comparison of small open source LLMs - 8b parameters or less |
| AI Test Zone | AI Testing reports - hosted on https://attogram.github.io/ai_test_zone/ |