The state of small local LLMs for coding as of July 2026

TLDR: I spent a whole day playing around with small LLMs which run on my laptop. Here’s what I figured out.

I’ve been reading a lot about people using small local LLMs to automate parts of their life, whether it’s summarizing their emails in the morning or some sort of content creation. However, I’m more interested in using them to write code and and so I downloaded a set of models for local LLM coding and I’ve started systematically evaluating them. The test bed was a bog standard Lenovo Thinkpad T14s G2 with 32 GB of RAM and no dedicated GPU, running Linux. The model runner was Ollama, running them entirely on the CPU.

Speed Test

For the speed test, I kept ’thinking’ off and the results are interesting. The absolute dark horse was qwen3-coder:30b. At an output rate of 12 tps it is fast! It compares in speed to qwen2.5-coder:3b, which is 10x smaller. I noticed that the output is more structured and terse than that of other models, but that is an advantage in a coding focused model IMO.

Gemma4:e4b on the other hand was quite slow for its size and qwen3.5:27b was an absolute dog. It would only be worth running if the output is significantly better than that of qwen3-coder:30b.

So as far as speed goes, we have a clear winner.

Coding Test

Now to test these models as coding agents I had them vibe code a very simple console based program. I then ran the output and examined it for code quality. I deliberately kept the prompt simple and ‘vibey’ to see how the model reacts and what assumptions it makes. Where available, I kept ’thinking’ mode ON this time to give the model the best chance to get things right, even though I don’t think Chain of Thought (CoT) has much value.

Overall, three things stood out to me:

1. General models are not very good at coding at all. Both gemma4:e4b and qwen3.5:27b were unable to produce running code. This matches my previous experience working with such models.
2. qwen2.5-coder:7b was better than it’s 3b variant but the 14b variant was not significantly better, although it was slower. It’s probable that for this family and other similar models, 7-9B params is the sweet spot.
3. qwen3-coder:30b was the clear winner. It was blazing fast and the code was clean and works well. I’ll be using this model on my personal machine for further work.

You can see details in the section titled Coding Test Process

Additional observations

I initially ran the test using Pi.dev as my agent harness rather than raw Ollama, but quickly ran into two issues. One was that Pi only loads models with tool calling support, which Qwn2.5-Coder doesn’t support, so I couldn’t test those models. Force loading those showed me another potential issue; the model was trying to use the web-search tool to get the code for the tic-tac-toe program from the web! Not at all what we wanted. Using ollama run side-steps both issues, but it was interesting to see this none the less.

Speed testing process

Models run with:

• ollama run <model>
• /set verbose
• /set nothink
• Prompt: “what are your capabilities?”

Raw results (grouped by family):

• gemma4:e4b
  • prompt eval rate: 39.50 tokens/s
  • eval rate: 8.50 tokens/s
• qwen2.5-coder:3b
  • prompt eval rate: 83.78 tokens/s
  • eval rate: 14.09 tokens/s
• qwen2.5-coder:7b
  • prompt eval rate: 37.09 tokens/s
  • eval rate: 6.74 tokens/s
• qwen2.5-coder:14b
  • prompt eval rate: 18.38 tokens/s
  • eval rate: 3.32 tokens/s
• qwen3.5:27b
  • prompt eval rate: 7.11 tokens/s
  • eval rate: 1.71 tokens/s
• qwen3-coder:30b
  • prompt eval rate: 37.36 tokens/s
  • eval rate: 12.37 tokens/s

Coding Test Process

Models run with:

• ollama run <model>
• /set verbose
• /set think #if the model supports ’thinking’
• Prompt: “Write a simple console based game of tic tac toe in python.”

Raw results (grouped by family):

• gemma4:e4b
  • Generated code was 140 lines long. It errored out at launch and did not run. Overall: 0/10
  • prompt eval rate: 48.71 tokens/s
  • eval rate: 7.57 tokens/s
• qwen2.5-coder:3b
  • Generated code was only 47 lines long. The display was slightly mis-alined but functional. The user has to specify the 0 indexed x,y co-ordinates to make his move. There was some error checking, but no proper boundary checks. Overall: 5/10
  • prompt eval rate: 81.83 tokens/s
  • eval rate: 12.66 tokens/s
• qwen2.5-coder:7b
  • Generated code was 58 lines long. The model also gave a clear explanation of how to play the game along with some sample outputs. The game output also mentions the 0 indexing requirement when taking input. Overall a better effort. Overall: 6/10
  • prompt eval rate: 17.08 tokens/s
  • eval rate: 3.17 tokens/s
• qwen2.5-coder:14b
  • Generated code was 60 lines long. The model output was quite terse, but the game had better input design, breaking up the users x and y positions into two distinct steps. The board display was still mis-aligned, but legible. Decent enough. Overall: 6/10
  • prompt eval rate: 17.16 tokens/s
  • eval rate: 3.27 tokens/s
• qwen3.5:27b
  • Generated code was 78 lines long. It contained four syntax errors right off the bat and did not run. Overall: 0/10
  • prompt eval rate: 1.64 tokens/s
  • eval rate: 1.66 tokens/s
• qwen3-coder:30b
  • Generated code was 89 lines long. No thinking mode and the token generation speed was impressive. The game also had a straight-forward interface and was the only one to display a well aligned board. Boundary checks were also present. Overall: 9/10
  • prompt eval rate: 42.50 tokens/s
  • eval rate: 10.37 tokens/s