Why is this not criminal?

As other have mentioned, the code isn't the valuable part. But also there are alternatives now to these LLM providers.

They should have voted on if pineapple deserves to be on pizza while they were at it. The guy had to go.

It's been fun to see people re-writing and flashing their own smartwatch software with LLMs. Looking forward to the first TV OS rewrite video.

WiFi / 5G faraday cages exist, but they have been downrated for affecting the signal quality somewhat...

> Running a large model locally comes down to one thing: how much RAM it actually needs in memory.

Not completely true. It's memory AND memory bandwidth. You can have 1tb of memory but if you have awful memory-bandwidth you'll also have slow tok/s. A3B helps with this, but so does MTP.

From my experience, you'd be better off running the dense 27b-mlx with MTP than the 3.6 version with A3B. You say your model is ~20GB of ram, but the 3.8:27b-mlx is 18GB and gets me very reasonable tok/s, and greater speed if you disable thinking when not required.

Running it depends on RAM, which is what I wrote, bandwidth is important for speed. I chose my words carefully, but you are absolutely right.

Coming back to this a few hours later, I've decided to add a section to explain this to the blog post. Thank you for flagging it.

The dense 27b Qwen on M4 Pro has a prompt processing speed of around 125tok/s which makes it ok to ask a quick question but impossible to use in an agent, as processing the first prompt of the agent with the tools and instruction can easily be 10 000 tokens

In this case the 35b a3b makes sense as it has a PP speed of around 800tok/s

True. This then boils down to a quality vs speed decision. the 3.8 27b is far better than 3.6 A3B from my experience. I'm happy taking the speed hit, given local models aren't as intelligent as frontier models. Anything that can get me closer to my CC experience both in reasonable speed and intelligence is worth it. With that said CC can also be slow at times, so it's locally the difference in experience is not always noticeable.

Qwen3.8:27b-mlx on 64GB MBP M4, I can get up to 42tok/s, more often than not in the ~30 range.

I like to think Stripe put $50B in a paypal account, and they got locked out with no way to get it back.

Haha, classic, sounds almost plausible, as anyone using PayPal painfully knows.

There are many topics, personalities and politicians we hear about daily who have no merit.

Qwen's advances do (currently) have merit.