7-8 figures annual spend will buy a hell of a lot of capable local inference hardware you can own, though it won't be at the absurd token/s rate, you'll be able to run almost anything on it... And it'll still have a good residual resale value after 4 years the way things are going now.
Feel like you could spend 6 figures building out a team and the rest renting compute for a whole year, and get the team to create a local inference solution with that kind of budget…
Lighting my codebase on fire at the speed of light. Like microwaving the spaghetti.
I genuinly only see these speeds being useful for customer service/transactional workflows. Of which much smaller models can do the job (but those dont make tons of money for companies like Cerebras that need to pay off massive amounts of debt).
Nobody needs to code at 600 words per second. Using a 100tps model for an hour or so will leave you with 4-8hrs of code review and revision work.
Human code review? What is this, 2025? The modality today is write with one LLM, review by a different one, (important: two different model families will catch errors one series won't) then deploy right to production.
I would be very curious to see how you explain it to your customers.
Is it going to sound similar to this?
> You see, our well-meaning AI-generated code has caused all your data to be permanently deleted. In case you are confused as to who to blame, we would like to clarify that we did not write, nor review the code. So we cannot possibly bear any responsibility for its mistakes. The responsibility lies with the LLMs, not us. We have already fired the LLM which did the coding and the review. And we are already using their main competitors. Hopefully that settles your concern with the quality of our service and we are looking to have you on board of our next products.
This seems to work well enough for people who deploy cloud instances without redundancy to us-east-1 then blame AWS when there's an outage. I say this somewhat unironically because if one pushes the "move fast and break things" slider all the way to the right then they're necessarily assuming that type of risk. Of course some people will try to have their cake and eat it too [1] as regards velocity and quality but that's a separate discussion.
[1] I've never understood this idiom because if one isn't in possession of their cake before eating it then they're eating stolen cake which is a decidedly anti-social activity and orthogonal to the point of the saying. It should be "eat their cake and keep it too" or something.
Yeah, idk. I work on serious things. Thats not how I do things. Not everything is webdev hobby projects. There's basically no instance where offloading your code review to an llm is acceptable behavior, except maybe for a one off tool you need personally.
And if you tell the reviewer the author is a competitors model it becomes extra snarky and vigilant. Then give the review results to the author and tell it it's from the competition and it will also become slightly outraged.
No you dont need to code at 600 w/s BUT at those speeds, you can start doing things like asking multiple different agents the same question and picking the best solution each time without noticing the lag.
I prefer a fast model too, but you cannot get more done just because its faster. You just get to the human parts a bit faster. Code review, revision ect.
Inco sucks. I tried their GLM 5.3 Flash and it was quantized to the point of hallucinating Chinese in the middle of English only agentic sessions. Never happened with any other provider.
Please do not try to use gpt-oss-120b over Cerebras. It is broken, screws up tool calls most of the time, forgets to end thinking blocks and has all sorts of other issues. The speed is amazing but it is absolutely not worth it, especially at that quite incredible cost. Think: $5–10/minute levels of cost with a single agent, because Cerebras also offers no cache pricing for input tokens at all.
I kept having experiences with gpt-oss-120b on Cerebras where it would get stuck in a thinking block and then start endlessly saying things like "Running the command now." or "Making the changes now." and then simply repeating similar sentences like that forever instead of actually making the tool call. It made tool calls other times, so it wasn't an issue with tool calls being impossible, but it just wasn't doing a good job of using them for real instead of simply saying it would. So this was not an issue of it starting a tool call and then putting invalid syntax inside of it, it just would not make the tool call it was supposed to whatsoever. There's no automatic way to retry that.
> There is no additional fee for using prompt caching. Input tokens, whether served from the cache or processed fresh, are billed at the standard input token rate for the respective model.
So what he’s saying is correct, there is no separate cache pricing, which by normal standards should be 10% of the cost, which can become exceedingly expensive for anything other than single turn. The way they are stating this is of course strange..
Yea i had some pretty meh results using gpt-oss-120b it in my evals where it should have benefited speed alot but it really under performed what i was expecting.
Every day 2 major organizations get hacked, whether by groups or state actors, and America continues to sit on its hands.
The government should be creating a new digital defense department to better defend our country, and fund the defense of our nation, but instead it is busy renaming lakes and renaming "AI".
This is simply untrue, and honestly anyone who thinks our legal system does not provide justice needs to go travel the world to see how it really works for 80% of the global population.
All we really need in the US is for arbitration to be optional, you can start with arbitration but if you believe on good merits you have a case and fail arbitration you should have the right to your day in court.
Throwing out our whole legal system because sometimes the lawyers make the most money is reductionist at best.
I believe what he's referring to is the cost of the legal system. Lawyers cost money, and if the other party has a lot more money, they can play all kinds of tactics to keep the case going (filing pointless motions, drowning your lawyer with huge amounts of documents, etc).
With arbitration, the arbiter can put an end to those shenanigans really quickly, because they're not legally obligated to allow for that kind of crap.
There are ups and downs to both systems. The benefits are there for parties of roughly equal size. The down side is there is no recourse if the arbiter makes a completely unfair judgement (e.g. because he thinks some law should not apply, etc).
> All we really need in the US is for arbitration to be optional, you can start with arbitration but if you believe on good merits you have a case and fail arbitration you should have the right to your day in court.
That just makes it the worst of both worlds! The whole point of arbitration is not to have a war of attrition, and to make the process efficient. Your solution doesn't take care of it. If I'm a business getting into a contract with another business, knowing that you can't outspend me in court is important, and lets both of us make a deal. If I know you still want the option to go to court, it's as if arbitration wasn't on the table.
A better system would be to change the law such that for certain, predefined offenses, you can't waive your right to go through the legal system (e.g. sexual harassment).
And of course, both parties should have say on who the arbiter is.
Finally, you are talking to someone who's had experience with arbitration. You might want to tone things down in your rhetoric if you don't have your own experience to bring to the table. Things might sound fine in theory, but can be really, really bad in practice. As an example, on How I Built This, whenever he's interviewing a founder that went to court to enforce a patent they owned, he asks if it was worth it. The answer almost always is "It was a bad idea and we shouldn't have filed a case." This is coming from those who won in court.
> This is simply untrue, and honestly anyone who thinks our legal system does not provide justice needs to go travel the world to see how it really works for 80% of the global population.
That's weird. Two things can be true at once: arbitrary is better than the American legal system, and the American legal system is better than what 80% of the global population live under.
> All we really need in the US is for arbitration to be optional, [...]
It is! You explicitly have to opt in, that's why Uber needs have it in their ToS that you accept. They can't just force it on you, when you have no contractual relationship with them.
I've used it on a few for fun projects and its decent but the speed is crazy to watch.
reply