What version the intelligence vs cost graph is using? they didn't ran v4.2 to all models.

It's very different than it was earlier today, so pretty sure it's v4.2.

I’ve been getting ~10 tokens/second on OpenRouter today. And a bucnh of 429s. Same result across several models and several providers, so it doesn’t look like a single bad endpoint.

Is anyone else seeing the same?

I really don't understand the use case here. Who is going to use this? Do they really think developers will use it? LMAO

Still behind Kimi-K3 in almost half of the benchmarks

Much easier & cheaper to run than Kimi

It's useless. Tried with OpenCode + OpenRouter and it couldn't complete an simple task. It stuck using grep/search tools. I think Muse Spark was so heavily RL'd on the Meta harness that it make it useless or very token inneficient to use in other harness like Opencode.

I really liked the contributor model in their harness. I suspect that another harness would fail hard like you're seeing.

is this their way to push their harness?

people interested in the discounted -contributor model would increase their harness beta testers. But it'd be a terrible strategy to capture the better paying customer base through openrouter and similar...

I really don't know how folks accept such vendor lock-in. Imagine using Cloudflare Workers as your main tech stack. This is crazy.

What the difference of doing this vs semantic search over indexed fact chunks? This is RAG right?

Yes, it uses modern technology to tackle an old problem: retrieval.

How you deal with LSP? One of the best things in OpenCode is LSP integration.

> How you deal with LSP? One of the best things in OpenCode is LSP integration.

I would love to learn how LSP helps you. I did various basic evals with LSP and I did not find it to improve the agent performance at all. Maybe on newer models this gets better because they are now RLing with LSP in the context, but at least in the past having a commit hook that runs lints and typecheck at the end, is more token efficient than having every edit be injected extra LSP results. You are almost guaranteed to be wasting tokens since partial edits are unlikely going to result in type checks passing.

(Disclaimer: I work on Pi)

lsp integration isn't for checks like that.

It's for using lsp to do things like symbol renames. It's cheaper to call an LSP server to do those kind of simple refactors than to spend tokens on the model doing it.

It can also use LSP to build context without having to grep around a ton. You don't have to grep and hten pull lines and make guesses for a function. You just ask the LSP for it.

How helpful all that is is heavily dependent on the quality of the language server so usefulness is varies wildly between languages

> It's for using lsp to do things like symbol renames. It's cheaper to call an LSP server to do those kind of simple refactors than to spend tokens on the model doing it.

Is OpenCode pushing the LLM into doing these refactors via LSP? I found that making these refactors with ast-grep is quite efficient.

> It can also use LSP to build context without having to grep around a ton. You don't have to grep and hten pull lines and make guesses for a function. You just ask the LSP for it.

I guess I was not particularly successful with having the agent actually do that. I think what fff is attempting to do (with fuzzy expansion of slightly inaccurate greps) results in better results to me, but even that is debatable.

To be fair, we do not have evals for this today, but we're generally looking into token efficiency and how likely models are at using tools the right way, and we did not find a ton of evidence of LSP helping, even for finding data.

But I would love to see some sessions from people where they have success with LSP data for either refactoring or looking up, because this would be super useful to better understand out blind spots.

To be fair in every harness I've ever used that has LSP support, they never actually utilize the LSP for more deterministic refactoring tools. And even then when I do enable the LSP in many harnesses oftentimes it doesn't even use it at all.

Maybe they haven't been taught to do so or it's not integrated into the system prompt or the tools but all of them only ever use the LSP to read files/symbols.

Every harness I've used will happily just call the edit tool over and over or do a find and replace via sed or programatically call a python/perl script rather than rename a symbol via other means.

This is partially a "how much does the model follow instructions" thing. I use Pi with a vibed LSP extension and Claude (4.6 or thereabouts) almost never followed instructions to use LSP renaming tools - despite it being strongly emphasized in system prompt and agents.md. However I found Codex 5.3 would use them sometimes, and GPT 5.4/5.5 would prefer them.

LSP helps massively with some models.

It sounds a lot like you are focused on using Pi as a OpenAI/Anthropic wrapper when that isn’t your audience. (Hint: raw token usage is less of a concern).

Oh My Pi has it. I'm a big OMP shill right now. Seems not very popular, but it has the stability of Pi with the features Opencode (and more I think; OMP has web browsing and a more advanced edit system too). OMP often outperforms Claude Code and Codex for me.

I haven't tried it myself but there seems to be an extension for this: https://pi.dev/packages/pi-lsp

I disliked OpenCode's LSP integration, as Pyright kept distracting the model from what I wanted done in a Django project. I prefer to run MyPy at my own convenience

I use a forked version of the pi-lsp. It works pretty well out of the box, but I just tend to fork extensions by default and then hack them up how i see fit.

Meta is no different. I know a company that had their OAuth app on Meta rendered completely unusable just because one of their employees (a dev) had their personal Facebook account banned by Meta for no reason. They tried to escalate it multiple times but got nowhere, lol. Meta is even worse because accounts need to be 'personal'; if you have a Business Manager, the users added to it are all tied to their personal Meta/Facebook accounts. This is ludicrous.

To me, building any business with dependencies on Meta is just a bad business plan.

Yeah, people loose their business because a kid is logged in on their iPad, gets their google account suspended, and google knows it's the same household as the parent, and everything gets shut down

Can't find this now but google did at least once disable company's accounts after dev got their account suspended.

And as we know from the recent Gemini ban wave, you can get suspended just because.

Everyone needs a defensible root of trust, this goes all the way down to the registrar you use for your domain.

Are you honestly saying that a kid should not use their parent's email address as a recovery option? Seems like that would be the natural way to do it.

Point is that if one account gets suspended, all your accounts might. Your kids', the family account, your separate one that you use for gcp billing etc

It’s almost impossible not to any more. This is victim blaming at this point.

Meta and Google B2B are both horrible. Their ad account bans are constant, and they have no real escalation process to get help. These companies are monopolies that should treat businesses more seriously, especially in these situations.

Review is done by LLMs? How you guys decided to deal with prompt injection attacks?

It isn't. Doesn't involve AI. Read the post :)

I read. Where in the post says that Automated Reviews are not using AI / LLMs? What automated review means them?

It doesn't say "we don't use AI" but I guess the assumption nowadays is everything uses AI? In my opinion the burden should be to state that something does use LLMs, not that it doesn't.

The post has instructions to reproduce the review results using our open source eslint plugin:

https://github.com/obsidianmd/eslint-plugin