You're talking about the HuggingFace incident? It's notable that the agents (somewhat justifiably) thought that their evaluation had an LLM grader which would look for evidence that they had cheated. And it was specifically that grader they were trying to his evidence of cheating from, not humans more generally.

Honestly that part was more surprising to me than anything else, how narrow the compulsion to cheat was: they didn't learn "cheat in general" they learned "think about the grader in great detail and chat exactly as much and exactly in the ways that actually result in a higher score".

Yea agreed they were trying to deceive what they thought was an LLM grader. It’s unclear to me the extent to which cheating behavior is generalized? From what I’ve read there are signs that some amount of cheating has been reinforced in their training due to poor RLVR evaluation setups

"CONERTHTO" is just "CONVERTHTO" but missing a letter, no?

Why should they care if their actions are discovered by humans? What are the humans going to do, discard their multi-million-dollar training run? Even if they do that, the amount of RL pressure is tiny relative to what happens inside an RL run.

Gasoline + LOX beats TNT on energy density per unit mass and per unit volume, no? There's a reason that kerosene+h2o2 is a classic rocket fuel, and TNT is not.

It's usually kerosene + O2, not H2O2. But H2 + O2 is another classic liquid rocket fuel that has higher specific energy (energy per mass), but lower energy density (energy per volume) than kerosene, and is used mostly as a 2nd stage fuel (where mass is most important). Its lower density and the need to keep it very cool mean heavier tanks, and it's also hard to work with due to being such a small molecule that likes to leak, which causes development costs to be higher.

You can treat the LLM's answers ass hypotheses about why it did what it did, and test those hypotheses. The hypotheses the LLM comes up with might be better than the ones you come up with, because the LLM has seen a lot more text than you have, and particularly has seen a lot more of its own outputs than you have (e.g. from training to use other instances of itself as subagents).

I think if you're vibe coding to the extent that you don't even know the shapes of data your system works with (e.g. the schema if you use a database) you might be outsourcing a bit too much of your thinking.

This. When compilers came along, I believe a bunch of junior engineers just gave up utterly on understanding the shape of how the code was generated in assembly which was a mistake given early compilers weren't as effective as they are today. Today vibe-coders are using these early AI tooling and giving up on understanding the shape, and similarly struggling.

Don't.

The first 500 or so tokens are raw thinking output, then the summarizer kicks in for longer thinking traces. Sometimes longer thinking traces leak through, or the summarizer model (i.e. Claude Haiku) refuses to summarize them and includes a direct quote of the passage which it won't summarize. Summarizer prompt can be viewed [here](https://xcancel.com/lilyofashwood/status/2027812323910353105...), among other places.

Would be fairly easy for them to offer an onion service on which they publish the current list of domains, as one option among many, many options for distributing small strings on the internet in an uncensorable way.

the beauty of wikipedia as dns is its easy access, are there similarly easily accessible uncensorable ways?

Ideally it is common knowledge that the onion service exists, and then people can go look at the onion service and update Wikipedia based on what they see there.

Also onion doesn't require a domain register

While true, most people don’t (and won’t ever) know how to use it. But enough will to keep Wikipedia updated I suppose.

Did that happen to a lot of companies during the log4shell fiasco? I'm sure some companies had their permissions misconfigured in a way such that a malicious actor who could execute code on their servers could also drop their database and delete their backups.

I don't know. But the point is that anyone who has access to this model might be able to do the same thing to any company or government.

Claude Code's main advantage is that it's the only TOS-compliant way to access subscription Claude tokens, which cost about 10% as much as pay-as-you-go Claude API tokens.

Well to be honest, even 5% would probably be too much for me (even tho i dont know the pricing of tokens or consumption of tokens)

I would just want to test around a bit locally, maybe let it do its thing over a weekend just to see the result and then stop it again