Okay but I’ve spent 8 years implementing bad code so far. I’ve got a lot of understanding but only half the code is done. Down the line I’ll be dead. Why not implement it now with AI?
Moreover is writing code the only way to understand it?
What is a soul, how do you measure and quantify it? If you can’t, how would you know whether the grammar mistakes change the soul? I’ve heard people say they keep the grammar mistakes in to prove the existence of a soul, so presumably correcting them would alter the soul in some measurable way.
>>Okay but why do you feel 3 trials say as much as 10,000?
I don't trust them so I've used 3 examples in incorrect questions/answers they have given and I got correct answers. I spend enough time with LLMs to know that if Grok answered it correctly and in detail then it wouldn't be a problem for GPT or Claude either.
The questions are also constructed in a way that it's easy to answer not fully (which they qualify as wrong). LLMs still answer them correctly and in detail though.
Nothing is solved in isolation but credit usually goes to wherever the new work in the paper comes from instead of the whole mountain of previous mathematics or existing tools used. The most relevant of those get referenced and then this reference tree builds a tree of collective base work needed across history.
Usually there has never been a tool which performed the part relevant to getting any credit.
E.g. in the first famous computer assisted proof (of the four color theorem) the computer only executed the resulting calculations defined from the new logic, it did not have part in the work needed to show those calculations could answer the problem nor did it come up with the actual calculations to do.
The V pitch was a bunch of features that seemed implausible like automatic C++ translation and near-GC ergonomics with near-manual-memory behavior, without Rust-style explicit lifetime annotations or pervasive reference counting. so basically the pitch was the holy grail of systems, which caused drama as the project was met with extreme skepticism.
That was 7 years ago and since then the project has not delivered the grail, instead settling to be more of a mashup between go and C.
Bend is completely different it was just announced last week. It doesn’t have any users and has a weird ai integration so I done see how it’s relevant as a contender in the systems space at all.
I think you’re right about people, but by volume, 99% of all future code will be written by machines. So there’s ample opportunity for other languages to flourish; if humans can’t get past imperative programming, machines can.
In my mind this hinges on whether LLMs become capable of actual comprehension of the code they’re writing. If not, for serious projects (especially those which are mission-critical), it still makes sense to optimize for human understanding when selecting languages so the code can reasonably be verified.
Exactly. Most restaurants you go to don't care about the food they serve you nor do they care about the products they use, as long as it doesn't harm their business. Same with groceries - manufacturers don't care, as long as what they sell you is acceptable and passes regulations. But once you go to a restaurant with standards, you can immediately tell the difference. I do not come from a wealthy family and even as such, I certainly prefer paying the higher price now that I can afford it.
When you’re required to approve thousands of lines a day (code you can’t possible understand), it certainly IS causing issues that didn’t exist before.
Every study I’ve seen correlates the use of AI with large increases in the number of bugs. Look at Amazon dialing back AI after massive outages. Microsoft patch Tuesday releases are bricking computers (they even managed to break notepad somehow). The rash of Facebook bugs also coincided with their move to AI. Leaks from Google have engineers saying AI either doesn’t save any time because it takes so to remote stuff or it causes breakages if they speed up.
These companies can afford to get the best devs. They have access to essentially unlimited token budgets. They have STILL fallen off a cliff in quality.
What more proof could there be that this isn’t sustainable?
A correlation-causation link between AI use and these bugs has not been established. Until it has proven to come from the region of AI-pagne, it's just sparkling enshittification.
This isn’t the human body or some other thing with billions of unknown parameters. It’s a single (relatively simple) math equation that you are ascribing tons of non-existent complexity to.
Left to its own devices, that system will result in AI autophagy (model collapse) and iterative degradation.
The only area with serious uncertainty is how humans interact, but we now have research showing humans suffer cognitive issues very quickly using AI (some studies indicate effects happen in as little as 10 minutes) with cognitive surrender being a particularly big issue.
In a lot of systems, the only new data seems to be a few brainstorming sentences (you can read slop as entropy decaying things). The AI slops that into requirements. That slop feeds into an agent which generates a bunch of “reasoning” slop, maybe compacts everything (more slop), and spins up agents that get handed slop. They then open files with who knows how many generations of slop (maybe never even touched by a human) and write out a bunch more slop (it’s ironic that humans get better the more they edit a file, but AI gets worse). That slop gets “tested” by another agent reading all the other slop and maybe all this recurses a few generations.
At the end of this AI equivalent to “the human centipede”, you get a developer who’s handed 10x or maybe even 100x more code than their brain would possible process. They are suffering complete cognitive surrender (not to mention often reaching mental and maybe physical collapse from the workload and stress). They don’t understand the system and rubber stamp it so they can move on to the next 50 PRs of the day.
From start to finish, it’s 100% entropy outside a handful of lines worth of human input.
Many people predicted bugs and even discussed entropy issues before AI coding was popular. The buggy mess timing aligns not only with AI adoption, but happens to each company ramping up as they ramp up AI usage.
This is like seeing Einstein’s predictions happen, but arguing he can’t prove correlation/causation. What evidence would you actually accept that is feasible to study?
So you're saying that LLMs let you accrue technical debt faster? I suppose that's like the fact that living on payday loans let you accrue monetary debt faster.
I think if you're on a team that cares about quality, LLMs can help you write quality code faster.
If you're on a team that's mindful about technical debt, you can have make practical trade-offs for velocity now at the expense of paying off technical debt later.
And if you're on a team that's unable to care about code quality ("I gotta merge this code now!"), then you can write mountains more code than you can understand.
reply