The only part that would really be novel is the liability.
I would be shocked if you could publish an iOS app without Apple being able to tell the government who you are. Less because Apple cares and more because Apple requires you to pay, which is very hard to do anonymously for something like this (I’d bet the options they offer are effectively “credit card only”).
It also relies on lifelong tenure, which would be a huge cultural shift for both employers and employees. Employees are often used to quitting rather than having to fix business issues (easier to quit than make the business fix X), and businesses are largely used to mistreating their employees because they can be replaced.
We’re culturally very far from even being able to attempt that.
Naively, I think some optimizations would require access to the whole codebase and that would make people nervous (plus incur more cost).
Eg absurd idea, but you could write something that minifies a codebase (by token, rather than byte) and then translates edits back into the expanded code. Probably an insane use of fuse lol. Partially minifying on each tool call sounds like a huge pain with a lot of state to track.
There’s also a lot of common situations where humans prefer solutions that take more tokens because it’s easier for us to read (eg for loop vs map vs list comprehension), which may have some gains.
I strongly suspect there is some form of token compression that works, but I don’t think it will be as simple as “pipe arbitrary text with no context into this tool”.
Jetbrains feels like a place this might come from. “Take this code, parse it to an AST, find the fewest token representation of it” feels like something they’d do, or maybe Astral (specifically in Python land, type checkers feel sort of adjacent as well).
I fully agree. Some of these tautologies, RAGs or RAG adjacent ideas feel like there is real potential. I know in my own testing adding a knowledge graph provided cost saving benefits.
Ultimately, the problem is people aren't actually benchmarking any of these ideas or if they are its via an extremely misleading methodology. For example, headroom violates cache frequently, nullifying any potential savings it could have; but does their benchmarks show that? Nope. [1]
As an aside, I wonder how many days are we away from Codex or Claude taking all the "read" tool calls and re-assembling a full map of your codebase and then offering it as a feature.
I think people don’t do genuine benchmarks because the market forces them to pretend their solution works for anything you can throw AI at. Companies whose valuation is based on them being the RAG/compression/routing/etc company. They can’t admit it only works well in a specific domain because then they’re immediately $300M in the hole.
I have more faith in companies with a more targeted approach. Eg gzip does fine, but video codecs beat compressing raw video by a ton.
> As an aside, I wonder how many days are we away from Codex or Claude
That sounds like SourceGraph but twice as expensive, although it does have “AI” so probably lol
I would guess they’ll spend ages arguing about what an “ad” is.
Eg if I ask Llama 3 to find me the best headphones, I think we can agree its response isn’t an ad.
If Claude responds with a banner that Sennheiser designed and paid Claude to show, that’s an ad.
They’ll live in the probabilistic grey area between those. If Sennheiser pays Anthropic to add “Sennheiser makes really good headphones” to the system prompt, is it an ad if Claude recommends Sennheiser headphones? What if they pay to ensure their site is included in the web search tool results for any queries about headphones?
It’s sort of akin to paying Google to rank higher in searches if that were a thing they did.
I do think they should have to declare those, but they’re not traditional advertising to me.
I’ve been looking along a similar line, but I came at it from the infra side rather than the software.
Mine pretends to be a local ACP agent, but it’s actually managing a Docker container and proxying the ACP connection into the container over websocket. You can specify a bunch of utility stuff in the YAML definition like directories to bind-mount, directories to copy from the host, scripts to run when the container starts, etc. You can also toggle whether ACP-native tools like read/write file and shells execute on your host or in the container (in container by default).
Works well, I forget whether I’m on that or pi-ACP directly until it spits out a time in the wrong timezone or I forget that it can’t check my DNS settings or something.
Really cuts down on the damage it can do. Mine is basically down to “it can delete my ~/.pi and the repo it’s working on” and that’s about it unless it can escape the container.
There’s a big gap between hobby self-hosting and enterprise.
Clouds are appealing (note, not technically better but more appealing) in the enterprise for numerous reasons. We already have a contract, I don’t have to spend a month with procurement. Their support is “respected” so “I asked AWS and they told me to pound sand” is an acceptable response to a lot of requests. No one ever got fired for picking AWS. The list goes on.
None of those friction points apply to hobby hosting, really. IAM is annoying because the alternative isn’t “fill out 7 forms and host 12 meetings to get a new vendor in”.
You also get less of the benefits. Low volume SQS can be trivially replaced, but if you use enough that you’re debating making a whole team to manage Kafka then paying the cloud tax can look appealing because of the predictability (your Kafka team might fail, SQS probably handles larger workloads currently).
At some scale, not having to wait 12 months for the message queuing team to support TLS for some regulation becomes worth a cloud tax.
I've worked on both sides and I really think it comes down to Dunbar's number [1] and thus the size of the company.
At a sufficiently small company, you can just trust people will generally do the right thing. I ask X to accomplish some compliance goal, I just trust that they'll do it right.
As the company scales, you actually know a smaller and smaller portion of the people you work with. You no longer just trust that they'll do the right thing, because you don't know them. You fix the lack of trust by adding processes that ensure the "right thing" happens, but each of those processes is a friction point. Eventually you have so many of those processes that they become the majority of the effort for a project.
That gets exacerbated by the usual turnover. People pissed off by all the processes they have to comply with will leave, and people who love having their own fiefdom will stay and double down.
That all ends in the evergreen stupidity of "spend $1,800 to have 12 people who make $150/hr argue about whether this app really needs a $5/month Aurora database".
That's a personal pet peeve, but is emblematic of the issue imo. Trust has broken down so severely that the company is willing to spend more than the lifetime cost of compute for the app to vet whether it's "necessary".
A similar vein is the "platform team" where all they do is take an open source project and wrap it in a custom DSL, so you get all the complexity and none of the searchability of the original project, plus the features are almost always a subset. I can't tell you how many times I've looked for solutions to a problem, found a snippet that will fix it, and then had to reverse engineer how to make a stupid DSL output that config.
The utility is always the same. The stock market provides price discovery, which drives efficient resource allocation (in theory).
Theoretically, more gamblers should mean better price discovery because the payouts for correctly taking the opposing side of the trade are higher.
The market solution would be that the gambling will eventually solve itself. They’ll either learn enough to be trading on knowledge rather than vibes, fueling price discovery, or they’ll exit the market when they’ve lost too much or everything.
My sticking point is that a lot of brokers offer leverage to people they really shouldn’t. I could have sworn you had to be a qualified investor to get leverage, but if that isn’t law it should be. Show the brokerage your certification, or a pile of cash large enough to convince them you can afford to lose the whole thing.
I don't know if "theoretically" is good enough here. The theory hasn't really been tested or proven. It's more of a hypothesis, and an ideologically driven one at that.
I'm not positing that as a moral good or bad. It's possible for stock market gambling to be bad for society but have better price discovery, in the same way that dictatorships are bad but tend to have faster response times to events.
My theory is mostly that "gambling" seems like it inherently means "buying stocks based on something other than their concrete value". More gambling means stocks drift further from their "true value", which means a higher payout for correcting them back to what their price should actually be.
The whole thing does get very fuzzy because of the "market can stay irrational longer than you can stay solvent" aspect. It's not enough to know what the correct price is, you have to know when other people will realize that as well, or else convince people that your price is "correct".
Acetaminophen overdose does actually have a surprisingly high rate of occurrence. A lot of people don’t realize how narrow the therapeutic band is.
Doubling your meds on a bad pain day can put you way beyond the safe limits. People think it’s safe because basically every other OTC has a huge therapeutic band, and double dosing is not recommended but not really dangerous.
CDC estimates it at 56,000 ER visits a year, 26,000 hospitalizations, 458 deaths, about a hundred unintentional deaths per year. As a point of reference, it’s about 3 accidental overdose deaths per child that dies from being locked in a hot car.
When I messed with it I used Kagi's search and I didn't have that issue (not claiming they're the best, they're the only one I tried).
They filter their results through their AI, though, so you get a sort of meta-summary of the top few results. It did well with geopolitical news stuff, but I've not tried a hard science sort of query.
I would be shocked if you could publish an iOS app without Apple being able to tell the government who you are. Less because Apple cares and more because Apple requires you to pay, which is very hard to do anonymously for something like this (I’d bet the options they offer are effectively “credit card only”).
reply