Per mile driven I expect it's true, just given their greater experience than most drivers. Not that I disagree with you that many Uber drivers are not actually great at driving, but neither is the average driver in North America. I expect we just notice rideshare drivers' mistakes more, and get exposed to them at a greater rate.
There's probably some sort of bias here when you observe someone else driving - it's like that joke - if someone drives slower than you - just curl up and die already boomer, and if they are driving faster than you - what an absolute psychopath.
I do agree that Lambert's coverage of Tesla appears to be pretty one-sided, and that the company has provided significant benefits to the world, but I also think there's a reasonable case to be made that Musk himself is a uniquely deleterious figure, even if the harm he brings isn't with his bare hands.
That may be true, but I feel that is a separate conversation (whether Elon Musk and his companies, taken whole, have had net good or net bad contributions to the world) from Lambert's coverage, which seems to pick any single _event_ and select the most heinous, corrupt, version full of malicious intent when often unwarranted, extremely bad faith and never adds informational value to that larger debate.
I can't speak for everything the author has written about Tesla, but I don't see any intent stated or implied in this particular article. Instead it seems to mostly be accurately describing an action taken by Tesla that doesn't cast the company in an especially good light. That's not bad journalism just because some people might like Tesla or Musk and want to give them the benefit of the doubt or downplay anything negative they might do.
It's there via omission. Others have said in these threads that the concept of an HVAC company or others investing in a particular manufacturer, does cost them money, and that can change over time. It's not unique to Tesla, but the writing, by way of omitting this.
"That is the bind Tesla’s certified installers are in: dependent on a brand that just walked away from a product line they built a business around, while still needing to sell its remaining hardware."
there is _nothing_ unique about that situation. That could happen with any company.
It's the cost of doing business. training your workforce, etc. Clearly these companies are happy to sell other Tesla products to the point where they don't want to jeopardize those relationships.
I would bet that llms have talked plenty of people both into and out of suicide at this point.
That nitpick aside, I think that's an excellent list. Especially being able to articulate what it does and doesn't know, or how confident it is. That's something that naively sounds pretty simple, but clearly isn't. And it's something humans aren't great at either (see: Dunning-Kruger), but so far LLMs don't even really have the capability to attempt it.
If the culture shifted such that people were skeptical of original results but excited about replication, replication would be the big news and people would want to do it. How to get there, I don't know.
I seem to remember a lot of hype about room-temperature superconductors, that croaked upon [lack of] replication. That wasn't too long ago.
Before that, cold fusion.
We still have the ion drive that doesn't require fuel. I don't think it has ever been successfully replicated, but I think that it also hasn't been successfully repudiated, either.
This kind of thing has been going on for a long time. I suspect that not all of it is deliberate fraud, but rather, people wanting their preconceived bias to be satisfied.
No, that would never happen. People would just be skeptical of most findings until they see replications (where we are headed in the social sciences, anyway). If people see replications the original discoverer will get all the accolades, not the (several) replications.
Exactly. It’s the funniest thing. These books are only being bought by AI companies. It seems pretty clear they only have value to AI companies. Otherwise all these people bemoaning the loss of rare books would be… buying them.
“How dare these companies buy rare and valuable books that nobody else values enough to buy” is a self-canceling argument.
But the choices are weighted based on those probabilities. This doesn't affect the weightings, only how the final weighted pseudo-random selection is made.
It's not just swapping synonyms. The way llms work is by predicting the likelihood of the next token. It's inherently probabilistic. Choices are made based on weighted random number generation, based on those probabilities. Changing how you generate the random numbers doesn't degrade the output.
1. As watermarked text is added to the training data, watermark-related tokens will be associated more with AI outputs and thus lower quality outputs which will hasten model collapse. Especially because every provider has its own secret key and they are all training on eachother's outputs anyway.
I guess they can at scale filter the watermarked documents (by necessarily allowing eachother to at scale checked for watermarks, but banning the labs not part of the watermarking-cabal). Makes me wonder how useful the human quality filter is on AI output - if a human judges a given output as genuinely good and posts it somewhere for the scrapers to find and take into the training sets, will these types of outputs also be filtered out?
2. (raw, pre-watermarked) Output token probability situations where 1 output token has the majority of the probability mass associated with it, but it is not in the watermarked set, will force the model with much higher probability to walk a non-optimal latent space. E.g., if the next OBVIOUS token for a given sentence would be a point, but the model is in this way not allowed to output it, it might put a comma and start off on a whole different tangent just to make the initial non-optimal comma grammatically make sense.
There are no watermark related tokens, there is no watermarked set - read the paper, it's public and not that complicated. It doesn't change the distribution of completions, and won't change the situation where one output token has the majority of probability mass.
I am going off the explanation in the declaude page (and related papers). But I see now anthropic mentions Aaronson's distortion-free watermarking.
Random watermarking functions colour the tokens based on (small) contexts and a secret key. Given watermarking functions are randomly chosen every single time they are used (so essentially not deterministically seeded by the context and secret key), then indeed the completion distributions are unchanged. However, they are deterministically chosen, a given string of text will always have the same corresponding watermarking functions. Tokens scored high by the function see an increased probability of being the chosen completion, those scored low see a reduced probability. I dont see it is different from merely talking about it as green/red and the points hold?
For a given short-context (hash function seeding) you do have detectable manipulation of the completion (how they can read the marking). But, because (assuming enough entropy in context) the hashing function is decoupled from the log-probs, the perturbations are independent from the underlying distribution, so you're still sampling from the same distribution quotient some noise.
The only way you'd notice this is they weren't independent, and the most plausible way that happens is if you're re-completing pre-fills (resampling the same hash function against the same log-probs).
Ok. I dont have a good feeling for the actual completion distributions. The noise sounds problematic. I can imagine it relates to the size of the context used for hashing. You want this as long as possible so that the entropy is higher, but you also want it as short as possible, because edits invalidate the hashing for all the tokens of which the edited tokens are part of the hashing context.
Anyway, there are lots of cases where text carries very little entropy. E.g. boilerplate code, exchanges of pleasantries, well-worn platitudes and jokes, etc. These are sequences of tokens that will be seen across many, many separate outputs. Watermarking here (on the token following the common sequence) would thus be easily detectable and noticed as a claude style. The longer the hashing context though, the lower the amount of pathological cases with low entropy. It would be interesting to understand the exact parametrization better!
These are services, so "how would <x> do this today?" is irrelevant.
The real question is: "What can manipulation of pseudo-random number generation do?"
We know that in the cryptographic world, attacking "randomness" is a key offensive capability. It will be here as well -- if Anthropic can watermark text as generated it's LLM, will it be able to watermark outputs as generated by "Spooky23/FooCorp"? Can I pay Anthropic to steer inquiries in a way that benefits my company or governemnt?
Pseudo-random to the end user appears random. Most treat it like a random chance. It is not.
Same reason that you don't just replace your rand() implementation with "return 4; // chosen by fair dice roll". If you need randomness for whatever reason, biasing the generator is compromising quality.
In case of LLMs, you can look at it from high and low level.
At low level - if you could do with less randomness, you can always lower temperature. You usually keep it (or for SOTA providers' chat UI, they keep it) at a level where it's about right level - high enough to allow for more creative leaps and interpretations, low enough that it doesn't go off into crazy land after the third paragraph.
At high level - creativity is driven by randomness. If you had an author (fiction or nonfiction) you like for their both broad and deep range of insightful thoughts, would you be happy if they suddenly developed an acute porn obsession and uncontrollably added lewd subtext to every other sentence? Still creative, still deep, but now with that one strong attractor that biases their every thought in a single direction? Would you trust/enjoy their output as much as you did before?
That, slightly exaggerating to make it more obvious, is what "loss of quality" means here.
You're missing the same point that the blog post is missing. What they're doing is much less like replacing rand() with 4 and much more like setting seed(4) before generating any numbers. There is no "loss of quality" unless you're already using a temperature of 0.
reply