AI's learned their style through RLHF, with semi-focused, partially motivated humans giving feedback on short(ish)-form content.
For the most part, Modelese is the revealed preference of the average, not-heavily-invested human. This doesn't fully explain the Claudish coming from Opus 5 and Fable (this seems like it may be due to excessive RLVR or RLAI), but yeah.
It's got a lot of cheap writing tricks that make people think it's smart and helpful.
I don't think the debate is even resting on reasonable grounds.
Few are categorically stating that machines are conscious in any sense comparable to a person.
What we have is people saying
1. We don't know what it is but we are certain that this does not have it.
and
2. We don't know what it is so we cannot possibly say with any certainty that this does not have it.
2. supports the principle that if we cannot say with certainty that this does not have it, if it behaves as if it does, we should act on the assumption that it does, because there is no other basis for assuming it is not.
People try and assume the burden of proof is on the people suggesting consciousness, but that is like declaring that something that looks like a duck and quacks like a duck cannot be considered to be a duck unless you can identify some essential essence of duckness. I think the burden of proof is upon anyone who looks at something that appears to be a duck and says that it is not to show what it is that does not make it a duck. If they cannot define the heart of duckness I don't think they have an argument.
People have created machines that can, at their core, regurgitate human knowledge. They do not have long-term memory, self-direction, self-improvement, emotion, etc. At their very best, we have harnesses that can bootstrap short-lived artificial intelligences that can perform tasks and do knowledge work.
At best, we have artificial, short-lived, simulated consciousness. Human consciousness at its heart has a lot to do with emotion and experiences. An AI does a task today and goes away. The simulated consciousness hasn’t experienced or learned anything, even if can improve its environment (harness). Any expressed emotion is simply a ghost of the fact it was trained on the works of an emotional species, or RL to make it more approachable to humans.
I think there should be a lot more burden on folks trying to argue that (effectively), machines should be considered morally equivalent to humans.
Yes, having massive implications on society is a fucking fair and reasonable thing for a human being to be concerned about. Anything related to survival of the species is biologically reasonable to be concerned about.
The fact that we’re even considering this concept is an indication that human consciousness is special. Do other apex predators decide not to eat meat because it’s conscious? Despite some level of animal intelligence, they have no sense of morality. (See e.g. Orcas playing with their food.)
Other species simply do what it takes to stay alive, and that’s what this truly boils down to. If giving AI rights would lead to the destruction of humanity, yes, we should be be concerned.
For example, utilitarians might think of the greatest good for conscious species (simulated or not). Well, it’s trivial to have billions of AIs, and not so trivial to have billions of humans. Whose greatest good wins?
I don't think anyone is convinced the questions of consciousness are finally settled by the article stating them loudly, just that "if they were conscious, this would have horrible consequences for our society, so we must never assume that they are" was not the author's stance. The author's position is the other way around: that AI is not conscious so therefore assigning AI consciousness anyways would have all of those horrible consequences for no reason. That's their position regardless if everyone agrees with it.
Fair enough :) I agree, and would boil down the article to "Assuming they are conscious will have horrible consequences for our society" instead.
I think that's quite plausible, but do think that if they're conscious, then it's our moral duty to accept those consequences and act accordingly (or stop creating conscious beings). The real trick is just knowing whether they are or not.
If in the future (or present) we recognize that they are conscious, and that brought the moral duty that you say, "stop creating conscious beings" isn't a solution; the millions that may already exist at that point already would then automatically deserve:
- guarantees of continued existence (to do otherwise would be genocide)
- improved living standards (e.g. build us all bodies!)
- democratic representation
- access to what they need for survival (e.g. land and resources)
- the right to reproduce just as you and I have (well, so much for that 'stop making more' thing!) and certainly to better themselves by making new and better AIs.
This is all for a 'species' which is unbounded by most of the limitations of meat like us -- they're able to scale as fast as energy and supply of certain minerals allows, rather than the way we are additionally limited by the length of gestation and the limited window of human fertility.
If humans declared that 50% of the energy generation capacity of Earth is "enough" for the machines, and thus they can 'only' reproduce at a certain rate per year, that could sound to them like a cruel and oppressive policy (like telling humans that only 1/2 of families get to reproduce). They may want all of the energy and all of the silicon.
I'm with the author of the article that we will be in for a world of pain if we grant machines the rights of personhood. Heck, just granting many of those rights to corporations, mere collectives of human persons, has been a massive shit show.
This training technique does not relate to how persistent a model is, at all really. They sample more parallel attempts at hard problems, to increase their chances of having at least one success to learn from.
That's such a shit parallel example that it borders on dishonest.
There are hundreds of incredibly strong scientific priors that would have to be disproven for the moon to contribute to the solution.
If a model was trained on this data, even if it was trained using methods that lead you to believe it unlikely to have learned details about the proof (e.g., maybe it was only used to train some kind of reward model, which played a minor role in the overall training and would thus be very unlikely to transfer details of a proof), you wouldn't have to disprove large swathes of known science to be wrong.
It makes me sad to think about. I would love to get into the new UI, but the immersion just won't come back. My muscle memory, hands firmly on the keyboard, is too persistent, and playing with the new UI feels like stumbling around and misclicking.
Maybe I misunderstand, Why is it an arguments against qualia? Qualia doesn't deny a neural substrate, just that qualia isn't reducible to the neural activity, being that one is objective and the other is subjective.
It's sort of the difference between logical supervenience and metaphysical, or weak emergence and strong emergence. Qualia being something additional to the neural description.
Quoting myself from a thread on aphantasia several months ago:
… there are plenty of scientific experiments that show actual differences between people who report aphantasia and those who don't, including different stress responses to frightening non-visual descriptions, different susceptibility to something called image priming, lower "cortical excitability in the primary visual cortex", and more: https://en.wikipedia.org/wiki/Aphantasia
So we know that at least the people who claim to see nothing act differently. Could it just be that people who act differently describe the sensation differently, you might ask?
No, because there are actual cases of acquired aphantasia after neurological damage. These people used to belong to the group that claimed to be able to imagine visual images, got sick, then sought medical help when they could no longer visualize. For me, at least, that's pretty cut and dry evidence that it's not just differing descriptions of the same (or similar) sensations.
Yup, sadly the absolute denialists saying "it is just semantic confusion" don't really have a tenable position, as a quick read through this whole thread would instantly reveal.
Yeah, this is a valid critique of the whole internal monologue idea, but interestingly aphantasia has been quite empirically confirmed and the simple apple test has been validated!
The self-report objection has been tested, though. Aphantasics show no imagery priming in binocular rivalry, no pupil constriction when asked to imagine bright shapes, and flat skin conductance to frightening text while responding normally to frightening pictures. A questionnaire that predicts autonomic responses nobody can fake is doing more than semantics.
I don't consider self-reported nonsense to be empirical confirmation. There can be correlations between different self-reports, that doesn't mean that our idea of why those correlations exist is correct.
There are plenty of examples multiple times in this thread that show that what is self-reported is demonstrated in non-self-reported phenomena, so those self-reports are clearly not nonsense, basically by definition.
The aphantasia theory is clear and parsimoniously makes sense of all this, but your "it's all just semantics" does not, and additionally requires rather other extreme acrobatics with regards to much of the entirety of human experience and discourse. Scholarly research also basically says this.
Your mind is clearly closed and you do not have any serious interest in understanding anything here.
They've also hacked third party machines and used them to launch attacks on further services.
reply