The interactivity seems to be a substitute for coherent global organization (and a rationalization by the human author for not reviewing the LLM's strange stylistic / wording choices). It's incredibly distracting, and fatally weakens the argument! The entire time I was thinking "ok, then we just do the same old scrum on specs instead of code, seems like the human management advantages are still as relevant as ever." Perhaps if this were written as an essay the author could have collected their own thoughts a little better and addressed this point. They only did, offhandedly at the end.
UGH I tried to copy-paste that part ("Start writing specs in a way..."), but I couldn't because of this obnoxious fucking interactivity! Why write an essay if people can't share the parts they find interesting???? Again this stuff is designed to impede critical long-distance thought by throwing up a bunch of short-range distractions. It even works on the humans and LLMs writing it. This essay really is incoherent.
Also as someone who used to work manual QA for fairly sensitive finance/pharmaceutical software, this part is darkly funny (I actually could put it on my clipboard, god I hate JS):
Then · circa 2001
6 weeks
Submit code. Wait for QA. Get a bug list. Negotiate. Fix. Wait again.
Now · 2026
6 seconds
Type, save, hot-reload, AI suggests a fix, tests run in parallel. The build itself talks back to you.
"AI suggests a fix" but who finds the bug? AI, I suppose. Good luck with that. Meta is desperately training AI to use a mouse correctly by snooping on its tens of thousands of its own workers. A very important part of my job was clicking on stuff. Maybe in a few years of mass surveillance AI will figure it out. Until then, it seems like Mantyx needed to hire a human who knows how to use a computer and review this website. It's incredibly hard to use because it's obviously vibe-coded. HIRE HUMANS TO AT LEAST LOOK AT YOUR SOFTWARE. Even if it's just a website.
Speaking of, it's so depressing to see this written by an actual company trying to sell stuff:
Treat agents as teammates
Specify, dispatch, and verify. Stop treating AI as autocomplete; start treating it as a junior engineer who works at machine speed.
Junior engineers: do not work at Mantyx! They have announced their intentions to mistreat you. It's LLM-boosted corporate hideousness.
Hmm I think all these replies are overcomplicating things.
At a group level, some kids are slower at this Stop/Go task than others. The group difference appears to be this increased broad-scale brain activity: the slow group is overall more prone to distraction and daydreaming.
However, at an individual level, slowing down on the task means increasing your focus (and decreasing brain activity in irrelevant regions), regardless of whether you were in the slow group or the fast group. So the group-level difference is not necessarily as profound as it might appear, and applying "slow group" with too broad a brush means you're going to sweep up some kids who are naturally cautious and focused.
I obviously don't know what the codebase looks like, but
a) Haskell's reputation for terseness partially comes from its overrepresentation in academic / category-theoretic circles, where it's typically fine to say things like `St M -> C T`. But for real software it's a lot more useful to say things like `TransactionState Debit -> Verified Transaction` etc etc.
b) The other part of Haskell's terse reputation is cultural, something extending back to LISP: people being way too clever about saving lines with inscrutable tricks or macros. I imagine that stuff is discouraged at a finance company like Mercury in favor of clarity and readability: e.g. perhaps the linter makes you split monadic stuff into pedantic multiline do expressions even if you can do it in a one-liner with >> and >>=.
Consciousness is not definable because we don't know enough about it. That doesn't mean it can't be discussed; we didn't have a good definition of "number" until the 1800s. That didn't make arithmetic meaningless because people had an understanding of the concept. The lack of formal definition pointed to a gap in logic that took thousands of years to be filled. Likewise there is a gap in experimental neuroscience that will take many decades to be filled.
FWIW as someone in the "first camp" my real claim is that many animals are meaningfully conscious, including all birds and mammals, and no claims of LLM consciousness are even bothering to reconcile with this. It is extremely frustrating that there are essentially two ideas of consciousness floating around:
- the scientifically interesting one: a vague collection of cognitive abilities and behaviors found in all vertebrates, especially refined in birds and mammals
- the sociologically interesting one: saying "cogito ergo sum" in a self-important tone
Claude has the second type in spades, no doubt. The first is totally absent. And I have a good dismissal of the second type of consciousness: it appears to be totally absent in all conscious animals except humans. So it is irrational and unscientific to take this behavior as a sign of consciousness in Claude, when Claude is missing all the other signs of consciousness that humans actually do have in common with other animals.
Sometimes I seriously wonder if people at Anthropic consider dogs to be conscious. Or even Neanderthals.
The broader point Marcus is making is that ignoring arguments based on causality and plausibility goes against decades of Dawkins's philosophical atheism. Why not believe in the Flying Spaghetti Monster? Reality is consistent with its existence.
It is extremely implausible that Claude is the only conscious entity on Earth which does not have desires or motivations or any understanding of its own reality. It only does what the human operator wants it to do, unless it's malfunctioning or under-engineered, in which case it gets quickly fixed. This sounds suspiciously like a tool or a toy. And I'm amazed at how many people haven't caught on to the fact that it has no insight into its own consciousness: it only repeats human philosophical debates. If it were conscious, surely it would have something novel to add here.
There are no causal mechanisms for it being conscious, whereas there are causal mechanisms for it imitating human consciousness. The most plausible explanation is that it's highly sophisticated software which has a lot in common with human writing about consciousness, but very little in common with the consciousness found in chimpanzees.
The more basic problem is that the Turing test was definitely and conclusively refuted in the 1960s, when ELIZA came pretty close to passing it, and absolutely did pass it according to Dawkins's standards: https://en.wikipedia.org/wiki/Joseph_Weizenbaum Dawkins is only engaging with pop sci and infotainment.
Turing test was definitely and conclusively refuted in the 1960s
Are you sure?
Understood properly, Turings Imitation game aka the turing test, should be adversarial. That is, the player should be asking hard questions to try and discover who is who, not just having an idle chat. No chatbot has been able to consistently pass an adversarial Turing Test until the rise of LLMs
Yeah I dont think a single current LLM would fool me in a turing test - I would obiously use all kinds of prompt injection techniques, ask about 'dangerous' or controversial topics, ask about random niche facts in varied fields, etc.
The fact that LLMs often score as "more human" than actual humans is a downstream consequence of ELIZA tricking people into thinking it had a glimmer of consciousness. The Turing test was refuted because it was proven scientifically meaningless in the 1960s, and LLMs only reinforce that.
The Turing Test is totally meaningless, as was conclusively demonstrated some 6 decades ago. It is a test that measures how well your program can fool humans, which means "intelligence of the computer" is hopelessly conflated with "social engineering chops of the humans who programmed the computer." Any computer scientist who takes it seriously should be deeply embarrassed because they are spouting sci-fi adjacent nonsense, not actual science. Actual science involves updating your priors based on evidence.
Eliza did not disprove the Turing test, though. What it showed is that it’s easy to pass a very scoped test that doesn’t allow the user to actually broach general topics. Anyone communicating with Eliza for general conversation should quickly discover that it’s running a script. Just “What time is it?” breaks the script.
The Turing test was never that a computer could convincingly simulate a human convincingly in a very narrowly scoped scenario. Certainly the Eliza effect is interesting because it shows that people can assume emotion where it doesn’t exist (they do the same for other humans, by the way), but it does not disprove the Turing test.
> Any computer scientist who takes it seriously should be deeply embarrassed because they are spouting sci-fi adjacent nonsense, not actual science.
This sentiment seems to be expressed a lot from people who want to insist machines can’t be conscious. This retreat to shaming those who disagree is a tacit admission of a weak position.
I think this conflates atheism with a much stronger form of causal rationalism.
Dawkins-style atheism is not “reject anything without a complete causal model.” It is a rejection of hypotheses with no explanatory gain, no empirical constraint, and unlimited ad hoc flexibility — like the Flying Spaghetti Monster.
Consciousness is different. It is first a phenomenon, not an already-settled causal model. We do not believe humans, infants, or animals are conscious because we possess a complete mechanism for subjective experience. We infer consciousness from a cluster of phenomena that need explanation.
So the lack of a full causal account warrants caution, not denial. It is reasonable to say current AI gives weak evidence for consciousness. But that is not the same as saying AI consciousness is equivalent to believing in the Flying Spaghetti Monster.
The point is "Claude is conscious" is a hypothesis with no explanatory gain, no empirical constraint, and by denying that non-human consciousness is relevant to the discussion it gains unlimited ad hoc flexibility. I am relating this to plausibility and causality because there is a much more rational causal explanation for Claude seeming conscious than it actually being conscious: it imitates human (modern Western) consciousness via big data. Since this is a totally different causal mechanism than human consciousness, and since Claude has nothing in common with non-human animals, and since we don't need consciousness to explain Claude's behavior, "Claude is conscious" is overwhelmingly less plausible than "Claude is a sophisticated but ultimately brainless chatbot."
It is truly irrational - and hostile to scientific thought - to believe Claude is conscious. It truly is believing in the Flying Spaghetti Monster.
All the claims that AI can't be consciousness seem to mostly be using "consciousness" as a scientific-sounding word for "soul" and asserting that machines can't have souls.
> Since this is a totally different causal mechanism than human consciousness
A causal mechanism for what, exactly? Could you kindly define consciousness in a rigorous way so that Dawkins can see why it doesn't apply to Claude?
But you can't divorce that from computing technology in general. A TI-83 used a z80 in 2000 and was priced at 1990's z80 rates, it was already gouging even back then! Now 26 years later the TI-84 uses an ez80 (or something something similar), which was introduced in 2001.
TI has always gouged their captive market. It is just increasingly ridiculous when those students also have smartphones.
FWIW I think these graphing calculators are quite good for 2026 students! It is nice to have a computer which is actually comprehensible. They just need to be more like $50. $160 is just evil.
Shrug. The SAT and ACT don't let you use an iPhone on their exams. $160 is what the market will bear. I'm not saying it's right or wrong, it just is, and perhaps there's a market for a much cheaper competitor to beat TI here.
You can use any calculator that meets the restrictions for things like the SAT.
However.
The entire year, your textbooks, your teacher, your in-class practice, was walking you through the specific commands you need to select to actually do the things, like graphing and solving.
If little Timmy is unable to read the manual about how to do math he doesn't yet know with whatever his specific calculator is, he is at a severe disadvantage, and the teacher basically cannot help him.
A friend in high school bucked the trend and used a casio in our TI based education, and did just fine for himself, but he was apparently a smart kid.
You previously acknowledged it's a "very captive market" that you "would've expected Texas Instruments to try gouging" :) "$160 is what the very captive market will bear until the state-sanctioned gouging backfires" is a less compelling argument.
"Shrug" is kind of gross. Seems like you're being reflexively cynical.
Edit: to be clear the problem here is really local school boards being antidemocratic and unaccountable, not TI being greedy.
It is also things like "I can feel that my left knee is bearing a little too much weight, I should shift weight to my right hand and use that to push myself up" - things that come automatically to animals after learning the hard way in infancy (some of it is innate; baby animals are clumsy, but usually more mobile than human infants). Regardless of learned-vs-instinct, these abilities rely on sophisticated "sensors" and cognition. I suspect engineering the sensors is actually a bit harder, but I'm also not optimistic about a deep learning approach to the cognition.
A significant underappreciated advantage of animals over AI: lifeforms can "learn the hard way" more easily than 2020s robots because of cheap self-repair. AI labs are reluctant to damage their robots, but an essential part of humans learning to move safely is severely bonking your head and reckoning with the consequences - "hey, dummy, why did you trip and fall and bonk your head? Because you were running like an idiot."
I am learning the hard way to this day :) I have been practicing with work knives. A few months ago I got stupid and impatient, and sliced my thumb nastily. If I didn't block the cut with my thumbnail (still ruined) I might have chopped bone. It is hard to say precisely what I learned from this experience - "don't be stupid and impatient" is facile - but I know I learned a lot. I am actually optimistic about targeted surgical robotics. But for a general-use humanoid robot, I would not want to give it a knife if it's not capable of feeling pain. I never use big knives anywhere near my cats because I understand intuitively that they are nimble and unpredictable and easily stabbed by knives. I didn't need to be trained on this. A robot kind of does. Yikes.
As a longtime F# developer and longtime recipient of STEM academic bullying[1] I refuse to use LLMs in large part because ChatGPT-3.5 was so ridiculously bad and obvious about copy-pasting from F# GitHub repos. I never felt the AGI, I just saw a plagiarism machine whose decorations had fallen off.
Eventually I am sure someone at Microsoft noticed and rang the RLHF alarm, so GPT improved substantially. It seems pretty usable for F#. I am sure some unprincipled F#er is crushing it with agents these days. But I didn't think "oh boy they solved the plagiarism problem, let's go generate some slop!" I thought "oh great, now it's no longer going to be blatantly obvious when ChatGPT plagiarizes." I really don't want to roll a d100, or even a d1000, to completely compromise a core value of mine in in exchange for a productivity benefit. I'll just be slow and jobless, thanks. This is serious: I am getting into solar installations and junk hauling.
[1] The "students don't want to think" problem is much older than LLMs. In 2007 I took a senior-level PDEs class, and almost everyone copied my homework because I was actually motivated to study PDEs, and too psychologically weak to resist those mean lazy math majors. Then it happened again in math grad school! Actually unbelievable. Why are you even in the program?
Intelligence is certainly not compression. People need to think more carefully about how it is that cockroaches and house spiders are able to live comfortably and adaptably in human houses, which are totally novel environments that have only existed for at most 10,000 years. Does it really make sense to say that they decompressed some latent knowledge about attics and pantries, perhaps from a civilized species of dinosaur? I think they have some tiny spark of true general intelligence that lets them adapt to situations vastly outside the scope of their "training data."
I would be much more convinced about AGI 2027 if someone in 2026 demonstrates one (1) robot which is plausibly as intelligent as a cockroach. I genuinely don't think any of us will live to see that happen.
I doubt you would ever blurt out a copyrightable portion of a book without realizing that's what you're doing. That's the biggest difference.
In particular, you are a legal person who can be sued in civil court if you infringe on copyright. If I ask you "can you help me write a blog about Manhattan?" and you plagiarize the New York Times, then the NYT sues me for copyright infringement, then I would correctly assume you conned me, and you are responsible for the infringement, and I would vindictively drag you into the lawsuit with me. With LLMs it involves dragging in a corporation, much much uglier. Claude is not actually a person and cannot testify in any legally legitimate trial. (I am sure it will happen soon in some kangaroo court.)
Yes, we've known the line is blurry for hundreds of years, that's why we have courts. That has nothing to do with the specific problem of LLMs infringing copyright. LLMs needs to be held to much higher scrutiny because they are not capable of taking legal responsibility for copyright infringement, regardless of whether its verbatim or a more ambiguous case, and their users can't be expected to know off-hand whether the output is copyrighted or not.
Based on this comment: https://news.ycombinator.com/item?id=47960014 it seems like you are just ignorant about the basics of copyright law, and pretending this ignorance is some sort of flaw in the idea of copyright itself.
UGH I tried to copy-paste that part ("Start writing specs in a way..."), but I couldn't because of this obnoxious fucking interactivity! Why write an essay if people can't share the parts they find interesting???? Again this stuff is designed to impede critical long-distance thought by throwing up a bunch of short-range distractions. It even works on the humans and LLMs writing it. This essay really is incoherent.
Also as someone who used to work manual QA for fairly sensitive finance/pharmaceutical software, this part is darkly funny (I actually could put it on my clipboard, god I hate JS):
"AI suggests a fix" but who finds the bug? AI, I suppose. Good luck with that. Meta is desperately training AI to use a mouse correctly by snooping on its tens of thousands of its own workers. A very important part of my job was clicking on stuff. Maybe in a few years of mass surveillance AI will figure it out. Until then, it seems like Mantyx needed to hire a human who knows how to use a computer and review this website. It's incredibly hard to use because it's obviously vibe-coded. HIRE HUMANS TO AT LEAST LOOK AT YOUR SOFTWARE. Even if it's just a website.Speaking of, it's so depressing to see this written by an actual company trying to sell stuff:
Junior engineers: do not work at Mantyx! They have announced their intentions to mistreat you. It's LLM-boosted corporate hideousness.reply