This sounds just like something my brother-in-law said. I think they are both technically correct and both missing the point. Does a calculator truly understand math when it spits out a correct answer? Of course not. And it doesn't matter. I have been really impressed with chatgpt, and when it comes to shiny new tech I am usually in the poo poo camp. If tech does something useful then it is useful tech. The fact that it is not true intelligence doesn't matter at all. Besides, what's intelligence anyway? Aren't we still debating that ourselves'?
> Does a calculator truly understand math when it spits out a correct answer? Of course not.
Unless you're using a definition of "understand" that implies conscience of self, I would argue that a calculator is a device that understands nothing except (a subset of) math. That's what makes a calculator reliable in ways that ChatGPT is not.
Philosophically speaking it could be argued that no software understands anything, but I think in the context of this discussion "understands" means "has a model of its context and the way one interacts with it", which is something a calculator (and plenty other software) definitely has and ChatGPT has not.
Calculators don't understand anything about arithmetic. They have no circuits for understanding, no code for understanding, nothing that could represent what humans mean by understanding.
They implement a set of physical processes that, when operated and interpreted by humans, can be mapped into a subset of arithmetic. There's a correspondence.
Correspondence is the most useful way to think about it IMO. If there's a correspodence between what the machine does, and things we humans understand, then the machine, as a tool, is useful.
Understanding is a loaded word. It has implications beyond correspondence when humans use it; it has aspects of qualia, of fact vs fiction, of situatedness in a graph of comprehension, of consonance or dissonance with a set of other concepts, and so on.
LLMs in my opinion have a good "situatedness" for words and concepts, relative to other concepts. Qualia - consciousness - arguably doesn't matter. Fact vs fiction, they're very shaky on. Consonance vs dissonance, they're useless at - LLMs IME tend to flatter the prompt, constructing arguments in whatever direction a loaded question leads. There's little to no coherence there at all.
I think this is where things can get kind of interesting, because future integrations of ChatGPT can farm the "real work" out to systems and tools which do have better models of the specific query.
The LLM approach may not be able to replicate the "knowledge" your calculator has, but it (or some pre/postprocessor) may be able to recognize that a given question is actually something a calculator can answer concretely, and then it can delegate the computation to traditional software that really does "know" how to answer the question.
That would work, but it seems antithetical to AI to have to treat every operation as a special case like that. They want GPT to be able to write computer programs but it'll never be able to work completely independent of humans if every possible domain needs its own plugin to be reliable.
I know lots of the SV VC oligarch cult just wants to race forward and create something like AGI that will help them conquer the world and achieve immortality somehow, but hopefully this remains in the realm of science fiction. As RMS says, LLMs at least don't seem to be "it" no matter how much user generated data they ingest, because they do have these inherent limitations.
The far more practical (profitable) outcome for what they currently built is to just make a useful tool, a "smarter" wolfram alpha, and that can be iterated upon by delegating relevant operations to specific techniques that are more applicable to the question at hand.
Because of you have a special case plug-in for everything then the AI is just a natural language processor, and there's no deep learning for the actual functionality.
>Our brains have different processing centers.
Uhh, no they don't? Did you know everything you know now about math when you were born, and are you also incapable of learning new things about math? Because that's how the wolfram plug-in works.
The Chinese Room Argument holds that a digital computer executing a program cannot have a "mind", "understanding", or "consciousness",[a] regardless of how intelligently or human-like the program may make the computer behave.
https://en.wikipedia.org/wiki/Chinese_room
The Chinese Room experiment shows that pattern matching would return correct results for staged inputs, one would not "learn" enough to evaluate an expression not contained in the data.
The Chinese Room thought experiment is not convincing to software engineers generally. It relies heavily on an intuition that looking things up in a book is clearly not "thinking". Software engineers know better: that "looking things up", if you can do it billions and trillions of times a second, can simulate a process which has a close correspondence to reasoning.
Addition and multiplication are trivially implemented using lookup, if you had a machine without arithmetic and only control flow and memory operations. You don't need much more than that for matrix operations, and now you have ChatGPT, a decent simulation of apparent thinking - which is all that is necessary to kill the intuition dead.
What is thinking if not a series of matrix operations? Your brain is just a huge network of neurons and their connections, is this not a (very complex) matrix?
Certainly not in any mathematical sense. The discrete N-dimensional coefficients of a matrix are not modeled by neurons, their connections, and the quantum mechanical electrical accidents that (a) constitute one's wetware, and (b) aren't completely captured by gates and code.
> (C1) Programs are neither constitutive of nor sufficient for minds.
> This should follow without controversy from the first three: Programs don't have semantics. Programs have only syntax, and syntax is insufficient for semantics. Every mind has semantics. Therefore no programs are minds.
---
I personally don't agree with it and believe that there is a flaw in:
> (A2) "Minds have mental contents (semantics)."
> Unlike the symbols used by a program, our thoughts have meaning: they represent things and we know what it is they represent.
While a person may know what they are thinking, examining the mind from the outside it isn't possible to know what the mind is thinking. I would contend that from the outside of a mind looking at the firings of neurons in a brain it is equally indecipherable to the connections of a neural net.
The only claim that "we know what it is they represent" is done from the privileged position of inside the mind.
I would argue that intelligence is more related to the Kolmogorov complexity exhibited by something.
( David Dowe: Minimum Message Length, Solomonoff-Kolmogorov complexity, intelligence, deep learning... https://youtu.be/jY_FuQbEtVM?t=886 )
That the model of GPT is much smaller than its input.
The Chinese room lookup table is enormously large.
If we attempt to relegate GPT as no better than a Chinese room, we can show that the Chinese room look up table is impossible with the amount of data that GPT has access to as part of its model.
If we say that its not a lookup table but instead an enormously complex interplay of inputs and variables, then the distinction between the room that GPT exists in and our own mind breaks down trying to distinguish which is which.
If we want to switch to consciousness, then possibly the argument can progress from there because GPT doesn't have any state once it is run (ChatGPT maintains state by feeding its output back into itself and then summarizing it when it runs out of space). However, in doing this we've separated consciousness and intelligence which means that the Chinese room shouldn't be applied as an intelligence test but rather a consciousness test.
Are GPT 3 and 4 conscious? I'll certainly agree that's a "no". Will some future GPT be conscious and if so, how do we test for it? For that matter, how do we test for consciousness for another entity that we're conversing with (and its not just Homer with a drinking bird tapping 'suggested replies' in Teams ( https://support.microsoft.com/en-gb/office/use-suggested-rep... ))?
Depends on what the topic of understanding is. In this case it's actually token relationships, right? It does know that very, very well. And there's a lot (.. potentially, hah) that we can do with token relationships.
By itself it's unlikely to ever be knowledge of course.. i see it more akin to NLP than knowledge. Which is to say, a general purpose language parsing tool which we can hand the result to something else. A conversational API, if you will, but we'll still need layers to actually run logic. To know math if you will.
Disclaimer: I know very little on the subject. Pure speculation.
The question is what happens when you go multimodal (which these things can do) and GPT(N+1) learns the associations between words and images/video, as well as the relationships between successive frames of video, at what point does it become unreasonable to claim that it doesn't "understand" something? How good at general-purpose predicting does an AI have to be in order for people to accept that it obviously has an internal model of things and is capable of abstractions?
(Assuming that this happens, of course. Diminishing returns could make scaling infeasible past some point, for instance.)
And additionally, whether our memory and long term learning - and even our goal-choosing - is fundamentally different from an indexed storage of strings of tokens that can be brought back into short-term context when “triggered” by their embedding-similarity to the current context.
I definitely have that question too. I view us as big LLMs.
But, even if we drop that interesting edge case i suspect we can make something very useful with the primitive that LLMs offer.. in the calculator example. ChainLang and co seem a really interesting tool for LLMs.
It matters when you want a correct answer and if it has no way to confirm correctness then you have a problem. The famous goof from chatgpt that I bring up: someone asked chatgpt what the differences were between cow eggs and chicken eggs, instead of answering based on knowledge, it listed out plausible differences based on what it knew about a cow minus the fact cows dont lay eggs.
GPT-4 gets this one now, just as it does the "diamond in a teacup" question. I asked about "pig eggs" instead of "cow eggs" in case it had memorized a public post about cow eggs.
---
Pig eggs and chicken eggs differ fundamentally, as pig eggs are not laid by pigs, while chicken eggs are laid by chickens. Let's clarify the differences:
Chicken eggs: These eggs are laid by female chickens (hens) and are a common food source for humans. They have a hard, calcium-based shell and contain the yolk, egg white, and other nutrients needed for a developing embryo. Chicken eggs are typically eaten for their nutritional content and can be cooked in various ways, such as boiled, fried, or scrambled.
Pig eggs: Pigs, being mammals, do not lay eggs like birds do. Instead, they reproduce through internal fertilization, and their offspring develop inside the mother's womb. When referring to "pig eggs," it's likely in reference to pig oocytes or ovum, which are the female reproductive cells involved in mammalian reproduction. These cells are microscopic and not something that can be consumed like a chicken egg.
In summary, chicken eggs are laid by hens and commonly consumed as food, while pig eggs (or more accurately, pig oocytes) are the female reproductive cells involved in pig reproduction and are not something that can be eaten.
Many of the answers ChatGPT gives would also what a human with average knowledge would give if you point a gun to their head and demand an answer.
My gut feeling is that global long term memory and what we call hallucinations right now might actually be the next step to get closer to general intelligence.
I’m not sure we have the right reinforcement models yet tho, as counter intuitive as it might sound I don’t think that a reinforcement model that is based on correctness only is what we need.
Humans make mistakes all the time, and we bullshit all the time. If anything I think that ChatGPT scares people not because it gets things right but because how confidently it gets things wrong which is something we do all the time.
Older “chat bots” and other NLP things like Watson seemed to be essentially glorified search engines they’ll either give you a correct answer or they won’t answer at all.
So they felt nothing more than a tool not unlike an encyclopedia. ChatGPT will produce an answer under most circumstances but it won’t be perfect and this is what people seem to anthropomorphize with the most.
This is on par with arguing that automobiles will never amount to anything beyond an aristocratic toy because they can't navigate trails on their own like a horse can, can't feed like a horse can, and couldn't even go very far anyway without something breaking.
> There is no such thing as cow eggs. Cows do not lay eggs. Chickens lay eggs that are commonly consumed by humans. The main difference between chicken eggs and other types of eggs is the way the chicken has been raised, treated, and fed. Different farming practices can affect the nutritional content of eggs12.
Something my toddler would probably say as well. Does not mean my toddler has no intelligence - just not wired up to do that yet. Just as humans grow and mature through time, further iterations of AI models will as well
These are fixable as the model is improved, and learns which information to discard. The fact that gpt4 is already so less error prone that gpt3 shows the speed at which this is happening
This is a poor analogy. A calculator automates a completely deterministic operation. You don't really verify that a calculator's output is correct, do you? So, there is no question of intelligence. It is a "dumb" computer by definition.
It matters insofar as determining whether something is intelligent and/or sentient, which is critical to determining whether this "AI" should be conferred human rights or not.
In some sense, I'm not sure intelligent/sentient is all that important when separated from what we think requires intelligence/sentience. I think we attribute less and less to sentience these days and its feels like its less part of the conversation as a result.
Rights haven't really been core to the discussion as I've seen it, and in fact it's the first time I've seen it in months. It's a fair discussion but not the one that has been the focus around whether these models are intelligent or not, e.g. Chomksy's article in the NYT and the MS analysis of gpt-4, or most discussions here for that matter.
All of the nerds complaining about GPT-4 not being perfect aren’t talking about it from the perspective of “conferring it human rights”. It’s all about implying that it not being [insert nebulous word like “Intelligent” or “creative”] somehow makes it useless or a gimmick.
I haven't seen anyone call these LLMs "useless" or "gimmicks." What I have seen is pushback against calling them general AIs or even "intelligence" in general. LLMs are not "intelligent." They do not reason by any reasonable interpretation of the word, and it is certainly an unfounded leap to suggest they "reason" the same way humans do. Especially considering we don't really know how humans reason.
I'm not, we are surrounded by intelligent peers who are not humans and thus do not enjoy human rights (they do enjoy animal rights).
Eventually, after we move from "AI" where we are now to actual artificial intelligence, we'll have to figure out what rights should be conferred if any.
You know perfectly well what he's getting at. Call them Common Rights or Sentience Rights or whatever if the word Human is really causing you confusion. FFS is that the best response you have.
Define machine rights and grant appropriate rights to qualifying machines… whatever those rights are. Like, what 40 hour workweek or right to be upgraded? Right to free output even if not based on any evidence?
We created machine because nobody wants 24 work-hours. Because we want more productivity.
If we gave machines some rights who will do the above for us?
> Does a calculator truly understand math when it spits out a correct answer?
I doesn't seem implausible that some of the first civilians who were exposed to calculators could have been convinced that they were capable of intelligent thought.
And 60s/70s science fiction is full of stuff that they imagined computers would do. Like asking questions and receiving answers that require both inference of facts and deduction like "computer, tell me what happened to this planet".
Eliza is the apt analogy. It's transparently just some if statements substituting phrases into the input, but laypeople that don't understand how it works read into it way more deeply.
Chatgpt is literally just a scaled up version of this. And there's been some kind of eternal September of people who don't understand how a computer works believing all sorts of stuff about it.
> For millions of years, mankind lived just like the animals. Then something happened which unleashed the power of our imagination. We learned to talk and we learned to listen. Speech has allowed the communication of ideas, enabling human beings to work together to build the impossible. Mankind's greatest achievements have come about by talking, and its greatest failures by not talking. It doesn't have to be like this. Our greatest hopes could become reality in the future. With the technology at our disposal, the possibilities are unbounded. All we need to do is make sure we keep talking.
We can see with our own minds and that of animals that there is something greater that emerges with the additional size and complexity of the mind that wasn't there in simpler approaches.
Is not not unreasonable to consider that between Eliza and GPT-4 that something greater has emerged that is able to maintain a consistent world model rather than the just playing with words.
Weizenbaum took a "short cut" for the world model by going down the path of a Rogerian psychotherapy which allowed him to intentionally avoid the need for a world model in order to work with the words that are fed in.
GPT-3 and even more so, GPT-4 has a world model that it is able to work with and interact with.
If we are going to call GPT "just an advanced chat bot" then I would contend it is equally appropriate to call a human "just an advanced sea squirt."
Are people not just a scaled up version of chat GPT? It's not like there's some magical substance floating around in our brains that's responsible for consciousness or intelligence or anything else. It's all just more or less deterministic chemistry.
I think this is the key question that hasn't been kicked around enough through all these discussions about AI. Whether or not ChatGPT is sentient or intelligent is kind of boring, and obviously the answer is currently no.
But are humans just slightly more advanced, chemical-based AI? I don't know. Certainly through the Internet, they seem like it. Go to Reddit and look at the comment section. Especially in political discussions. I'm not convinced the "humans" posting there are not just the dumb output of language models--certainly not much more advanced than ChatGPT. When you [think you] have an opinion about something, how do you know it's actually an original thought, and not just the algorithmic output of your brain's many years of "model training". The word I type next in this comment may, when you peel back all the superstition about "souls" and "free will", simply be my language model's nextWord() function. What is original art? If I paint a picture or compose music, it's not original. It's based on my many years of observing the world, looking at other art, listening to other music. What hubris to think that just because it leapt from human fingers onto a canvas that it's somehow imbued with originality!
Human mind as prediction machine is actually pretty popular theory. See for example “Surfing Uncertainty. Prediction, Action, and the Embodied Cognition” by Andy Clark
> Until now scientists believed that our brain processes the stimuli received from the environment from the “bottom-up”, that is, when we hear someone speak, the auditory cortex of the brain processes the sound first and then activates other areas that are responsible for speech comprehension.
> However, more and more neuroscientists seem to support the theory that the brain ultimately analyzes the external stimuli from the “top-down”, which makes the brain a kind of “prediction machine”.
> As reported by U.S. researchers, our brain anticipates constantly in order to be able to respond lightning-fast and accurately to anything that is going to happen. For example, it is able to predict words and sounds from the context. From the phrase “grass is…” we can easily predict the continuation – it is probably the word “green”.
> > “Our findings show that the brain of both the speaker and the listener uses the process of language prediction. This results in similar brain patterns in both interlocutors,” said the study’s senior author Dr. Suzanne Dikker from the Department of Psychology, University of New York. “This happens even before the speaker utters the phrase he is thinking“.
Again, human brains are not token-prediction transformers. You can see one part of cognition and see that it has a somewhat analogous relation, but that's only one part of human cognition. Labeling a brain a scaled up GPT is mistaking a model for reality, and it's also mistaking a part for a whole.
> Does a calculator truly understand math when it spits out a correct answer? Of course not. And it doesn't matter.
It starts to matter the moment we teach people at large to call calculators "Artificial Math Professor", projecting the image that they do indeed comprehend mathematics at a higher level.
It starts to matter when I meet my aunt (a psychiatrist no less) for lunch and she is absolutely convinced that there is a conscious, intelligent entity living inside her calculator, and she wants to debate the ethics of this with me, getting angry and upset when I doubt the premise.
It starts to matter when I turn on the TV later that day and see our countries minister of education in a panel discussion, debating the future of our schools when AMPs will teach math to our kids instead of flawed, human teachers.
It starts to matter when our government starts debating new laws about "spreading dis-calculation" on social media, convinced that we can just let AMPs read and comprehend all those maths posts in real-time, reporting them to higher authority for posting wrong-calc, deciding on human lives in full-auto-mode.
It starts to matter when I discuss the above with my mother (a lawyer) over lunch and she doesn't understand what the problem is either, convinced that we actually have flawless AMPs that can do all of this, baffled that I, supposedly a "tech guy", am opposed to legislative decisions ushering in new, fancy hype tech.
It starts to matter when this is brought up on places like HN and people with a level of actual technological knowledge, possibly involved in some of this, fail to see the consequences and instead focus on high concept, philosophical debates about what really constitutes a "math professor", constantly moving goal posts around, hooked on their their own hype, rather than marveling the wondrous new technologies of the last years, but for what they actually are.
For people at large, a computer has always been a magic black box that they anthropomorphized anyway. When marketing starts using fancy SciFi words, re-defining their actual meaning to stir up hype, people happily go with the literal meaning of what they are told (as intended; hence the hype). They are absolutely willing to believe that we indeed trapped a Math Professor in a box, Max Headroom style.
Some of the people happen to be in places were they can make decisions, and that's when it really starts to matter.
People didn't call the printing press, calculators, computers, or pagerank "AI", but they all mechanize some aspect of "intelligence". (Though Bool did call his algebra the "laws of thought".)
How will its impact rank within that set?
(I think it's fair to include future improvement; but not new "AI" inventions, since the term "AI" is vague, little more precise than technology for processing information in new ways).
> Does a calculator truly understand math when it spits out a correct answer? Of course not.
Yes, but the difference is that nobody's trying to argue that the calculator is an intelligent being. I've seen people here on HN who are convinced that ChatGPT is a sentient lifeform which deserves its own rights.
Other big difference is that the calculator doesn't ever make things up like ChatGPT does.
Reminds me of Dijkstra's line: "the question of whether a computer can think is no more interesting than the question of whether a submarine can swim".
Amen to that and I'd even go a step further: The fact that it says untrue and invented things with much confidence is unfortunate but doesn't prevent its usefulness. People tell BS all the time and we deal with it.
No - we also don't. It depends on the context, and it depends on the stakes. Some contexts are more "bullshit" tolerant, others are critical.
Conversely to your statement, " """we""" do not condone the error of the politician, of the manager, of the doctor, of the lawyer, of the worker - and we want good warranties for any entity responsible - repeat, responsible".
Which, incidentally, is a reason why Decision Support Systems are Decision /Support/ Systems.
Of course it does. But that's the only thing it understands and it can never understand anything else. It has IMHO the very smallest possible kind of frozen intelligence.
Yes, insofar as we know exactly how the adder and shift registers are implemented we can safely say a calculator "understands" the math its limited button set can be asked to perform. We could certainly replace those circuits with a GPT-like set of internals that would be unreliable as Stallman means, perhaps more reliable than text generation, but fundamentally just as untrustable.
I think Stallman is right.
It's really the term 'intelligence', that's the issue here.
We should stop using using that term. I personally just use 'machine learning' or '(statistical/mathematical) model'. But then there's marketing, i know.
Somebody might have a problem with using machine for non-mechanical devices. I like statistical/mathematical model, but then that seems like what “real” intelligence probably is. So maybe the problem isn’t with the “I” part of “AI”, but with the “A”.
If intelligence is a full spectrum thing and not a binary condition, then maybe LLMs have intelligence somewhere between mold and birds?
'Machine' comes from the root best represented by current 'might', so it is generic enough to cover a good amount of expressions of the area of "enabler" - if those «some» want to be reassured.
> somewhere between mold and birds
In our context, then you should employ it where you would have employed mold and birds in absence of a non-human facilitator.
There would be little issue with the term 'Artificial Intelligence', provided that it were clear it means "problem solver that could replace a human professional, in finding clever solutions".
So, in normal logic, when you call a Statistical Language Model - or an Expert System, or Sorting, whatever - "Artificial Intelligence", it should be clear that you implied the above nuance.
I'd say RMS is right with the implication to not use the term "artificial intelligence" - it groups many different types of systems under one heading, and seems better suited for use as a marketing term rather than in technical discussions.
But I also think it's critically wrong to casually dismiss LLM's as being "not intelligent". This feels like religious-woo thinking that insists there must be something special going on with our minds that exists outside the physical world [0]. While this is possibly true, it's an unscientific null hypothesis - similar to insisting that only you yourself are conscious while everyone else is an NPC.
Rather LLMs and other ML models are just different, on many dimensions. Current ones write better than a high school sophomore. They're often confidently wrong, but so are high school sophomores. The breadth of the topics they know seemingly exceeds any human (so did web search engines a decade ago).
Is it possible to use a current LLM to perform the jobs of many paper-pushing office workers? Probably. Is it possible to take a given LLM instance, as configured for having dialog, and teach it for another "10 years" to get it from HS sophomore to graduate student or skilled tradesperson? Seemingly not.
Ultimately, instead of coarse assertions like "not intelligent" due to not meeting our bar in some aspects, or due to how little working memory it's using answer queries, we need to be be making specific quantitative comparisons to human intelligence rather than trying to pigeonhole the technology with discrete judgements about what "it is" or isn't.
[0] IMO the proper conclusion from the Chinese Room thought experiment is that the understanding exists within the system of rules.
No context was given but here was his actual statement—-
> I can't foretell the future, but it is important to realize that ChatGPT is not artificial intelligence. It has no intelligence; it doesn't know anything and doesn't understand anything. It plays games with words to make plausible-sounding English text, but any statements made in it are liable to be false. It can't avoid that because it doesn't know what the words _mean_.
His comments might have some weight if he actually provided evidence of his assertions. For instance he says "it doesn't know what the words _mean_." How do we indicate/prove that we know what a word means? We can give a definition. We can offer an example of how to use it in a sentence. We can explain and/or demonstrate how its meaning can vary depending upon context. Etc... ChatGPT can do all those things.
So, is there something more fundamental going on in our own brains? If so, what is it?
As far as false vs. true, humans also make many mistakes, confuse references, draw false conclusions drawn from unrelated evidence, and on and on. We also often get things right. ChatGPT is similar.
He just makes conclusory statements so it is just an opinion. I'm not saying any true intelligence has been demonstrated yet, but the progress that has been made in just the last year or two has been enormous. Does anyone doubt the progress won't continue? Something very important is happening and I personally think it is more than just "playing games with words." Just my opinion...
English isn't my first language. I learned a lot of words simply by inference when reading English texts. I used a lot of them before really I knew what they meant, because I had built my own understanding of the words through pure inference. Often that understanding was close to a words true meaning, other times not.
Clearly some meaning of words has been trained into the coefficients that make up ChatGPT, after all it's producing more eligible texts than several people I know...
Meaning isn't defined in the brain, though it may effect and be effected by the contents of the brain.
Tracking the meanings of words over history shows some words stand still (rock, tree, bird), some words gain new meanings and lose older meanings, and some words change the underlying meaning entirely (silly, nice).
These words appear somewhere in history for a first time and are then repeated and modified over the centuries. Meaning-making of words is a cognitive, social and structural process that exists across society as a whole.
The key part is that they are never "proven", merely shown to exist and in use by people. Proving the meaning of a word, is to treat meaning like a science. Meaning cannot be reduced into the scientific process. It grows and changes organically, just as much as everything else does.
Got to say, I miss superhuman computing progress. Chess engines completely destroying chess players, even the world champions, was exciting. A human-equal chatGPT robot is marvelous and a huge step forward. But not "superhuman exciting"....
A lot of people talk about what ChatGPT does. Few seem to talk about the data it was trained on. That data came from us, from people. So besides the algorithm, there is the training which is...what precisely? Everything on the web? Scientific papers? Everything in archive.org? I don't know. But what ChatGPT responds with is a reflection of people, which is a very interesting idea (its mistakes are our mistakes as reflected in our online speech?) People say ChatGPT "hallucinates", where did that come from?
It turns out that accurately predicting text using a model that's a couple orders of magnitude smaller than the inputs requires understanding what the words mean.
And your conclusion would be? If they show to have what it takes we will call them intelligent, otherwise unintelligent.
Some seem to want to cling to a weak definition of intelligence, covering all sentient beings in compassion - but we are engineering /tools/, things that have to be effective, "have what it takes"... This puts the "intelligence" of the narcoleptic carp in a special corner with little relevance to the context.
Surprisingly not mentioned in the post: The whole Free Software movement aside, RMS has actually spent a chunk of his academic career at the MIT AI lab, researching artificial intelligence and co-authored some papers during his time there.
Granted, many of their research topics at the time are now no longer considered AI topics, part of what ultimately lead to the AI winter at the time.
Particularly because of that connection, however, I think it could indeed be interesting to hear some more from his perspective on developments in recent years.
I would argue that ChatGPT has reached a certain level of understanding about what it's saying. That's because you can ask it questions about what it says, and it can continue to reason along the line of what it's previously said. It does sometimes make mistakes in this, but this is improving. It's just that people want understanding to look a certain way, that seems more familiar to us.
I think you are right in a sense. Whether ChatGPT 'understands' might be moot if it produces responses to that effect. Understanding to me implies perceiving the intended meaning, which to me implies sentience. Does it matter though if it doesn't have this property? As the aphorism goes (paraphrasing) "asking whether machines think is like asking whether submarines swim", ie. it's kind of beside the point. If you get the result you want, in this case understanding-like responses, then that's enough right?
Richard was super sensitive about the power companies have over users with closed source software, and had a great impact on our culture (just look at the controversy over the name OpenAI), but it seems like he's deeply underestimating the much much bigger power AI is (and soon will be) having over us, even though there have been countless books and movies predicting it, and we feel it coming.
I view it like Google. Google (web Search) had a ton of impact on humanity on the availability and accessibility of knowledge. Even imagining we continue to improve AI accuracy but assuming we struggle with consciousness i think we'll at the very least be due for a similar impact to that of Web Search.
Which is to say knowledge will be even easier to transfer. Linking complex ideas will be easier to transfer. Having LLMs construct arguments for you. These might have sources cited, might not, who knows. The little details like if they obscure information or cite resources will have quite the interesting impact too.
Then of course there will be the entertainment impact. Potentially being able to guide AI on all sorts of things like creating media for games, story lines, etc - even with lots of human guidance for the creative parts, has a lot of potential for human impact. Not claiming good or bad, but .. something, heh.
In his recent talk for the FSF at Boston Stallman suggested that published weights are open source. I guess because they’re modifiable and auditable. It’s an interesting argument. So far, I’ve managed to modify llama myself so I guess so??
There's no ChatGPT own stuff. It generates perfectly sound text based on trained data. Its capabilites ends when it has to deduce something new. Still, it is really impressive achievement as mostly there's no need for such capability and synthesis on the existing knowledge is more than enough.
It doesn't really know where this training data comes from. It probably isn't a problem yet, but at some point in the future it will be for sure. It already happened with many smaller languages in the world – the languages live their own life in "automatic translation engines are trained by data generated by translation engines" loop.
You can't expect coherent output from a system that lacks a physical component. There's a very strong physical component to human intelligence that machines lack.
It’s shocking to see so many famous intellectuals like Stallman and Chomsky get this so wrong. It seems like they have preconceptions so strongly held that no amount of hard evidence could convince them otherwise.
GPT-4 deeply understands what it is talking about. I was able to pose it difficult classical physics problems that would be hard for a physics undergrad to solve and it could give correct answers consistently. These are things I made up that are nothing like existing examples. When asked to explain it’s work, or modify it for unusual scenarios, it could do so. I could also get it to invent new terms and concepts to describe the work, and then use those correctly. It even could reason out appropriate simplifying approximations when demands exceeded its calculating abilities. If gpt-4 isn’t really understanding what it is saying, or not intelligent than neither are any humans.
Because it's assembling data from an enormous dataset with correct answers. It's a language model, it is physically impossible for it to 'think' or 'feel'.
Those answers aren't in the training set, nor is anything like it. It's assembling a correct and general model of physics from examples in the training set, and applying that understanding to be able to solve any arbitrary problem.
Nobody knows what "think" or "feel" really mean, and I think this surprising phenomena is a clue to understanding that- it isn't what it seems. Most of our theories about what the human mind, language, and thinking are imply this should be impossible, yet it's clearly not.
I have Firefox redirrect reddit links with the "privacy redirect" extension. Unfortunately, it isn't whitelisted for mobile, but it works well if you use a different list on nightly or Fennec.
>I can't foretell the future, but it is important to realize that ChatGPT is not artificial intelligence. It has no intelligence; it doesn't know anything and doesn't understand anything. It plays games with words to make plausible-sounding English text, but any statements made in it are liable to be false. It can't avoid that because it doesn't know what the words _mean_.
I asked ChatGTP plus what it thought:
>do you understand what words mean?
and it replied:
>Yes, as an AI language model, I have been trained on vast amounts of data to understand the meanings of words and their usage in various contexts. I use this knowledge to generate responses to questions and statements that are as accurate and informative as possible. However, it is important to note that language is a complex and nuanced system, and there may be cases where my understanding of a word or its context may not be entirely accurate or complete.
Which I guess is not so different from humans although we have other inputs from the physical world I guess.
This is just a snippet of text out of context posted somewhere
It states something pretty uncontroversial, repeated several times through HN
And does not contain an opinion about ChatGPT utility or impact on humanity
But we already know his opinion opaque software is bad. And the widespread interest in llama and alpaca shows a lot of people agree, want to run it locally, learn how it works, reproduce it
> It plays games with words to make plausible-sounding English text, but any statements made in it are liable to be false
Well, that could also be perfectly applied to any human. The difference being that humans may make purposefully or not false statement (i.e. by lying or by ignorance), whereas the AI will only make false statements by ignorance.
Also... "It can't avoid that because it doesn't know what the words _mean_.". That's very subjective to understand what a word "means", is that again about lies and ignorance? And we need the AI to lie to declare it intelligent?
Maybe, ChatGPT has ironically a modest answer to this claim, when asked what it's thinking about Richard Stallman's statement:
```
Richard Stallman raises some valid concerns about the limitations of AI language models like ChatGPT. It is true that these models, while highly advanced and capable of generating human-like text, still have limitations in terms of understanding and reasoning.
ChatGPT, being a product of GPT-4 architecture, learns from vast amounts of text data and is designed to predict and generate plausible-sounding responses based on the input it receives. However, it lacks the capability to truly comprehend meaning or context in the same way a human can.
It is important to recognize these limitations and not overestimate the abilities of AI language models. Users should be aware that the information provided by ChatGPT might not always be accurate or reliable, and it is essential to cross-verify any critical information obtained from such models with trusted sources.
In summary, while Richard Stallman's statement highlights important concerns, it is worth noting that AI language models like ChatGPT can still provide valuable insights and assistance when used responsibly and with an understanding of their limitations.
>> It plays games with words to make plausible-sounding English text, but any statements made in it are liable to be false
> Well, that could also be perfectly applied to any human.
I agree completely with that part.
> The difference being that humans may make purposefully or not false statement (i.e. by lying or by ignorance), whereas the AI will only make false statements by ignorance.
Sometimes it's difficult to be sure if a humans is overconfident or trying to sound smart, and classify the answer as a honest error, bullshit or a lie. (And sometimes the answer is a mix of them.)
Most of the times human don't lie for fun, but because they have a secret objective. The problem is that ChatGPT has no objective (except trying to sound smart bulshiting plausible-sounding English text).
It's better to discuss about an AI that has an objective, like AlphaStar that "wants" to win a StarCraftII game. Can AlphaStar lie?
* Can AlphaStar send an empty transport plane to the right so the player tries to defend that fake attack, and then AlphaStar attack at the left with real troops?
* Can AlphaStar create fake hallucinate troops that look real for the player but actually can't attack, so the player gets scared and run away or attack the fake troops instead of the real ones?
So much of human "reasoning" is also just automated reflex. We ping pong back and forth the same sequence of sentences all the time. The fact that an LLM can do this as well is a little unsettling.
"Hondas are reliable". To many people this is true not because they independently concluded the brand makes reliable vehicles but someone else told them so. This line of "reasoning" is also how LLM functions.
> This line of "reasoning" is also how LLM functions
And this would rank them below rubbish, because that behaviour, of unfounded repetition, is directly unintelligent, and is what you want to avoid, the opposite of what we seek.
> So much of human "reasoning"
Description is not prescription; that millions of Spanish speaking people say 'Argelia' does not mean it is a good idea; that billions are illiterate constitutes a frequentistic norm, not a normative norm...
> We ping pong
Please don't. Evaluate your ideas and statements foundationally, and through foundational inheritance, and through rankings of doubts, and frequent checks and doubts, like the others do.
he couldnt be more wrong about " it doesn't know what the words _mean_."
That's precisely what transformers do do. They know what ever word and semantical structure means. They are super fast substitution and relationship machines that can take any word and replace it with its definition, and inherit the link between each word in the definition with all its relationships.
It's precisely its ability to distill something into its pure meaning, and to reply to that "meaning" that makes its autocomplete abilities so powerful.
This seems to miss the forest for the trees. Whether LLMs will have an impact similar to the industrial revolution will not depend on whether they pass some arbitrary threshold where everyone is convinced that it is an AGI and understands what words mean. It will depend on the utility they provide. And the utility is there right now. GPT4 is so immensely useful for so many things. At even a reasonable pace of improvement, it is hard to see why LLMs would not be able to do more and more things.
Morever and slightly off topic, most humans also don't care what words mean or what numbers mean in a philosophical sense. If you talk of "The Axiom of Choice" in a big company software meeting, people will ask if it is the new ice cream flavour in the cafeteria. That doesn't prevent people from getting value out of both words and numbers.
The problem is that most tasks require general intelligence at some point. So the question is more: given that in the vast majority of cases, weak-AI cannot go from "0 to 100%" how much should we care?
The whole "the sky is falling" schtick i think requires weak-AI systems to be getting very near 100%s reliably across industries. Anything even 90% and we're talking more about assisting and empowering users than replacing them.
> This seems to miss the forest for the trees ... most humans also don't care what words mean
You miss the point, because you are reasoning around the different perspective of "hav[ing] an impact similar to the industrial revolution", so you take the semantic "payload" as if it were "just an expression".
The point is, that it produces statements, and in general we want true and founded output from an entity producing statements - "intelligence" is the condition enabling that.
So, when the machine says
"Napoleon was addicted to cotton candy"
, we "must know" that the machine has verified it, like a "good" statement producer would (as opposed to all the other reasons to produce that statement);
and when the machine says
"The current war will be won by the white faction"
, we must know that it effectively reasoned on the relevant conditions and factors, on par with the best Intelligence Analysts (as opposed to all the other reasons to produce that statement).
I doubt it. I would expect that if GPT's "knowledge" reaches some limit, OpenAI could just hire subject matter experts to write books for the model to read, and then use those to (effectively) down-rank material that differs substantially.
If we're talking about the modeling itself, I think we're a long long ways from any limits. There's room for improvements such as cheap personalized results, faster hardware with more memory for lower costs, bot-to-bot communication allowing my instance to interface with yours to come up with mutual understanding, etc. Pretty much all of the components for this exist already in some form, they just need to be optimized and implemented on a large scale and made available to us humans.
I found this Veritasium video interesting, about a company that is paring down models so they fit on analog chips, so the chips can be used in self-contained devices. It also appears that quantum computers may be even better for matrix math that transformers rely on. If all these hardware advances pan out...
It takes months to years to write a well researched and in-depth book and subject matter experts are not necessarily good authors. Beyond that, each book only moves the needle a little bit as well because of the size of the training corpus. If there's going to be improvement from that path, it will be at best slow and incremental.
You're right, maybe a whole book is unrealistic. I guess it's more likely that they'd have someone make small corrections, like if enough people flag a common result as wrong or poor.
Self driving cars have a very different problem. Regardless of the improvements and sufficiency they reach, their errors can be fatal. One news headline about a self driving car crash will ensure that the technology doesn't get any approvals.
ChatGPT for all it's flaws can't jump out of the computer and harm the user. It can harm users in others ways, but those are less known or understood by the general public, and thus it is much easier to take to general availability.
I agree on the limit part that it might be possible the technology has reached a limit, however, it is still better to think about what happens if it hasn't and the rate of improvement continues.
One one hand, it would be more interesting to hear RMS' views in the implications, if any, on software freedom (personally I think there are many angles here).
On the other hand, the comment attributed to him is correct, if simple and pretty obvious.
I actually think it shows incredible reasoning ability already. It can change its answers based on new content you provide. For example, you can show it a Java program and ask how it will behave, then show it release notes of a new Java version it has never seen before and ask it how the functionality may change, and it will get it right. Most programmers won't because our ability to attend to information is far inferior unless we really try hard. Focus is not something most humans excel at. Our brains are more capable but are less utilized most of the time.
> It has no intelligence; it doesn't know anything and doesn't understand anything.
I don't like the word "intelligence". It's too arbitrary and depends on the language. In my native language, there are two synonyms which imply different thresholds for something to be considered intelligent. I'm sure in other languages it's also pretty arbitrary.
Instead, let's compare those models with complex biological systems we typically consider to be at least somewhat intelligent.
- biological systems use spiking networks and are incredibly power efficient. This is more or less irrelevant for capabilities.
- biological systems have a lot of surrounding neural and biochemical hardware - hardwired motorics, senses processing, internal regulators. Complex I/O is missing from these models, but is being added as we're talking. The large downside of current models is that it cannot understand what drives humans as it has different hardware, it's trained on their output, and has to "reverse engineer" the motivation. Which might or might not be possible, but it makes them different.
- biological systems are autonomous agents in their world. They exist on an uninterrupted timeline, with input and output streams constantly present. Those models don't exist on a timeline, they are activated by the user each time.
- biological systems have some form of memory; they compress incoming data into higher order concepts on the fly, and store them. This is a HUGE DEAL. The model has no equivalent of memory or neuroplasticity, it's a function that doesn't keep any state. LLMs have the context which can be turned into a sliding window with an external feedback loop (chatbots do that), however it's not equivalent to biological memory at all, as it just stores tokens verbatim, instead of trying to compress the incoming data.
- biological systems exhibit highly complex emergent behavior. This also happens in LLMs and even simpler models.
- biological systems are social. Birds compose songs from tokens, and spread them through the population. Dogs, monkeys, and humans teach their kids. The mental capacity of a human isn't that great; every time you think you're smart, remember that you stand on the shoulders of giants. The model does have much more capacity than a human.
My own conclusion: sparks of "intelligence"? Undeniably, the emergent behavior alone is enough. They do understand things, in the conventional terms. However, they are also profoundly different than human intelligence, and still lack key elements like the memory.
Thanks for the downvotes, but the point is not that Stallman should prove it but the point is that you should reflect on the fact that the point being made cannot be proven. Ever. Also not for you, yourself.
Sure, it has nothing to do with the resources required, the costs involved, "profit motive" to euphemize crippling human greed and selfishness and lust for control and power.
The resources required are peanuts compared to funding of moderate silicon valley startups. There are estimates of GPT-4 training run priced around 10e6$
There is no culture of trusting engineers with this kind of money almost everywhere except a few orgs like openai, that's for sure.
And if we ask "why is such culture in place" we get a long tail of answers, one of them in the link above.
I think his answer is driven by a preference for the status quo and a reluctance to face difficult changes.
ChatGPT, and especially GPT-4, seem to do much more than just play games with words. You can’t overlook the “emergent” phenomena that manifest themselves when using them.
If we look at when this "emergence" takes place you'll see it's an illusion. Namely, predictions (text output) are good-looking when sampling from data which are similar to the prompts; and they're bad-looking when not.
So, when do models go from being "not intelligent" to "intelligent", well no surprise: when their data includes everything ever written. Is this emergence? No, it's just sampling from *data with the relevant properties*.
How does data acquire those properties? Well when people produce it.
What "emerges" from AI is not from AI; its from the properties of the data its using. The data here is produced by intelligent agents, and it is light of their intelligence which shines thru' GPTs like a prism.
I struggle with this take as it’s not different from humans.
Am I intelligent at age 3? Then I learn language. Am I intelligent? Then I progress through education. I become more and more intelligent as I am exposed to more data.
I don’t know about computer science, so I go expose myself to computer science literature. I take courses, and see examples. What emerges is a “more intelligent” version of myself that.
But doesn’t it imply that once we have something that can “see” its environment, hear nearby sounds, and have some sort of haptic sense — wouldn’t that be AI, then? Given your definition?
If, of course, such a thing used all that input to “create” concepts like its physical place within an environment, what may or may not come next given the interplay of those inputs, and how to navigate the environment based on all that.
But then I’d say, “GPT-4 can handle image data. And what might come in the future with more modalities?”
They are sometimes bad looking, and they are always better than randomness, which does indicate understanding even if the conclusions are faulty.
>What "emerges" from AI is not from AI; its from the properties of the data its using
This is squabbling over semantics. If the properties of the data are used to the effect of displaying intelligence, then the fact is that what you are looking at is intelligence.
If I define a word I have never seen before using deductive logic according to the passage I find it in, I've still acquired an understanding of the word, even though I've not referred to anything external.
> it is important to realize that ChatGPT is not artificial intelligence.
The first is to assume that there is a technical, precise, objective and clear definition of "Artificial Intelligence". There isn't. He should know that.
> it doesn't know anything and doesn't understand anything.
And what does "know" or "understand" means in the context of a machine that doesn't even have self-consciousness?
Besides, are you implying that human beings know stuff? The overwhelming majority of people knows very, very little. Most of the people I know are too lazy to think or doing the hard work of studying. I'd suggest Kahneman's "Thinking Fast and Slow" before putting any faith in people's "knowledge".
> It plays games with words to make plausible-sounding English text, but any statements made in it are liable to be false.
I think we can we apply the same judgment to Stallman's argument itself, since his concepts are so badly defined.
And thank you for an open democratic society where every statement is liable to be false, regardless if it comes from a machine, Richard Stalman or Putin and Jiping.
I'll take Turing's test approach: if it looks intelligent to me then it is certainly more intelligent than me.
Also, I'll take Dijkstra approach: "the question on whether computers can think is as irrelevant as whether submarines can swim".
Edit: to all Stallman's fanboys downvoting this: got any good argument?
If you consider stallman is using the colloquial definition of artificial intelligence (which, as pop culture uses it, is basically artificial consciousness), his answer makes a lot more sense.
"Definition" is a rigorous concept and technique in philosophy and logic. It comes straight from Aristotle and is a foundation for the whole western culture.
I won't consider any colloquial definition of something so precious.
That's a pretty weird take. If you understand what definition he is using you can better understand his point. Otherwise you're just arguing against a strawman.
