It's not really even a question. It's an obvious boondoggle. The forecasted net new energy requirements for the AI buildout over the next couple of years are roughly equivalent to all of Western Europe's power demand today.
That's absurd. It's a physical impossibility to bring that much power online that quickly. And the cost to get even close would make AI more expensive than just hiring knowledge workers to do the same tasks.
And it's all predicated on a tower of wobbly or broken assumptions -- chief among them that increasing the size of these models yields better performance.
We're going to look back on this era and wonder why anybody took any of the outrageous claims of tech CEOs seriously.
> Wobbly assumption that increasing the size of these models yields better performance.
I'm assuming you disagree that larger models are better? Can you expand on what indicates that AI will hit a wall in scaling given the evidence of the last 9 years of scaling transformers (or other models)? Where on the plot does the line go from exponential to flat?
Leaks from within OpenAI have made it pretty clear that they've been struggling to achieve significant improvements lately by simply scaling up parameter size. Experts like LeCunn have also been vocal that blindly scaling up is a dead end.
(Incidentally, the line of skill improvement isn't "exponential". It's been incremental in improvements per generation, but generations have been coming thick and fast of late, and have grown in parameter count exponentially since 2017.)
Speaking more broadly, LLMs don't have to "hit a wall" in scaling to become uneconomical. If incremental improvement continues to come at exponential cost, however, then the fundamental value argument falls apart.
Setting all that aside, even presuming that model performance scales linearly with dimensionality, there are just fundamental limits to the size of the training corpuses. Quality training data is not unbounded and infinite. Given the same size corpus of training data, there's a hard theoretical limit to how much meaning and inference a model can wring out of it.
And then there are other issues with the whole business model. For one thing, it's predicated on continual full scale retraining to achieve even modest gains in skill and relevancy. Topical and targeted learning requires a full retraining. Etc cetera.
I think that the next generation of AI will lean more heavily on RL to be useful beyond a few months. I also think that the energy requirements of a particular technology are a good proxy to whether it's got a realistic future.
Why do you believe progress is currently exponential? There’s one dubious chart showing “exponential growth” in a single narrow domain, and otherwise zero evidence to suggest exponential improvement.
The curve flattened out years ago. The exponential was going from GPT-2 to GPT-4 (or thereabouts). After that, it was painfully obvious to anyone observing without a vested interest in believing otherwise that the progress had slowed.
Now, it's not just that progress has slowed: it's that the exponential has reversed. In order to get marginal gains, they have to throw exponentially more hardware at the training.
It quite likely won't become a $9T bust though because the investment so far is more like $0.8T. If things slump shortly it might be more like a $0.5T bust if you assume the things are worth 50% of what is being paid - there are a lot of paying users.
$0.18/hr is the (massively) subsidized price of AI services. Once these companies are required to turn a profit for their investors, they'll raise the price. Then the math doesn't look so lopsided. We're already seeing this process unfold with token windows and ad rollout.
It's not that subsidised, this is just wishful thinking. You can run a local model like Qwen for equivalent prices. You might see it go up to $0.50/hr but you're definitely not going to see it at $22
I do run open models locally, but let's not fool ourselves into thinking that they're functionally competitive. I'm extremely skeptical of anybody claiming they've obviated a $22/hr job with an open model. Qwen is a big step down in capability. I can play with something like k2.5 for awhile, but if I want real work done I'm going back to a frontier model, which has significant runtime requirements for inference.
You're also ignoring the cost of purchasing and amortizing dedicated hardware in your local model example.
Inference isn't really that expensive, its the training of new foundational models that is. With whatever highly optimized setup the big providers are using, they should be able to pack quite a lot of concurrent users onto a deployment of a model. Just think too, it's very possible their use case would be served just fine by a 100B model deployed to a $4,000 DGX Spark.
> You and Kurt Vonnegut seem to disagree here. He made liberal use of em dashes and hated semicolons.
Got some samples you would care to share? I'm skeptical we disagree, honestly. One can have preferences for different writing styles, and write in a way that works with preferred punctuation rather than against it.
> The problem with your definition of "good writing" is that it's entirely subjective.
I disagree. Subjectivity is only true to a point. Someone might like Independence Day 2 more than Citizen Kane, but the latter is objectively a better film.
His word choice about semicolons is problematic for other reasons, so I won't quote it here, but Vonnegut made his views on punctuation and story structure very known. An internet search will provide it to you readily. And anybody who's read his works is familiar with his love of the em dash.
But more to his -- and my -- point: He also regularly encouraged people to flout rules and standards. His famous quote about semicolons, when read in its original context, is followed by a sentence with a semicolon!
He was a subversive author who abhorred mindless compliance and begged us to remain inquisitive. Subversion of accepted standards lies at the heart of all creativity. And as creative works enter the broader discourse, they themselves shape new standards. It's why our languages are always changing.
Your point about Citizen Kane and Independence Day 2 is nonsensical, and presumes we all have the same goals when consuming entertainment. I'm not going to engage in that argument.
> An internet search will provide it to you readily.
I'm not interested in doing the work to support your point for you.
> Your point about Citizen Kane and Independence Day 2 is nonsensical, and presumes we all have the same goals when consuming entertainment. I'm not going to engage in that argument.
It's not nonsensical at all, makes no presumptions and makes the point perfectly. You don't want to engage because you are unable to do so, which is absolutely fine.
This is just repackaged cognitive or dialectical behavioral therapy, with cutesy names like "whisper" and "virtue garnish" to make it seem novel. But if it speaks to people, I see no harm in that.
If you're genuinely interested in changing your habits, I recommend investigating these therapies, as they're backed by decades of research and results.
And if you want to tune in to these "whispers" in the first place, there's really no substitute for meditation and mindfulness practice.
I don't know if you've tried Obsidian, but it's just a tool that sits atop your pile 'o Markdown files. No vendor lock-in or special databases. I use Syncthing to sync the files between devices and git to periodically back them up to a private repo.
I've had to be careful to steer clear of all the plugin nonsense that's tempting to dive into as a distraction from actually using the tool, but Obsidian is surprisingly awesome right out of the box.
I use the daily note template tool to generate a structured agenda for each day, which removes the friction that used to keep me from daily journaling and second brain stuff. Now I can't live without it. It's been life-changing for me, as a person previously crippled with ADHD and perpetually living in a state of intense anxiety.
If you have any questions, I'm happy to help out. I was also an Evernote (and Joplin, and...) user for years and was never satisfied until I made a list of my requirements and discovered that Obsidian ticks all the boxes. Haven't turned back since.
Reach out to the NWCDC (https://nwcdc.coop/) or NCBA (https://ncbaclusa.coop/) for resources on how to draft bylaws and structure a worker-owned business that meets your needs. In my experience, they're extremely passionate, knowledgeable and helpful.
I'm a little surprised that there are so many takes like yours in this thread. I don't mean to pick on you, but you seem awfully certain the briber is more ethically in the right here.
I wonder, would you feel the same if this was a meat packer or kindergarten or dairy or hospital who was bribing inspectors?
Building inspections are deadly serious, and when corrupt developers (who, in this case as in all cases, have the capacity to offer life-changing wealth to otherwise underpaid public bureaucrats) pay to cut corners, innocent people die.
It took two to tango, here. The rich asshole paying to circumvent safety regulations is at least as culpable as the motion bureaucrat who accepted the bribe.
> you seem awfully certain the briber is more ethically in the right here
No, I'm not. I don't advocate for bribery and the developer should be charged, although the prosecution's priority should always be on the person accepting the bribe, in my opinion. People are barking up the wrong tree because it's easier for some people to hate a rich guy than someone working for their preferred political party's government.
That being said, how many fewer housing units would exist in SF without this rich asshole bribing his way into making the new housing complexes? And where is the evidence that there was something wrong with the structural plans that was causing problems getting the permit? If you knew anything about SF, you would know that having a perfect plan does not get a building approved. The problem is with the approval process. By being a rich asshole and unethically aiming to help only himself, he objectively improved the city more than most, because the regulatory process he was circumventing is horribly flawed. Can you rectify that with your black and white worldview?
> would you feel the same if this was a meat packer or kindergarten or dairy or hospital who was bribing inspectors?
This isn't a story about a meat packing plant -- this is a story about a city with one of the worst housing shortages in the world, and the worst homelessness crisis in the country.
Some people have it so ingrained in their brains that evading regulation for personal benefit is bad that they fail to evaluate the possibility that the regulation is bad.
Both can be true! One can be more important! Which one is more important can depend on the specific circumstances!
But it really doesn't. What it has is a lot of industry-funded "research" muddying the waters with ambiguous or misleading results.
When I worked at Monsanto, the mantra was that __the burden of proof was on those alleging harm__.
I don't know about you, but I expect a higher standard from the system that regulates the food I put in my body.
Broad spectrum pesticides are terribly complex to test and it should take years if not decades to properly prove them safe for agricultural use and human consumption.
That's absurd. It's a physical impossibility to bring that much power online that quickly. And the cost to get even close would make AI more expensive than just hiring knowledge workers to do the same tasks.
And it's all predicated on a tower of wobbly or broken assumptions -- chief among them that increasing the size of these models yields better performance.
We're going to look back on this era and wonder why anybody took any of the outrageous claims of tech CEOs seriously.
reply