When I was a child, I was spending my summers at my grandparents.
They had a cozy house in the village.
They worked the land, they had a few animals.
They grew their food.
They sold some wine and fruit at the market.
They had a big house, land, clean air, clean water and they were healthy.
They celebrated the holidays, had many friends, went to church, weddings, funerals, etc.
Villagers always greeted them and stopped for a quick chat when they met on the street.
Now compare with life in a big modern city.
I design complicated distributed systems using AI in order to provide shelter and food for myself. Those are tools which other people use to achieve their goal of providing shelter and food for themselves.
Tons of cars, the air is polluted, constant noise, fake bling, restaurants selling food at 20x price, stressed, depressed, lonely people.
Each in their own digital rabbit hole on their phones all the time.
Smiles for money only.
I'm really struggling to understand what we've grown into and why this rat race is considered 'better' than what people have had for millennia without destroying nature in the process.
We're obviously richer, too. Your grandparents had a cozy house - did they have good fresh food all winter when growing up? Could they keep food from going bad in the summer? What about indoor plumbing? These things are so ubiquitous now it's hard to even remember that they aren't just part of the basic fabric of reality.
It's easy to look back with nostalgia (and literal survivor bias - "my ancestors all survived") at the past. But if you actually look at history you will see that "what people have had for millenia" was ... pretty awful. It's an AMAZING time to be alive.
1870 is not a great span of time when OP is comparing to some idyllic village life unencumbered by urbanism. Late 19th century had many people in a “rat race” in the city, like work twelve hour days six or seven days a week in a factory type of dead-end race.
But there was something that happened later:
> For those countries with long-run data in this chart, we can see three distinct periods: From 1870 to 1913 there was a relatively slow decline; then from 1913 to 1938 the decline in hours steepened in the midst of the powerful sociopolitical, technological, and economic changes that took shape with World War I, the Great Depression, and the lead-up to World War II; and then after an uptick in hours during and just after World War II, the decline in hours continued for many countries, albeit at a slower pace and with large differences between countries.
The god knows what “sociopolitical changes” could have been about.
While there was a rat race in the city, typical farmlife was more brutal: work from early childhood and then from dawn to dusk. City dwellers had at least one day off per week, while working with animals is a job without holidays.
The lifestyle you describe is very cheap and you can certainly afford it, if you wanted. There are plenty of rural small towns you could choose from. Why not do it if it's what you think you want?
What are these "rat races"? What makes you think you're doing something worthless? Do you really believe everything is worthless? That's preposterous and nihilistic. If what you are doing is worthless, do something that isn't.
Grandiosity and pride may make you feel unsatisfied, but that's a problem with ego. Our culture is big on empty posturing and hollow spectacle. You don't need to buy into it. Humanity is how it is; perpetually flawed and immature. The main thing you need to escape are your own vices. Your vices are what give foolishness and evil their power over you.
While the secondary purpose of a job is the good it produces, the primary purpose is your growth as a human being. The job, like all other aspects of life, is an occasion to work out your virtue and your humanity in the concrete. Measure your existence according to the objective good of the inner life, not against external things.
Do not try to "immanentize the eschaton". Do not try to locate transcendence in the immanent. That's a major source of much misery. People have an intrinsic yearning for transcendence. When it is misdirected, it results in the hedonistic and ultimately fatal and fruitless hunt for the transcendental within the immanent, of "vertical divinity" within "horizontal creation". Some try to simulate transcendence within the immanent with all sorts of silly gimmicks, but predictably, this always fails. One must locate the transcendence where it actually is; everything is death and superstition. Your job is not the right place to look for it.
There's even places in Spain or Italy where they will give you the house for free, and possibly even cash to move in. Still with these benefits they are not reversing the decline of the country side.
I'd say that is nowadays almost completely just a direct consequence of general population decline, and not some more specific effect; this just hits rural communities harder because attracting new people is already difficult there (=> most job opportunities are elsewhere), and it is much easier to fall below "viability thresholds" (i.e. too small to sustain a general store) than it is for cities.
I'd argue that the "real" urbanization mostly finished in the 1980s or so, and the "urbanization" we see now is mostly incidental (and happens at lower rates, too).
Leading a slower existence in harmony with nature and community has its trade-offs.
There are so many things to consider. It's a fascinating topic. For example, if you give up access to restaurants in order to live simply, how does that impact your approach to food in general?
How about losing access to a hospital? What changes do you make to prepare for, or respond to, health crises?
The questions I ask above are from one direction, and only a sample. I think they're demonstrative of the kind of wide context a decision like this has, though.
> if you give up access to restaurants in order to live simply, how does that impact your approach to food in general?
I think many people would develop a much healthier relationship with food. We live so disconnected from the reality of all the resources and labor it actually takes to bring food to your plate that we've lost appreciation for the interconnected nature of how we live.
Oddly enough, it's the individualist style of home cooking for ourself/only our immediate family that's a departure from the more community-focused lifestyle humans once lived, where cooking and eating involved the entire tribe/community. It was a shared experience.
When people in this thread are nostalgic for a more rural lifestyle or debating rural vs urban, I think that's missing the forest for the trees. What we are really longing for is a sense of community and connection that has been lost. And that community and connection can happen no matter what the actual setting is (urban vs rural). "Where ever you go, there you are."
Living in a big city is usually better for the environment than living in a rural area. City-dwellers live in smaller spaces closer together, so they consume fewer resources and emit less carbon.
That's assuming that city people do the same things as rural people, and the only difference is whether they own a car and a house. City people earn much more money which they spend more freely and use to do more things, which require orders of magnitude more resources, even if they may be slightly more efficient when traveling within city limits (just think about the multiple flights per year every urbanite I know takes, versus the one or two per decade everyone else takes).
Detached single family homes use more energy than apartments per resident[1]. You need more sewer pipe, more road, more wiring to service the same number of people living in detached single family homes.
It's absolutely the worst place to be in any sort of crisis though, whether it's war, pandemic, rioting, natural disaster.
So many people, potentially desparate people, concentrated in one place, utterly dependent on supplies being shipped in from elsewhere.
And we can't have everyone living ever-smaller lives in ever-more-dense cities anyway, as you need all the food production, manufacturing, energy production, and resource extraction to keep those cities alive. And for now, that still requires a lot of human labour (far too much of it overseas, given increasing geopolitical instability)
The only justification would be that we are in some liminal space and are living in the confusing period leading up to some tech utopia.
You also can still live in the way your grandparents did. In the rural USA 10+ acres is going to cost less than a condo in the city. We are just terrified of uncertainty nowadays. Maybe all the technology and distractions keep us from hitting that tipping point of despair/discomfort that would drive us to take risky actions.
> In the rural USA 10+ acres is going to cost less than a condo in the city
Cheap land is cheap for a reason, even in the middle of nowhere. We aren't in the old days of homesteading. Good chance that cheap land has no utility access and it will cost 10s of thousands to bring it in. Another really good chance you don't own the rights to the water beneath it, and spending $30k+ to dig a well that may not even hit a viable aquifer. The land could be cheap because its rocky, acidic, or maybe its prone to flooding.
What OP is also probably missing is that it's extremely difficult to grow enough calories to sustain a human life for a year. Most "homesteaders" actually still buy most of their calories at the store. Fine so long as your new farm (assuming you even can farm on the cheap land) can turn a profit for you, but now you've just traded one rat race for another.
Point being, the rat race is survival, and there is no escaping it. Happiness has to come from within, moving to the middle of nowhere and starting a farm isn't going to magically make your life better. It will probably actually make it worse.
You’re right. Ignore the other comments not recognizing that happiness and health can be had without a constant competition with everyone else. The modern way of life is broken. The old way of lives in many older civilizations are better. We need to stop glorifying our western order and look east.
Depending on how old you are, I might suggest that you were simply not aware of the details of your grandparents' lives at that time. And it just sounds like you have surrounded yourself with assholes in adulthood.
Your problem is that you were happy as a child and unhappy as an adult. That's a personal problem, and has little to do with rural or urban living. As other commenters have pointed out, you can most likely afford to buy a farm if you really want that lifestyle. I suspect you don't, because it sucks major ass to have to do manual labor all day, to eat almost entirely the same things and see the same people for decades on end, and to choke off your access to most of the joy and beauty in the world.
The comment was for the article 'Obsession with growth is destroying nature'. Read it. It's not about me personally.
In fact, I live a very comfortable life in a beautiful city.
But this comfort has a price, described in the article. It's huge.
It's also not static, it's growing and accelerating.
Our children will have to pay these bills in one way or another.
What I do with my life (or you with yours) has zero consequence on this, the process can't be stopped. But it will one day, because math.
> it sucks major ass to have to do manual labor all day, to eat almost entirely the same things and see the same people for decades on end, and to choke off your access to most of the joy and beauty in the world.
2 of these aren't bad things.
1 is a complete non-sequitur
1 is a good recipe for staying active/healthy and non-obese if done in moderation. Farming equipment is an option - you don't have to till 2-3 acres just by hand.
The one thing you neglected to mention (that is a pretty important blocker) is that with farming/country-side living it's pretty hard to leave your kids a legacy outside of leaving them the farm itself, and locking them into the same lifestyle.
Also, hugely increased risks in the 21st century due to global warming.
People from the countryside were forced into cities in Britain as a result of the Enclosure acts, reduced demand for agricultural workers, population rises etc
Revealed preference could just show it was the least bad option in that period
The answer is that most people did not live like what you remember.
For most of the last many thousands of years most people worked very hard for very little and died young of disease and war. If you got injured or sick there was no real health care. Women had a high chance of dying in childbirth. Your kids had a high chance of dying before 10.
That stressful city life is better than how most people in history have lived.
Of course we could make it quite a bit better still. We are nowhere near an optimum. A lot of our wounds are self inflicted: addiction to constant doom scrolling, hedonic treadmill for bling, real estate hyperinflation because we refuse to build enough housing.
But that description of the past is very sugar coated and selective.
boomers didnt want to let go of the welfare state their parents and grandparents had introduced, at the same time they live longer, work less, not nearly had as many kids, and did not effectuate the required productivity growth to offset that.
they will go down in history as the worst generation in modern history, 100% certain of that.
This is a completely nostalgic, one-sided view of the urban vs. rural divide. It's also ignorant of the data on the impact of cities on nature.
Cities have a lower carbon footprint per capita, lower land use per capita, people own fewer cars, use public transportation more often. If everyone lived in a city, nature would be better off.
People live in cities because they are vibrant, they have culture, the arts, intellectuals, innovation, etc. Yes there are areas with high traffic and noise, but there are also quiet neighborhoods where everyone walks everywhere, you can pop into a bar or a cafe on every corner, eat 20 different types of cuisine, go to a book store, go see a show on any night of the week.
Your picture of the friendly villagers might be true in your experience, but in reality a lot of those people are nasty when they encounter any kind of cultural diversity or difference.
Yeah, but unfortunately false nostalgia for subsistence farming is widespread and has traction in the discourse. I guess it's probably because every American who ever suffered from that lifestyle is dead, in other words the same reason that it is now increasingly popular to die from measles.
In reality, the laser-leveled, fully-automated, county-scale factory farm is the only reason anyone on this forum has ever experienced the phenomenon known as "free time".
Farming is harder than people who haven't done it think, and surviving on the production of only your family's property is really, really hard. Source: I grew up in very rural areas, and I've seen what it entails. My grandparents lived through the Great Depression in a farming community on a homestead.
However, I don't think that's the ask, here. You don't have to choose MEGASUPERTROPOLIS or remote solo farm. There's a huge gradient between the two.
It's possible to put a little effort into gardening, share with your community, and massively reduce the overall cost of food while still having free time.
> However, I don't think that's the ask, here. You don't have to choose MEGASUPERTROPOLIS or remote solo farm. There's a huge gradient between the two.
The gradient is where you start to consume a lot of carbon unnecessarily.
It’s an extremely privileged kind of nostalgia. Only the wealthy can romanticize poverty.
The solution to nostalgia for farming is for people to try it.
I used to live in Asheville NC years ago. There were a lot of hippie back to the land types that came there to find themselves. It’s that kind of place. Sometimes they’d decide to become farmers. This usually lasted at best one year. Sometimes it lasted weeks.
There's a book about being the child of back-to-the-land hippies called "Against the Country" that I enjoyed and recommend to all who might be tempted.
Some of the groups around Asheville were "eco villages" that could get rather culty, including the kinds of high control abuses that occur in cults. All groups weren't like this but I recall hearing some crazy stories involving shaming people into giving money to the group (while the higher ups are "more equal" than everyone else) and sexual exploitation.
There was one case where the group leader/organizer ran away with a bunch of cash, and one or two people tried to sue but they'd set up some kind of shell company structure and had conned people into signing things that made it hard to sue or press criminal charges. I think they also left the other members on the hook for a property lease, since the things they'd been conned into signing did that too. It looked a lot like an explicit long con targeting the hippie trust fund kids that would come around. Sex abuse too apparently.
I heard more than one story that mentioned members of the Hells Angels, at least allegedly.
This is another aspect of small insular communities in the good old days that the trads and anarcho-primitivists (left-trads) don't talk about. It still happens in small communities today, but a lot less due to the presence of larger scale nation states with laws and law enforcement. If the sheriff in your tiny rural town rapes you, you could try to bring charges at the state level or at least sue. Worst case you can move, and with a global money system you can sell your home and take the cash with you. Without that larger context, there is no recourse. If your local tribal chief, town sheriff, or cult leader wants to take your stuff, pressure you into polygamy, or rape your kids, there's nothing you can do about it. Back to nature!
Albeit I feel like OP was right on something else: his grandparents weren't heavy consumers, but that transcends city vs rural debates.
I see us modern people, except very old folks being extremely heavy consumers.
Sometime I pay attention to my friends and relatives and how much do they consume.
E.g. I spent 5 days with my mother in December at my grandmas and I've noticed that she just bought stuff non-stop, but her metric is money, not "stuff".
So, e.g., she bought a new pillow for my grandma even though my grandma didn't need/want one (she doesn't use it), bought plenty of plastic toys for her own dog, bought a set of new dishes just because the old ones were old, changed her worn phone leather case for a new one, bought plenty of Christmas lights because she didn't want to dig for the old ones she couldn't quickly find, bought some kind of table hook for purses for herself and her friends, etc, etc.
At the end of the week she didn't even spend 250 euros (her metric), so she doesn't realizes, yet, the amount of borderline useless stuff she bought was major and her ecological impact quite huge, especially for how little to none the payoff or utility is.
I had colleagues in my office, back when I was in the office, that just had Amazon packages coming every single day...And here's a smartphone holder, here's some gadget that keeps your mouse cord, here's a yet a new pedal for the drums, here's a set of pens, here's a rubber duck to talk to when debugging, etc, etc.
I mean, I have even a difficult time pointing out that there's something wrong with any of those items per se in isolation, but when it's a lifestyle of non-stop consistent consumerism I think the trend is worrying.
There's so many things that are so cheap nowadays that it's hard to say "why not?", yet they feed into this endless life style that's toxic for the planet but feeds this neverending more more and more.
> Your picture of the friendly villagers might be true in your experience, but in reality a lot of those people are nasty when they encounter any kind of cultural diversity or difference.
Cities are all the things you describe, including the myopic perspective and the inflexibility on display in this quote.
And thats why city states - which is what effectively those you describe are - eventually collapse into "dark ages" and future generation of people can't understand why
That's only because cities outsource their carbon intensive activities. There is no "divide" here. It's one planet and focusing on the wrong categories has destroyed your ability to reason.
That's simply not true. The per capita emissions account for things that are produced outside of cities like food. The primary sources of emissions from individual human activities are food, energy production, and transportation. High density areas are more efficient at providing people's heating and transportation needs. As far as I know, people in cities don't eat more food than people in other areas, if you control for income/standard of living.
It's not only because cities outsource carbon intensive activities. Sure, there's some of that with farming, mining, etc. that must be done elsewhere. But there's also lots of savings from things like residents walking/biking/using public transit instead of driving, living in more efficient apartments, etc. The suburbs are pretty wasteful, they don't generate anything unique and they just waste more resources.
But you do realize that all the positives are mostly hedonistic ?
Yeah, there are more places to enjoy yourself and have fun, more entertainment.
I'm passionate about going out to clubs, electronic music events, concerts, restaurants, flying around on a plane or driving my car on the endless roads.
All of this is great, but according to TFA and my own experience, we're absolutely shitting on the natural world to have our nice drink or exotic food which will be gone from our system in 12h.
We've 'borrowed' from the future generations to have our fun and I'm not sure it's all worth the price.
Don't project the emptiness of your existence on the rest of us. Cities are the only reason we have orchestras and ballets, vibrant sports leagues, and other things central to family life. I do not want my kids to have to live in a "village" too small to field a brass quintet.
As someone who is sensitive to noise, rural areas bother me far more than urban ones. Traffic is low-frequency and only rarely annoying, but the few times I have lived in rural areas or gone camping, I have been woken up repeatedly by the horrific screeching of birds. Louder, shriller, less predictable than any city noise.
There are birds in cities too, and they are annoying, but they are thankfully drowned out by the cars.
> Cities have a lower carbon footprint per capita, lower land use per capita, people own fewer cars, use public transportation more often. If everyone lived in a city, nature would be better off.
I think that's apples to oranges. If we didn't have cities, we also wouldn't have eight billion people in the world.
A better question for the parent is how do you enforce that vision of everyone living on their 20 acres in harmony with nature? This is not something that capitalism or some other -ism does to us. Your neighbor will have children, these children will have children, and before long, you have a settlement of 50 people on these 20 acres, most certainly no longer living in harmony with nature. At that point, they must build infrastructure. That infrastructure may be feasible to build if they pool their resources with the neighbors. Boom, you have a village, then a town, then a city.
So what's the solution here? Do we forcibly sterilize people? Lock them up if they have children? What's the anti-growth strategy we're actually advocating for?
Because the habitable surface of the planet is less than 100 million square kilometers and only a fraction of that is suitable for subsistence farming. The only reason we can accommodate 8 billion is that the majority of them live in high-density settlements and that food is grown on an industrial scale elsewhere.
This is obviously not a reversible trend. People having close proximity to one another, creating economies of scale where everyone does what they are best at instead of everyone doing everything for themselves is what allows big cities to be possible.
I'm sure all of this was inevitable as there likely hasn't ever been a time where humans were not getting together to form communities when it was beneficial to do so.
So what's the solution here? Do we forcibly sterilize people? Lock them up if they have children? What's the anti-growth strategy we're actually advocating for?
You don't need to. Fertility rate per woman in wealthy countries decreases every year.
> Do we forcibly sterilize people? Lock them up if they have children?
Oh give me a break. Developed countries have had below-replacement fertility rates for decades.
If your goal is to reduce birth rates, we do the things that we already know do that naturally: comprehensive, fact-based sex ed; cheap and easy access to contraceptives; social safety nets and support; etc etc etc.
It's not a mystery and it's not even difficult. You don't need to jump to straight to abhorrent crimes against humanity.
I tried maintaining chat hostory and summary in a 'changes' dir in the repo. Claude creates a md file before commiting (timestamp.md, commit hash doesn't work as filename because rebase/squash).
I had to stop doing this because it greatly slowed down and confuse the model, when it did a repo search and found results in some old md files. Plus token usage went through the roof.
So keeping changes in the open like that in the repo doesn't work.
Not sure how tfa works, but hopefully the model doesn't see that data.
It's very tempting to agree to the 'gambling' part, given that both a jackpot and progress towards the goal in your project will give you a hit of dopamine.
The difference is that in gambling 'the house always wins', but in our case we do make progress towards our goal of conquering the world with our newly minted apps.
The situation where this comparison holds is when vibe coding leads nowhere and you don't accomplish anything but just burn through tokens.
> The difference is that in gambling 'the house always wins', but in our case we do make progress towards our goal of conquering the world with our newly minted apps.
What? Your vibe coded slop is just going to be competing with someone else's vibe coded slop.
The motivations for wanting to make the slop could be commercial profit, or it could be simply you trying to solve a problem for yourself. In either case, the slop is the goal and, if the agent isn't giving you complete trash, you should be converging towards your goal. The gambling analogy doesn't work.
Sounds like you've had too much TV. It really does rot your brain, this is obvious to anybody who doesn't watch TV, but completely imperceptible to those who do.
When you watch television, or television on your computer screen (that makes no difference) you get hypnotized by the tube into a passive state of consumption. Watch people when they watch TV. Watch their slack jawed faces when the commercials stay on and their attention stays glued to the advertisements pitching Alzheimer drugs. Critical though suspended, minds off in space.
unless the book contains instructions on how to do things... then youre just outsourcing thinking to the book right? people have to remember less with the printed word full stop. so whats the difference?
Great work! Obviously the goal of this is not to replace sqlite, but to show that agents can do this today.
That said, I'm a lot more curious about the Harness part ( Bootstrap_Prompt, Agent_Prompt, etc) then I am in what the agents have accomplished. Eg, how can I repeat this myself ? I couldn't find that in the repo...
Bottlenecks. Yes. Company structures these days are not compatible with efficient use of these new AI models.
Software engineers work on Jira tickets, created by product managers and several layers of middle managers.
But the power of recent models is not in working on cogs, their true power is in working on the entire mechanism.
When talking about a piece of software that a company produces, I'll use the analogy of a puzzle.
A human hierarchy (read: company) works on designing the big puzzle at the top and delegating the individual pieces to human engineers. This process goes back and forth between levels in the hierarchy until the whole puzzle slowly emerges.
Until recently, AI could only help on improving the pieces of the puzzle.
Latest models got really good at working on the entire puzzle - big picture and pieces.
This makes human hierarchy obsolete and a bottleneck.
The future seems to be one operator working on the entire puzzle, minus the hierarchy of people.
Of course, it's not just about the software, but streams of information - customer support, bug tickets, testing, changing customer requirements.. but all of these can be handled by AI even today. And it will only get better.
This means different things depending on which angle you look at it - yes, it will mean companies will become obsolete, but also that each employee can become a company.
I’m a pretty big generalist professionally. I’ve done software engineering in a broad category of fields (Game engines, SaaS, OSS, distributed systems, highly polished UX and consumer products), while also having the experience of growing and managing Product and Design teams. I’ve worn a lot of hats over the years.
My most recent role I’m working on a net new product for the company and have basically been given fully agency over this product: from technical, budget, team, process, marketing, branding and positioning.
Give someone experienced like me capital, AI and freedom and you absolutely can build high quality software and a pretty blinding pace.
I’m starting to get the feeling than many folks struggling with adopting or embracing AI well for their job has more to do with their job/company than AI
This gives me a lot of hope for a decentralized future for all kinds of service industries. Why would you go to a big-name accounting firm where the small number of humans can only give you a sliver of attention, when you can go to a one-man shop and get much more of the one human’s attention? Especially if you know that the “work” will be done by the same tools? So many of the barriers to entry in various services - law, accounting, financial advising, etc. - is that you need a team to run even the smallest operation that can generate enough revenue to put food on your table. Perhaps that won’t be the case for long - and the folks that used to be that “team” can branch out and be the captains of their own ships, too.
> The future seems to be one operator working on the entire puzzle, minus the hierarchy of people.
Given the rest of your argument that makes no sense. Why should that one operator exist? If AI is good at big picture and the entire puzzle, I don’t see why that operator shouldn’t be automated away by the AI [company] itself?
It's worth remembering that this is all happening because of video games !
It is highly unlikely that the hardware which makes LLMs possible would have been developed otherwise.
Isn't that amazing ?
Just like internet grew because of p*rn, AI grew because of video games.
Of course, that's just a funny angle.
The way I see it, AI isn't accidental. Its inception has been in the first chips, the Internet, Open Source, Github, ...
AI is not just the neural networks - it's also the data used to train it, the OSes, APIs, the Cloud computing, the data centers, the scalable architectures.. everything we've been working on over the last decades was inevitably leading us to this.
And even before the chips, it was the maths, the physics ..
Singularity it seems, is inevitable and it was inevitable for longer than we can remember.
Remember that games are just simulations. Physics, light, sound, object boundaries - it not real, just a rough simulation of the real thing.
You can say that ML/AI/LLM's are also just very distilled simulations. Except they simulate text, speech, images, and some other niche models. It is still very rough around the edges - meaning that even though it seems intelligent, we know it doesn't really have intelligence, emotions and intentions.
Just as game simulations are 100% biased towards what the game developers, writers and artists had in mind, AI is also constrained to the dataset they were trained on.
I think it's a bit hard to say that this is definitively true: people have always been interested in running linear algebra on computers. In the absence of NVIDIA some other company would likely have found a different industry and sold linear algebra processing hardware to them!
It's pretty interesting that consumer GPUs started to really be a thing in the early 90s and the first Bitcoin GPU miner was around 2011. That's only 20 years. That caused a GPU and asic gold rush. The major breakthroughs around LLMs started to snowball in the academic scene right around that time. It's been a crazy and relatively quick ride in the grand scheme of things. Even this silicone shortage will pass and we'll look back on this time as quaint.
Of course you are right, but in addition they wouldn't have even made them if GPUs hadn't made ML on CPU so relatively incapable. Competition drives a lot of these decisions, not just raw performance.
I'm not missing the point. If you recall your computer architecture class there are many vector processing architectures out there. Long before there was nvidia the world's largest and most expensive computers were vector processors. It's inaccurate to say "gaming built SIMD".
You are missing the point - it's an economic point. Very little R&D was put into said processors. The scale wasn't there. The software stack wasn't there (because the scale wasn't there).
No one is suggesting gaming chips were the first time someone thought of such an architecture or built a chip with such an architecture. They are suggesting the gaming industry produced the required scale to actually do all the work which lead to that hardware and software being really good, and useful for other purposes. In chip world, scale matters a lot.
The Cray-1, which produced half a billion USD in revenue in today's dollars, at a time when computing was still science fiction, did not demonstrate scale? I just can't take you in good faith because there has never been a time when large scale SIMD computing was not advanced by commercial interests.
In this context scale = enough units/revenue to spread fixed costs.
I'll take your word on lifetime revenue numbers for Cray 1.
So yes, in todays dollars, $500 million of lifetime revenue - maybe 60-70 million per year, todays dollars - is not even close to the scale we are seeing today. Even 10 years ago Nvidia was doing ~$5 billion per year (almost 100x your number) and AMD a few bill(another 60-70x ish)
Even if you meant $500m in annual (instead of lifetime), Nvidia was 10x that in 2015. And AMDs GPU revenue which was a few billion that year, so it's more like 17x.
That's a large difference in scale. At the low end 17x and at the high end 170x. Gaming drove that scale. Gaming drove Nvidia to have enough to spend on CUDA. Gaming drove NVidia to have enough to produce chip designs optimized for other types of workloads. CUDA enabled ML work that wasn't possible before. That drove Google to realize they needed to move away from ML on CPU if they wanted to be competitive.
You don't need any faith, just understand the history and how competition drives behavior.
Google DeepMind can trace part of it's evolution back to a playtester for the video game Syndicate who saw an opportunity to improve the AI of game NPCs.
Sci-fi ideas of robots have been around for ages and work on AI and the term singularity kicked off around 1950, so a while ago - well before chips or me being born.
One thing that could happen is that someone might decide to vibe code a Discord clone, without all the extra crap. I'm sure there are people out there doing this already.
There's this interesting arc of growth for apps which are successful.
At first users love it, company grows, founders get rich, they hire expensive people to develop the product and increase revenue until eventually the initial culture and mission is replaced by internal politics and processes.
Software starts getting features which users don't want or need, side effects of the company size and their Q4 roadmap to 'optimize' revenue|engagement|profits|growth|...
Users become tools in the hands of the app they initially used as a tool.
This model worked well so far and built some of the biggest companies in history.
AI could make this business model less effective. Once a piece of software becomes successful and veers off into crap territory, people will start cloning it, keeping only the features that made that software successful initially. Companies who try to strong arm their users will see users jump ship, or rather, de-board on islands.
There's some irony in the fact that LLMs are in large part possible because of open source software.
From the tools which were used to design and develop the models (programming languages, libraries) to the operating systems running them to the databases used for storing training data .. plus of course they were trained mostly on open source code.
If OSS didn't exist, it's highly unlikely that LLMs would have been built.
Not really. This db allows traversing the (deeply nested) data structures without loading them into memory. Eg. In Clojure you can do
```
(get-in db [:people "john" :address :city])
```
Where `:people` is a key in a huge (larger than memory) map. This database will only touch the referenced nodes when traversing, without loading the whole thing into memory.
So the 'query language' is actually your programming language. To the programmer this database looks like an in-memory data structure, when in fact it's efficiently reading data from the disk. Plus immutability of course (meaning you can go back in history).
They had a big house, land, clean air, clean water and they were healthy. They celebrated the holidays, had many friends, went to church, weddings, funerals, etc. Villagers always greeted them and stopped for a quick chat when they met on the street.
Now compare with life in a big modern city.
I design complicated distributed systems using AI in order to provide shelter and food for myself. Those are tools which other people use to achieve their goal of providing shelter and food for themselves.
Tons of cars, the air is polluted, constant noise, fake bling, restaurants selling food at 20x price, stressed, depressed, lonely people. Each in their own digital rabbit hole on their phones all the time. Smiles for money only.
I'm really struggling to understand what we've grown into and why this rat race is considered 'better' than what people have had for millennia without destroying nature in the process.