Yeah -- 93 commits in there. Not to say AI can't make an improvement in a domain like this, but it's definitely not a YOLO situation right now. I use AI code heavily at work and it's quite prolific but also often creates bugs of various severities.
Having done a lot of optimizer work, there's definitely a category of things that I've avoided producing preliminary numbers for because I don't see how to get robust production code out of it. A lot of people are going to have to learn (and develop social scripts to explain) that not every 50% speedup is a good idea to pursue.
Well the thing is -- when you have rampant inflation, the cost of everything in USD goes up -- be it houses, eggs, or even stocks. So when you subtract out how much the value of the USD has gone down relative to other currencies it's basically another 10% drop.
Eggs are pretty cheap right now. Housing and healthcare are why Americans struggle with affordability. Food, gas, etc. are all just drops in the bucket.
Eggs may be cheaper than when they were during culling/avian flu, but food sure isn't. Where I am most grocery items are now 50-70% more expensive than last January. The milk I buy weekly at the grocery store has increased 67% since then. Beef is roughly 50% more, and I haven't even had steak since late 2024 at this point. That isn't sustainable.
Why did you go to 'eggs'? -- An avian flu issue, not an inflation issue. We're struggling not just because of healthcare or housing but because everything is being optimized/maximized for value extraction. Variable pricing, rampant plagiarism, counterfeiting, fraud in every industry, in every political office. the stocks are being propped up with the promise of value that has circular investing, monopolies are basically encouraged. Gas and food is relatively benign even with shrinkflation and petrol dollar (not for long) preference.
It's not the biggest expense, but adding another $100 a month to gas when Americans are already relying on debt to cover bills doesn't spell a good omen.
Well, when a companies have 100billion dollar incentives to make discoveries like this, I don't know if we should assume this is the only optimization that will happen.
Given that increasing model size doesn't yield proportional increases in intelligence, there is a world where these datacenters don't have a positive ROI if we make these models even a fraction as effective as the human brain.
I guess I'm trying to understand. I'm hearing this paper has been around for a year -- I would think that many companies would have already implemented and measured its performance in production by now... is that not the case?
Okay, I spent about half an hour reading about this and asking gemini I guess my best understanding is this:
The main breakthrough [rotating by an orthogonal matrix to make important outliers averaged acrossed more dimensions] comes from RaBitQ. Sounds like the RaBitQ team was much more involved, and earlier, and the turbo quant paper very deliberately tries to avoid crediting and acknowledging RaBitQ.
My understanding is that the efficacy of these methods isn't in dispute, what turboquant did was adapt the method that was being used in vector databases and adapted it for transformers, and passed it of more as a new invention than an adaptation.
Well, in the sense that it's possible to be eating zero calories in the time between meals. You still need the meals. If you're just looking at brief snapshots, it doesn't tell you much.
It was a widely considered impossible to have more than single digit percentage of renewables even for instantaneous figures. That "limit" has been raised again and again as the world gets more experience with it.
It's great that we've made so much progress that people can say "it's just 90% renewable for 30 mins" but that's a result of decades of hard work.
You are conflating ideas here and it's getting muddled up. Literally nobody ever said that we couldn't handle 100% renewables for brief periods. The single digit percentage you're referring to is not about the renewables, it's about when the renewables aren't there. Its about maintaining grid stability when you don't have dispatchable sources to do it with. Essentially what we've built is a system where the renewables provide a tertiary function- providing power when they want to, but not in a reliable way, so we still have the same carbon based dispatchable resources.
This is not a serious system. We've done a bit of work on th cheap, easy part. Installing some solar panels is easy and costs almost nothing. The storage and transmission of power is 90% of the actual work!
It is literally written into regulations in many places that renewables aren't allowed to go above certain percentages even for short periods e.g in Ireland they are raising the cap by 5% every so often with an aim to get to 95% by 2035, it's currently 75% and was 50% when they brought in that particular rule.
In Australia they keep reducing the number of synchronous generators that are required to be connected as they gain more experience.
Renewables started as fringe tech and are now a trillion dollar industry. But they faced skepticsims at every stage along the way.
But it would still be an achievement worth sharing if, never in your life before, you thought you could go on a full day without eating, and then you go and do it, wouldn't it? And then you might argue back "oh but it's not an achievement to go a full day without eating" or "Anybody can do it". To which i say "Well that's the problem of the metaphor you made. It's still an achievement and a milestone that we could get to 90%+, even if just briefly.
Not saying these are bad, but I haven't seen any place where these seem to make much difference.
I think the concept that you have a process for peer-review of architectural decisions open to anybody is important -- whether it's in writing, whatever you call it, where it goes I'm not sure those things matter.
After a certain number of years I begin to wonder if architect type roles love ADRs so much because their impact is hard to measure/validate, so they overcompensate or attempt to put a more verbose process on others to try to produce documents for optics reasons.
Well I think a good way to differentiate things that are guilty-pleasures like a twinky and gambling is to take a survey of people and see what % say "I wish I had never ever gambled in the first place" vs "I wish I never had been allowed to buy twinkies"
It'd actually be quite easy to set certain sane limits on gambling like you can't gamble more than 1% of your annual income per year, but I bet gambling platforms would fight that like the plague because those are their whales, the true addicts.
I know 100s of people who've been to vegas and had a good time gambling, not one of them would say "I wish I'd never gambled in the first place". I personally know zero people who gambled so much they regret it. I'm not denying those people exist, but I suspect if you ask everyone, a very small percent have had a strong negative experience
Yes it's a very small % of the population, but it's actually a significant percent of the earnings of these platforms.
Just for illustration -- when I worked at Zynga they'd sell in-game purchases for over $10k USD for virtual items because there were people who just couldn't help themselves, and those "whales" were actually the bulk of the profit of the company.
That's why these platforms should have mandatory sane limits that can only be exempted with special circumstances.
> Well I think a good way to differentiate things that are guilty-pleasures like a twinky and gambling is to take a survey of people and see what % say "I wish I had never ever gambled in the first place" vs "I wish I never had been allowed to buy twinkies"
I don't think this is a fair comparison, because it is much easier to tie losing all your money to gambling than it is to tie your health issues to twinkies. For one, it isn't just twinkies, it is a bunch of different foods, and the consequences are temporally separated from the action; you don't eat a twinkie and immediately notice you are bigger and less healthy. Your heart attack will come years down the line, and there was no one action you took that you can regret, so the feeling is not the same. Gambling is very easy to feel the pain, you lose a bet and you lose the money, immediately.
> interfering in a foreign election should be understood to be grounds for war
Requires a rigorous definition of interference.
The allegations here—trying to catch politicians on tape being sleazy and then releasing them with sketchy editing-doesn’t seem to rise to the level of calling for a kinetic response.
What I do think is that nations should 1) interfere back, and 2) make their citizens more resilient to foreign propaganda. And I specifically don't mean building a firewall. In fact do the opposite: if a firewalled nation is leaking out propaganda, ensure firewall-breaking tools leak in.
Actually, no. The decentralization of power means that it takes a lot more effort to subvert each country individually, rather than propping up a few candidates for the entire region like they do in the US.
The EU is perfectly capable of collaborating even when it can't reach full consensus or when it wants to include peripheral states without them becoming full members. See for example the Schengen area, Eurozone, European Economic Area, and more recently (and specifically to circumvent member state vetos) when the enhanced cooperation procedures were invoked to lend money to Ukraine.
That’s true but that fragmentation is also what limits the propagation of fractures. You can see it like sandboxing.
A deal with foreign intelligence is a dead with the devil that comes with a lifetime of subservience. And subservience to foreign powers is a greater evil than yo usual internal corruption. At least the locally corrupt in a democracy have some interest in things going somewhat well in their country. The foreign actors only care about theirs.
reply