As technology improves, it becomes increasingly easy to run AI locally at home.
The hardware companies/data center AI companies eventually fuck each other over because big corporations always fuck everyone else over eventually and it stops becoming profitable.
NVIDIA etc. come crawling back to the consumer market, offering things at reasonable prices again. Data centers sell off all their shit and absolutely flood the market with cheap RAM & graphics cards.
Some other company has started making the hardware in the meantime and the public tells NVIDIA to go fuck themselves and we watch them go into a slow IBM style decline.
Although what will probably happen is the companies will fuck each other over, prices will stay high forever and the data centers that go out of business will just burn all their hardware and all of this will have meant nothing.
I agree with almost all of this, however, much of the RAM during data centre selloffs would be in the SOCAMM format, so not useful for anything consumer nor anything older than a couple years in the enterprise space. If there is such an influx however, I suspect that SOCAMM might become a new format for consumer and enterprise electronics after some time. Other than that, there would also be standard DDR5 DIMMs.
As technology improves, it becomes increasingly easy to run AI locally at home.
Yes, but what ai? Yes, more models will be easier to run at home as hardware increments over time. However training is the expensive part, plus they’re probably expecting that to keep scaling with ever higher end models
Gonna rate my take on this point by point because you’re speaking my language and I like where your head is at
AI will by nature become easier to run. We’re pushing Moore’s law already but technology will in general keep improving. I don’t know how worth it it’ll be, and most companies seem to be into the idea of the average consumer not actively owning the computing power, instead of doing everything on the cloud. Time will tell how that goes.
Big corporations will likely eat each other when the AI bubble pops, whatever that looks like. Should be a fun time though.
NVIDIA will do the same thing over and over. We saw it with crypto, same with AI. They will surrender to whichever monetary source is biggest at the time, and dumpster the loyal customers. If they change for the better, it’s either by accident or because customers lost faith in the company and went elsewhere. I don’t have high hopes for that company.
We are seeing china start to make ram, I vaguely remember seeing another graphics card manufacturer? It’s very possible that there’s a market niche that will be filled here.
It’s much more likely to be like internet fiber. Some was needed/used.
datacenters will always leverage scale, and AI is only economic at 16+ concurrent users. delivers 3x the tokens/s of a single user. Current rental rates for H200s are below their runcosts. Capacity is already too high in US. Innovations for smaller, faster, cheaper models are providing significant value for less hardware. Gemini flash 3.5 is very small and fast, at much lower cost as top 2 US labs. Deepseek v4 has massive cost reductions that will filter down to rest on industry, especially for context compression which is what allows more users on a single GPU cluster. Qwen 3.6 does bring size down enough to run 3-4 month old state of the art models on consumer hardware, but again multi user service at (pro instead of industrial) 96gb ram.
MTP and Turboquant are other technologies that increase tps delivery at less ram. Software stacks making better use of GPUs is eating token demand growth by itself even as exaggerated capacity comes online at slower pace than hardware investment values justified.
Fun fact: the “walkie talkie” was the backpack radio (scr-300) in WWII, and the smaller handheld (scr-536) was the “handie talkie”, but the former became the common phrase phrase used for handheld radios.
I’m thinking data centres might be like landlines. For a while they were the only option, but then mobiles came along.
In today’s context, the mobile is analogous to a fast home computer that can run AI locally.
We might end up with more telephone exchanges than we need.
(for the international audience, mobile = cell phone / handy; landline = inland phone, PSTN.)
Similar analogy can apply to trains etc.
My ideal scenario:
Although what will probably happen is the companies will fuck each other over, prices will stay high forever and the data centers that go out of business will just burn all their hardware and all of this will have meant nothing.
I agree with almost all of this, however, much of the RAM during data centre selloffs would be in the SOCAMM format, so not useful for anything consumer nor anything older than a couple years in the enterprise space. If there is such an influx however, I suspect that SOCAMM might become a new format for consumer and enterprise electronics after some time. Other than that, there would also be standard DDR5 DIMMs.
Oh, Nvidia want’s locally ran ai, but not on your hardware. They want their locked down hardware on your property, using your power and resources.
Yes, but what ai? Yes, more models will be easier to run at home as hardware increments over time. However training is the expensive part, plus they’re probably expecting that to keep scaling with ever higher end models
Gonna rate my take on this point by point because you’re speaking my language and I like where your head is at
AI will by nature become easier to run. We’re pushing Moore’s law already but technology will in general keep improving. I don’t know how worth it it’ll be, and most companies seem to be into the idea of the average consumer not actively owning the computing power, instead of doing everything on the cloud. Time will tell how that goes.
Big corporations will likely eat each other when the AI bubble pops, whatever that looks like. Should be a fun time though.
NVIDIA will do the same thing over and over. We saw it with crypto, same with AI. They will surrender to whichever monetary source is biggest at the time, and dumpster the loyal customers. If they change for the better, it’s either by accident or because customers lost faith in the company and went elsewhere. I don’t have high hopes for that company.
We are seeing china start to make ram, I vaguely remember seeing another graphics card manufacturer? It’s very possible that there’s a market niche that will be filled here.
Prob thinking of the Lisuan LX 7G100, which is a bit under 500 bucks USD with performance comparable to 3060’s.
It’s much more likely to be like internet fiber. Some was needed/used.
datacenters will always leverage scale, and AI is only economic at 16+ concurrent users. delivers 3x the tokens/s of a single user. Current rental rates for H200s are below their runcosts. Capacity is already too high in US. Innovations for smaller, faster, cheaper models are providing significant value for less hardware. Gemini flash 3.5 is very small and fast, at much lower cost as top 2 US labs. Deepseek v4 has massive cost reductions that will filter down to rest on industry, especially for context compression which is what allows more users on a single GPU cluster. Qwen 3.6 does bring size down enough to run 3-4 month old state of the art models on consumer hardware, but again multi user service at (pro instead of industrial) 96gb ram.
MTP and Turboquant are other technologies that increase tps delivery at less ram. Software stacks making better use of GPUs is eating token demand growth by itself even as exaggerated capacity comes online at slower pace than hardware investment values justified.
We don’t have time for a handy, Jim
I mean, it’ll only take 30 seconds.
Who uses handy? I’ve never heard that before. Now I’ve heard it I think we need to use handy and landy.
Germans do. And prolly german-speaking countries like Austria and Switzerland as well.
Yes, I heard that name for them on QI years ago. “Mein Handy” - though not sure if they spell it handy or handie.
Fun fact: the “walkie talkie” was the backpack radio (scr-300) in WWII, and the smaller handheld (scr-536) was the “handie talkie”, but the former became the common phrase phrase used for handheld radios.