Companies told us AI would replace human workers and cut costs. Turns out the math doesn’t work. An Nvidia VP just confirmed that compute costs for his team exceed what they pay their employees. Uber burned through its entire 2026 AI budget in four months. And a startup CEO ran up a $113,000 monthly AI bill, with a four-person team.
This episode breaks down exactly why AI is costing companies more than the humans they laid off, what “tokenmaxxing” is and why engineers are doing it, and whether any of this spending is actually returning anything. Because at some point the ROI question stops being awkward and starts being a board-level emergency.
The fun part about this is that tokenmaxxing is basically allowing (even encouraging) employees to waste company resources.
That is their “better” world. Their “better” world encourages their workers to waste their money.
Our company is like that and I don’t have the slightest problem with it.
And we’re still in the “let’s be cheap and try and undercut each other” phase, before the snake eats its own tail. Things only get more expensive from here.
Meanwhile whole careers are in shambles because of these greedy asshats.
What a fucking joke.
We might jump into a new phase of this as tech advances. Other companies are trying to create different ways of running the models which will be substantially cheaper.
For example, one is exploring etching the models directly into the silicon and have built a rapid workflow to go from model to silicon, while another is trying to etch the transformer architecture into the silcon.
If any of these new ideas work, it could really upend things and start another phase of everyone trying to undercut everyone, and also be really bad for the likes of Nvidia.
Edit: Just as an example, the etched model one gets just shy of 17,000 tokens/s on a Llama 3.1 8b model, where a Nvidia H200 gets 230. But how they’re going to scale this up to a more meaningfully sized model I dunno.
Trying to etch models into a chip is a dead end until we reach “peak” quality.
However, unless they include some kind of LoRA (low-rank adaptation) adapter onto the silicon, it severely limits the utility of whatever model or architecture they choose. Being able to modify the weights is way more useful.
Honestly, diffusion decoders are probably where we’ll end up some day. Not end there, but that’s probably the next logical step in the throughput chain.
General purpose compute is infinitely more valuable during times of great software improvements than highly specialized compute.
Things like Tensor Processing Units (TPUs) still aren’t ubiquitous yet, even though they’ve been around for 10+ years. They’re Too specialized to allow for reasonable flexibility on testing.
They claim they can apply Lora’s to it, and that at a data centre scale, it will pay for itself in a year vs existing GPU methods… but who knows if any of that is true.
They’d need a pretty good recycle process set up to get rid of cards that are no longer useful after a couple years as well.
But ya, maybe this is future future, once we have these amazing models that don’t need to be changing often.
Edit: and some models would be better suited for it than others. A creative writing model is less likely to suffer not being updated as frequently as a programming model for example.
If you think it’s expensive now; wait until the AI companies start raising prices to recoup investments. Then it’s gonna be way more expensive than just hiring a guy.
Now imagine if they were actually subject to environmental regulations.
I really think a lot of CEO’s out there thinking “Wow, this is so cheap” and really expect it to stay like that.
With employees, they get to just deny raises and laugh all the way to the bank. But when your ‘agents’ are owned by another company who’s also trying to make a profit, when they jack up the price 20% you don’t get to just be like “oh, times are tough, CEO needed a new yacht, sorry”. And the more you cut out people, the more specialized ‘agents’ you’ll need to purchase because genuine creativity will be gone.
Has AI spontaneously invented\created something that hasn’t existed? Has it been able to take a real working process and find ways to improve it (like, looking at CPU/GPU architecture and improving it)?
Yes, AI has created great things, but most of those AIs are not what people think they are. LLMs are one branch, and the one that gets most hate. But there are others.
For example look at what they: https://leap71.com/ did.
This is the type of AI I’m ok with. This is useful stuff. This is the kind of AI I want to learn how to do.
It always amused me that all paid AI subscriptions didn’t start in this way already.
As an NVIDIA VP I assume he’s rich enough to casually walk away from the mess while tossing a lit match over his shoulder.
Nvidia isn’t actually losing money, they’re selling GPUs to the companies that sell the compute back to them, so the effective outcome is still profit for Nvidia. They seem smart enough to have a down ramp from all this.
Nah, if there’s anything I’ve learned, it’s that 99% of people/companies/businesses follow the “if I made money yesterday, and I made money today, I’ll make money forever” rule. Greed is fundamentally short-sighted.
Sure, especially when they let shareholders have a say, who don’t care about the long-term viability of a company, just short-term profits. They can always sell.
Yet Nvidia isn’t doing things like that as egregiously. The other tech giants fire, they hire.
No shit hunh.
“For now. But trust us when we said it’s gonna be cheaper, better, faster, and bigger in the future.” i bet some CEhOes thinking this.
Fuck AI, of course.
I can imagine them staying the course for the same reason that we stuck with solar panels before they were efficient enough to be practical.That’s the insidious part: they don’t get that solar panels get more efficient as they go, and usually less expensive, as they created that cost/improvement curve we all know is out there.
Ai may never hit that curve. The simplest LLM is so power-starved that we’re several dozen generations of tech below where we’ll even see that efficiency come up.
So does that mean we can make Google burn through money if we use Gemini daily for nonsense (or privacy-preserving/basic queries)?
These exe
