No? If everyone who uses LLMs globally switched over to a local LLM (after buying the necessary hardware), that’d still be a crazy amount of energy usage, just less centralised.
What I’m saying is that not all local LLMs are more power efficient, and that we are currently massively over-using the technology for things it isn’t all that useful for.
If you are browsing Piefed you are probably using about 10-20% of your computer’s resources (if it is a computer capable of running a 4B model).
When you question the model and it pegs your system out, that is between 5 and 10 TIMES more resource utilization and power waste just to have a question answered that you could have taken the time to research, improve your own critical thinking skills, and learned something without changing your utilization at all.
Megacorps don’t burn resources for the shake of it. They do it to train/run LLMS.
If all that running/training is done by the individual instead, the individual is going to be the one burning the resources.
In fact, it would probably be more resource intensive.
Since centralization and scale usually improves efficiency.
Most people don’t have computer hardware that was built with the sole purpose of training/running LLMs. LLM data centers do. Purpose-built hardware is more efficient than generic hardware.
For counting letters in a word? That’s like a one line expression in Python or JavaScript (and other languages). JavaScript console is right there in your browser.
Except for local LLMS.
No? If everyone who uses LLMs globally switched over to a local LLM (after buying the necessary hardware), that’d still be a crazy amount of energy usage, just less centralised.
More energy actually.
nah like using the current Gaming PC/Laptop you have, Like Intel CPU + intergrated graphics or smth.
I suppose, but that’s a very different class of model. I think a more important question might be whether people actually need LLMs at all.
this is personal preference ig.
What I’m saying is that not all local LLMs are more power efficient, and that we are currently massively over-using the technology for things it isn’t all that useful for.
i see
Good to see so many people misunderstanding how local hosting works.
Running your own local LLM is no worse than running a graphically intensive game like cyberpunk 2077 or red dead redemption 2.
Yes. But compared to the number of people who currently use LLMs, not that many have hardware capable of doing that.
thats what i was thinking
No one in this post about Google is talking about local LLMs.
i was trying to say Local LLMS dont destroy the environment(cause its using the power of your pc instead)
Training the models still uses a vast amount of resources
well atleast every prompt you ask it wont harm the environment
If you are browsing Piefed you are probably using about 10-20% of your computer’s resources (if it is a computer capable of running a 4B model).
When you question the model and it pegs your system out, that is between 5 and 10 TIMES more resource utilization and power waste just to have a question answered that you could have taken the time to research, improve your own critical thinking skills, and learned something without changing your utilization at all.
Not counting the training debt.
i forgot about critical thinking, hehe.
but i hope my use of AI doesnt ruin it that much.
btw the best respones here.
Megacorps don’t burn resources for the shake of it. They do it to train/run LLMS.
If all that running/training is done by the individual instead, the individual is going to be the one burning the resources.
In fact, it would probably be more resource intensive.
Since centralization and scale usually improves efficiency.
Most people don’t have computer hardware that was built with the sole purpose of training/running LLMs. LLM data centers do. Purpose-built hardware is more efficient than generic hardware.
“I’m not the one deforesting the amazon but I really love teak furniture, so I’M not the one hurting anybody by buying it”
Your computer uses more power when the GPU is “thinking” than when its idle, so there would still be environmental damage.
Yeah, but Google search isn’t using local models.
sadly true
deleted by creator
There are so many better tools for this kind of thing.
like what?
plus i tend to use a search engine more often if i can find what i want.
For counting letters in a word? That’s like a one line expression in Python or JavaScript (and other languages). JavaScript console is right there in your browser.
if javascript can do it,tell me how to do it?
const countOfT = (word) => word.split('').filter(i => i === 't').length countOfT('colonialism') 0 countOfT('this is an example sentence') 2There’s probably a more elegant way to do it.
nice to know that does not need AI ig