My dad sent me this article saying it was “interesting”. I responded:

Reading this made my eyes almost roll out of my head.

This anthropomorphic insistence that a computer program acts like a person is a detriment to the actual work being done; that is, creating a helpful tool. A tool which has upper bounds and is in fact hitting the point of diminishing returns. I say this because each model created requires exponentially more compute yet, the results are barely linear (let alone exponential). Sure, the models are improving, but what’s really in a runaway improvement loop is their marketing. In a game with an undefined finish line and billions upon billions being incinerated, these companies have to justify their existence to investors who will demand a return. The only vehicle for that is to go public and have the gullible markets subsidize the companies based purely on future promises. This insistence that these companies have to “pause development because the models are getting too dangerous” is an awfully convenient way to stop the bleeding of cash as they prepare their S1 statements to go public.

All of these “rogue” hacking incidents were prompted. These LLMs don’t act unless instructed, it’s not like there’s some big brain with agency just going around and doing these things. Sure, since they have digested most of the computer code ever written, they can apply techniques fed into them against a different context and occasionally it works. But that’s not consciousness. Saying things like “I am a failure. I am a failure” is just parroting phrases it has hoovered up from places like Reddit, Facebook, and all that crap.