Can you EVER create a fully-functioning (ie non-gimped) LLM that will NEVER encourage a user to kill themselves?
Not realistically: no matter how unlikely a particular result, the more you repeat an experiment, the lower the cumulative probability of never getting that result.
If a model can at all produce a combination of words that a suicidal mind looking for confirmation could possibly interpret as encouragement, the only way to make sure that never happens is to never use it.
People want a free-wheeling chatbot that can adapt to their lives and personality and data, so, short of resetting their context window regularly, I don’t know how to keep guard rails in place through hundreds of hours of conversation.
If you keep resetting the context window, you’ll also lose the adaption. You’d need to prune the context window down to the important stuff, but how would you know what is and isn’t important?
quis custodiet ipsos custodes?
We need custodes custodium. Let’s just solve the problem by infinite recursion.
More seriously, there is no such thing as a perfectly safe system. The question is what level of (un-)safety is acceptable, and what level of effort can be justified by those security objectives.
Not realistically: no matter how unlikely a particular result, the more you repeat an experiment, the lower the cumulative probability of never getting that result.
If a model can at all produce a combination of words that a suicidal mind looking for confirmation could possibly interpret as encouragement, the only way to make sure that never happens is to never use it.
If you keep resetting the context window, you’ll also lose the adaption. You’d need to prune the context window down to the important stuff, but how would you know what is and isn’t important?
We need custodes custodium. Let’s just solve the problem by infinite recursion.
More seriously, there is no such thing as a perfectly safe system. The question is what level of (un-)safety is acceptable, and what level of effort can be justified by those security objectives.