Excellent question. I have absolutely NO idea. Friend helped me set one up on my PC and it even churns through “tokens” on my GPU. So I have zero clue. :D I guess I could ask “AI”. lol
Tokens are what an LLM actually predicts, one after another. If you have very slow models that produce less than 5 tokens/s or so, you can easily follow it with your own eyes. A token is what appears at once. Often it is an entire word but it can also be parts of a word or individual letters, digits, special signs for uncommone words or special formatting, number stuff.
What is a token, anyway? I’ve never seen any mention of them with GPT.
Excellent question. I have absolutely NO idea. Friend helped me set one up on my PC and it even churns through “tokens” on my GPU. So I have zero clue. :D I guess I could ask “AI”. lol
Tokens are what an LLM actually predicts, one after another. If you have very slow models that produce less than 5 tokens/s or so, you can easily follow it with your own eyes. A token is what appears at once. Often it is an entire word but it can also be parts of a word or individual letters, digits, special signs for uncommone words or special formatting, number stuff.