51·
3 days agoAnecdotally, I use it a lot and I feel like my responses are better when I’m polite. I have a couple of theories as to why.
-
More tokens in the context window of your question, and a clear separator between ideas in a conversation make it easier for the inference tokenizer to recognize disparate ideas.
-
Higher quality datasets contain american boomer/millennial notions of “politeness” and when responses are structured in kind, they’re more likely to contain tokens from those higher quality datasets.
I haven’t mathematically proven any of this within the llama.cpp tokenizer, but I strongly suspect that I could at least prove a correlation between polite token input and dataset representation output tokens
Yes they were, so I’m offering you an actual theory as to why this may actually be true, yet difficult to “prove”.
Smoking was bad for your health long before anyone sat down and took the time to prove it. Autoregressive LLM tokenizer are a very new field of computer science and it’s going to take a while for the community to collectively understand everything we’re currently doing by trial and error.