Log probability generation #9500
-
|
I am using curl requests to display the log probabilities of the next tokens. Below is part of the raw response I get from the curl request. Here, the content is 4, but the token with the highest probability (of 0.419) is 2, not 4. Can someone explain why this is happening? Shouldn't the content that is outputted be the same as the token with the highest log probability?
|
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
No - this would be the case only with the "greedy" sampling strategy. For example, if you set the temperature in your request to <= 0.0, it will always choose the highest-probability token. But by default, the temperature is positive so there is proportional chance to select some of the lower-probability tokens. |
Beta Was this translation helpful? Give feedback.
No - this would be the case only with the "greedy" sampling strategy. For example, if you set the temperature in your request to <= 0.0, it will always choose the highest-probability token. But by default, the temperature is positive so there is proportional chance to select some of the lower-probability tokens.