You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We find that LLaMA-3.1-70B-Instruct outputs duplicated brackets at a high frequency. For example,
Since you didn't provide the full error message, I couldn `[ [ [ [ [ [ [ [ [ [ [ [ [ [ []]]]]]]]]]]]]]]]]]]]. If you still encounter issues, please provide the full error message......
We use vllm-0.5.3-post1 as the inference background.
Do anyone has ideas on this behavior? Thanks a lot.
The text was updated successfully, but these errors were encountered:
Hi! May I know your prompt to test this? Can you double check if the 70B model is the latest? The chat template has been changed in this PR, please make sure 70B model's chat template is the latest.
We find that LLaMA-3.1-70B-Instruct outputs duplicated brackets at a high frequency. For example,
We use vllm-0.5.3-post1 as the inference background.
Do anyone has ideas on this behavior? Thanks a lot.
The text was updated successfully, but these errors were encountered: