[Bug]: Wrong Response with Gemma2 with 8k context length

[hahmad2008]

serving gemma2-9b on 4 GPUs with 8k context length. I disabled the sliding window and enable flashinfer backend, however when I send a long prompt I got weird responses with repeated same first sentences!!

example:

• prompt is to answer a question given a paper content.

• response:

"content": "\n\nThe paper you'\n\nLet me know if you'\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper.\n\nLet me know if you have any other questions about the paper"

How to solve this issue?