You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Benchmark results were achieved by running the open source evaluation repository ShishirPatil/gorilla on commit 7bef000 without any further changes.
I checked out the gorilla repo at this commit id, only found these items in /model_handler/handler_map.py:
"meta-llama/Meta-Llama-3-8B-Instruct": LlamaHandler,
"meta-llama/Meta-Llama-3-70B-Instruct": LlamaHandler,
Hi, I found this section:
https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/eval_details.md#berkeley-function-calling-leaderboard-bfcl
Benchmark results were achieved by running the open source evaluation repository ShishirPatil/gorilla on commit 7bef000 without any further changes.
I checked out the gorilla repo at this commit id, only found these items in /model_handler/handler_map.py:
"meta-llama/Meta-Llama-3-8B-Instruct": LlamaHandler,
"meta-llama/Meta-Llama-3-70B-Instruct": LlamaHandler,
in /eval_checker/eval_runner_helper.py
"meta-llama/Meta-Llama-3-8B-Instruct": [
"Meta-Llama-3-8B-Instruct (Prompt)",
"https://llama.meta.com/llama3",
"Meta",
"Meta Llama 3 Community",
],
"meta-llama/Meta-Llama-3-70B-Instruct": [
"Meta-Llama-3-70B-Instruct (Prompt)",
"https://llama.meta.com/llama3",
"Meta",
"Meta Llama 3 Community",
],
Did you assign model_id as llama3 instead add a new model_id for llama3.1 ?
And thus the model handler using the LlamaHander?
The text was updated successfully, but these errors were encountered: