How to upload your model performance to MTEB leaderboard

[archersama]

1.Evaluating Your Model Using the MTEB Package

To evaluate your model using the MTEB package, follow the steps below:

import mteb
import logging
from sentence_transformers import SentenceTransformer
from mteb import MTEB

logger = logging.getLogger(__name__)

model_name = 'intfloat/e5-base-v2'
model = SentenceTransformer(model_name)
tasks = mteb.get_tasks(
    tasks=[
        "AppsRetrieval",
        "CodeFeedbackMT",
        "CodeFeedbackST",
        "CodeTransOceanContest",
        "CodeTransOceanDL",
        "CosQA",
        "SyntheticText2SQL",
        "StackOverflowQA",
        "COIRCodeSearchNetRetrieval",
        "CodeSearchNetCCRetrieval",
    ]
)
evaluation = MTEB(tasks=tasks)
results = evaluation.run(
    model=model,
    overwrite_results=True
)
print(results)

2. Uploading Your Results to the Leaderboard

To upload your results to the leaderboard, please follow the instructions provided in the following guide: Adding a Model to the MTEB Leaderboard