How do you evaluate model performance?

[kanodiaayush]

The API as I can see, gives predictions. Does it give also the probabilties of each class? I would like to access these to evaluate log likelihood/perplexity metrics.

[OKUA1]

The model does not provide probabilities, so calculating log-likelihood is not really possible.

[arbiestoo]

Is it possible to calculate or return the sentiment score? Thank you

I just done with scikit-llm classifier.

Reviews text:
"The food was delicious and the service was excellent. A wonderful dining experience!"

Well done with ZeroShotGPTClassifier:
"Positive"

Well done with MultiLabelZeroShotGPTClassifier:
["Food", "Service"]

Well done also with MultiLabelZeroShotGPTClassifier:
["Positive Food", "Positive Service"]

I need more with the score of sentiment like this:
["Positive (0.82) Food", "Positive (0.75) Service"]

Another example:
Review text: "The prices were a bit high, but the food quality were excellent.",
["Negative (-0.76) Price", "Positive (0.85) Food"]

[arbiestoo]

Update:

With a few modifications to the prompt and label validation, I got the expected output. Thanks
[
['predicted_category': 'Positive Food', 'probability_value': 0.7, 'sentiment_score': 0.8],
['predicted_category': 'Positive Service', 'probability_value': 0.65, 'sentiment_score': 0.7]
]

[kanodiaayush]

Okay, thank you!