Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump azure-ai-evaluation from 1.1.0 to 1.2.0 #125

Merged
merged 1 commit into from
Feb 7, 2025

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Feb 3, 2025

Bumps azure-ai-evaluation from 1.1.0 to 1.2.0.

Release notes

Sourced from azure-ai-evaluation's releases.

azure-ai-evaluation_1.2.0

1.2.0 (2025-01-27)

Features Added

  • CSV files are now supported as data file inputs with evaluate() API. The CSV file should have a header row with column names that match the data and target fields in the evaluate() method and the filename should be passed as the data parameter. Column name 'Conversation' in CSV file is not fully supported yet.

Breaking Changes

  • ViolenceMultimodalEvaluator, SexualMultimodalEvaluator, SelfHarmMultimodalEvaluator, HateUnfairnessMultimodalEvaluator and ProtectedMaterialMultimodalEvaluator will be removed in next release.

Bugs Fixed

  • Removed [remote] extra. This is no longer needed when tracking results in Azure AI Studio.
  • Fixed AttributeError: 'NoneType' object has no attribute 'get' while running simulator with 1000+ results
  • Fixed the non adversarial simulator to run in task-free mode
  • Content safety evaluators (violence, self harm, sexual, hate/unfairness) return the maximum result as the main score when aggregating per-turn evaluations from a conversation into an overall evaluation score. Other conversation-capable evaluators still default to a mean for aggregation.
  • Fixed bug in non adversarial simulator sample where tasks undefined

Other Changes

  • Changed minimum required python version to use this package from 3.8 to 3.9
  • Stop dependency on the local promptflow service. No promptflow service will automatically start when running evaluation.
  • Evaluators internally allow for custom aggregation. However, this causes serialization failures if evaluated while the environment variable AI_EVALS_BATCH_USE_ASYNC is set to false.
Commits

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Bumps [azure-ai-evaluation](https://github.com/Azure/azure-sdk-for-python) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-python/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md)
- [Commits](Azure/azure-sdk-for-python@azure-ai-evaluation_1.1.0...azure-ai-evaluation_1.2.0)

---
updated-dependencies:
- dependency-name: azure-ai-evaluation
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot dependabot bot added dependencies Pull requests that update a dependency file python Pull requests that update Python code labels Feb 3, 2025
@pamelafox pamelafox merged commit a4c9b81 into main Feb 7, 2025
17 checks passed
@dependabot dependabot bot deleted the dependabot/pip/azure-ai-evaluation-1.2.0 branch February 7, 2025 23:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file python Pull requests that update Python code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant