Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[evaluation] Add support for using evaluate() with evaluators that have missing inputs #38299

Merged

Conversation

diondrapeck
Copy link
Contributor

Description

Cherry-pick of #38276

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

Sorry, something went wrong.

…ve missing inputs (#38276)

* Update evaluate to allow optional params + add tests

* Record tests

* Exclude optional params from 'missing inputs' warning

* Add tests

* Record tests

* Fix linting errors

* Merge from main and fix linting error

* Add unit test for missing output

* Update convo test to assert per turn results

* Run linting

* Add test data

* Re-record test
@diondrapeck diondrapeck requested a review from a team as a code owner November 4, 2024 16:08
@github-actions github-actions bot added the Evaluation Issues related to the client library for Azure AI Evaluation label Nov 4, 2024
@azure-sdk
Copy link
Collaborator

API change check

API changes are not detected in this pull request.

@diondrapeck diondrapeck merged commit 88f13fd into release/azure-ai-evaluation/1.0.0 Nov 4, 2024
23 checks passed
@diondrapeck diondrapeck deleted the evaluate-api-missing-input branch November 4, 2024 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Evaluation Issues related to the client library for Azure AI Evaluation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants