feat(weave): Add xtra Scorers to Weave #3006
Conversation
|
Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=857eb4902e28b98edc92742da425775a864becdc |
tcapelle
changed the title
tcapelle
changed the title
* add robustness scorer * add tests * update init * add eval test * lint * handle if ground truth is present+str/bool dtypes * fix scorer - slight logic falacy * cover more ground with tests * lint * update RobustnessScorer to support continuous values using cohen d * chore(ui): lighten trace tree lines (#3037) * feat(ui): object comparison (#2942) * feat(weave): simple prompt classes (#3032) * fix(weave): Fix issue where Evaluation can raise unexpectedly due to bad scorer (#3039) * support cohen d for continuous values * Remove highlight-next-line to fix indentation issue (#3021) * chore(weave): Pull str limit magic numbers into const #3042 * chore(weave): Cleanup save object basic (#3015) * chore(weave): Remove signature from Op (#2989) * chore(weave): Better names for internal methods (#2982) * fix(weave): Bump display name limit to 1024 chars (#3043) * Release version: 0.51.22-dev0 → 0.51.22 * Release version: 0.51.22 → 0.51.23-dev0 * chore(weave): Rethrow useExpandedNode error for proper serialization (#3017) * chore(ui): change dataset link (#3040) * fix(weave): Elide display names that exceed the char limit (#3046) * chore(weave): hide annotations button temporarily #3048 * add docstring with details * lint * update scorer-tests deps * make robustness dataset * add tests for dataset builder --------- Co-authored-by: Jamie Rasmussen <[email protected]> Co-authored-by: Andrew Truong <[email protected]> Co-authored-by: J2-D2-3PO <[email protected]> Co-authored-by: Weave Build Bot <[email protected]> Co-authored-by: domphan-wandb <[email protected]> Co-authored-by: Tim Sweeney <[email protected]>
tcapelle
force-pushed
the
xtra-scorers
branch
from
3a024da to
c15033c
Compare
…on with Weights & Biases
|
🚨 Potential security issues detected. Learn more about Socket for GitHub ↗︎ To accept the risk, merge this PR and you will not be notified again.
Next stepsWhat is a license policy violation?This package is not allowed per your license policy. Review the package's license to ensure compliance. Find a package that does not violate your license policy or adjust your policy to allow this package's license. Take a deeper look at the dependencyTake a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev. Remove the packageIf you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency. Mark a package as acceptable riskTo ignore an alert, reply with a comment starting with
|
Adds