feat(ui): Very basic MVP for viewing a Scorer's logs #3556
Conversation
tssweeney
changed the title
|
Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=c7e916e9e6c17d2661b1ec8bf80cb018dc65930a |
| @@ -283,6 +285,33 @@ const ObjectVersionPageInner: React.FC<{ | |||
| <UserLink userId={objectVersion.userId} includeName /> | |||
| </div> | |||
| )} | |||
| {isScorer && ( | |||
There was a problem hiding this comment.
Baby changes:
- How would you feel about calling it "Scores" and then maybe "View traces"?
- Could we use the hover arrow interaction like we do on the column to the left (since we push to a new page)?
There was a problem hiding this comment.
made the name changes. I suspect this will change very soon anyway maybe with it's own tab or something
| entity={entityName} | ||
| project={projectName} | ||
| neverPeek | ||
| gridFilters={{ |
There was a problem hiding this comment.
Bit of a bigger ask - been looking at cleaning up the filters area.
Would it be possible if we're pushing to the traces view to display an object, to just display the object name with an identifier of what it is?
As an example:
There was a problem hiding this comment.
This will need to be a followup. Incremental change. Don't want to mess with filters here as the intention of this PR is to setup the edge
| @@ -236,7 +238,7 @@ const ObjectVersionPageInner: React.FC<{ | |||
| } | |||
| headerContent={ | |||
| <Tailwind> | |||
| <div className="grid w-full grid-flow-col grid-cols-[auto_auto_auto_1fr] gap-[16px] text-[14px]"> | |||
| <div className="grid w-full grid-flow-col grid-cols-[auto_auto_auto_auto_1fr] gap-[16px] text-[14px]"> | |||
There was a problem hiding this comment.
Now that we have more than a couple columns on all of the objects views we can set grid-cols-auto here.
- Previously we were prescriptive about the column count so that it didn't feel weird having really large spaces between each column.
- Now that we have a delete icon on the farthest right column it will anchor all of the views.
There was a problem hiding this comment.
Uggg i despise Tailwind. Yeah i will fix
|
@tssweeney my proposed refactor of this section #3066 (never merged) contains a builtin scorers page - can we reconcile the two https://github.com/wandb/weave/blob/f0fb79782a1e4f6f1b413c616d1f6b22a793beda/docs/docs/guides/scorers/built-in-scorers.md? It may be the case that none of my changes in 3066 are something you want to apply here, but I think it would be worth looking at |
weave-js/src/components/PagePanelComponents/Home/Browse3/pages/common/Links.tsx
Outdated
Show resolved
Hide resolved
weave/trace/weave_client.py
Outdated
|
|
||
| Returns: | ||
| An iterator of calls. | ||
| """ | ||
| if filter is None: | ||
| filter = CallsFilter() | ||
|
|
||
| # This logic might be pushed down to the server soon, but for now it lives here: | ||
| if scored_by and len(scored_by) > 0: |
There was a problem hiding this comment.
len clause seems redundant since empty list would be falsey
weave/trace/weave_client.py
Outdated
| @@ -859,13 +870,44 @@ def get_calls( | |||
| columns: A list of columns to include in the response. If None, | |||
| all columns are included. Specifying fewer columns may be more performant. | |||
| Some columns are always included: id, project_id, trace_id, op_name, started_at | |||
| scored_by: A list of Scorers to filter by. Multiple scorers are ANDed together. You can | |||
There was a problem hiding this comment.
It would be nice if you could also pass a single string here instead of a list with one item.
It would also be nice if instead of a full ref URL you could specify just :v1 or the digest
A note to reviewers. I will break this up into 3 PRs after I get approval for the purposes of rollout, but i figured it might be easier to review in bulk because everything fits together:
In order to help complete the user flow for scoring, we need a way to actually get the scored calls. The user as a few downstream options: build a dataset, python analytics, view in UI, etc... However, as of right now our data fetch side is limited to just: view the scores on a call... we don't have an easy way to view all cals scored by some metric. This PR puts in place:
a. the backend query layer to fetch calls scored by a particular version of the scorer
b. adds a super lightweight (emphasis on super) to navigate to the calls table for a given scorer
c. adds a very basic query param to the client that allows fetching these calls
This docs page summarizes the work: