Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Explain Missing Documents to All Rated Documents #406

Closed
mkr opened this issue Sep 10, 2021 · 3 comments · Fixed by #407
Closed

Change Explain Missing Documents to All Rated Documents #406

mkr opened this issue Sep 10, 2021 · 3 comments · Fixed by #407
Assignees

Comments

@mkr
Copy link
Contributor

mkr commented Sep 10, 2021

Is your feature request related to a problem? Please describe.
I have difficulties understanding the "Explain Missing Documents" feature. Does it list documents that match the query but are not in the top n list of documents? Or documents that should match the query but for some reason don't match at all? Or documents that were previously rated for this query but are no longer part of the result? And where is the "explanation"?

Describe the solution you'd like
Split "Explain Missing Documents" into
a) "All rated documents": a feature to see all documents rated for this query whether they match the query or not. It would be good to clearly indicate in that list whether a document actually matched the query or not
b) "Missing documents": a feature where a user can enter a query get a list of documents back and somehow find out why a certain document does not match the query.
I don't necessarily need b) but a) would be good to have.

Describe alternatives you've considered

Additional context

@epugh epugh self-assigned this Sep 10, 2021
@epugh
Copy link
Member

epugh commented Sep 10, 2021

image

Thoughts on this title? I don't love the explanatory text, maybe too verbose?

@epugh
Copy link
Member

epugh commented Sep 10, 2021

Wow, that text is really not great... Ideas @mkr or @DmitryKey ????

@epugh epugh linked a pull request Sep 10, 2021 that will close this issue
@DmitryKey
Copy link
Contributor

I think "Explain Missing Documents" might be indeed a little overloaded. From @mkr 's suggestion and from @epugh 's screenshot, I see two functions for it:

  1. Show documents that were previously rated for this query, but are currently missing. When can this realistically happen? One reason is data drift: if we refresh the data in our search engine, Quepid will get new results and might not include the previously rated documents (say more fresh docs appear at the top).

  2. Actively identify and add a document that should also match this query semantically, but doesn't. In this case the scorer should account for this (perhaps even in a way controlled by the user?) and lower the score. I think the "Explain Missing Documents" button has been designed with this in mind, and so the name for it should be what it allows to do ("Actively identify and add a missing document"). So, we can rename this button into: "Identify and rate a missing document". Then for @mkr 's request we need a logical switch: a) see all the rated documents that do match this query right now b) see all the rated documents, including the currently missing ones -- in this case the combined set of documents should have distinctive labels (matching / not matching). Both button and the switch can co-exist and be used independently.

Thoughts are welcome :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants