You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As in #72 human baseline. Monitor metrics other than the loss that are human-interpretable and checkable (e.g. accuracy). Whenever possible evaluate your own (human) accuracy and compare to it. Alternatively, annotate the test data twice and for each example treat one annotation as prediction and the second as ground truth.
For this we can only warn user to use metrics different than just loss.
The text was updated successfully, but these errors were encountered:
As in #72
human baseline. Monitor metrics other than the loss that are human-interpretable and checkable (e.g. accuracy). Whenever possible evaluate your own (human) accuracy and compare to it. Alternatively, annotate the test data twice and for each example treat one annotation as prediction and the second as ground truth.
For this we can only warn user to use metrics different than just loss.
The text was updated successfully, but these errors were encountered: