You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently MTEB does not consistently use a ground truth for vector manipulation. A solution would be to adopt the Array API spec. A potentially simpler solution is to use either torch or numpy.
The benefit of using numpy is that it is very stable and rarely leads to OOM and related GPU issues.
The benefit of using PyTorch is that it potentially allows evaluation using GPUs (currently, sklearn does not support this, but might in the future, when they adopt they API spec)
Good point; I think using numpy would also allow us to make torch an optional dependency which could be nice as I assume some models do not require it so people have to install it just for mteb
Currently MTEB does not consistently use a ground truth for vector manipulation. A solution would be to adopt the Array API spec. A potentially simpler solution is to use either torch or numpy.
partly related to the discussion in: #883
This is not a pressing concern atm.
The text was updated successfully, but these errors were encountered: