You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The implementation of log method in KTOTrainer uses wrong prefix eval/ instead of eval_ (similar with train metrics).
This results in names of metrics not being properly processed by logging callbacks (I tested on WandbCallback). The problem is specifically with function rewrite_logs, which expects correct prefixes. This results in metric names in wandb like train/train/rewards/margins or train/eval/rewards/margins (see picture below). If you would like I can submit a pull request for this issue since the fix should be pretty trivial?
The implementation of log method in
KTOTrainer
uses wrong prefixeval/
instead ofeval_
(similar with train metrics).This results in names of metrics not being properly processed by logging callbacks (I tested on
WandbCallback
). The problem is specifically with function rewrite_logs, which expects correct prefixes. This results in metric names in wandb liketrain/train/rewards/margins
ortrain/eval/rewards/margins
(see picture below). If you would like I can submit a pull request for this issue since the fix should be pretty trivial?The text was updated successfully, but these errors were encountered: