Custom callbacks

[alainjungo]

It would be great if the Trainer could support any custom callbacks that follow the Callback structure.

[jeffling]

We've been talking about this for a while. I'm very for this.

One aspect I would like to add to this would be deterministic ordering of callbacks. It can be very difficult to maintain callbacks if the order of callback execution is not pre-defined, we end up having to handle a lot of edge cases for each callback as a result.

[jeremyjordan]

a very simple yet flexible way to achieve deterministic ordering is to use the order in which they're provided.

eg. Trainer(..., callbacks=[custom_callback_1, custom_callback_2, model_checkpoint_callback]) would first call custom_callback_1, then custom_callback_2, etc. for each hook

otherwise, enforcing order seems like a very daunting task.

see keras for an example.

[davidlrobinson]

As @jeremyjordan suggested, using a Keras-style ordered list of custom callbacks would be incredibly useful.

[Borda]

Yeah, after splitting callback in #849 we opened a discussion in #896...
@alainjungo do you have anything else you would add to this topic?
I would close this request in favour of #896 so pls let's continue the discussion there 🤖
btw @davidlrobinson do you remember the issue it was discussed so we can link it here too?