Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #2
This PR proposes a way to make the adjoint method work with tensorflow custom_gradient interface. The main changes are in
tfdiffeq/adjoint.py
and can be summarized as:OdeintAdjointMethod
function. We instead get these parameters from thevariables
keyword argument ofgrad
function.tf.custom_gradient
requiresgrad
function to return two sets of gradients as a pair. These arei. The gradient with respect to the inputs of
OdeintAdjointMethod
. These arex0
andt
in our case.ii. The gradient with respect to the parameters which are
tf.Variable
objects stored in our ODE object.tf.Variable
objects created in adams optimizer, we mark them as non-trainable. However, there still seemsCaveats: I wasn't able to make the method work with the adams method (therefore adams - adjoint test is not enabled either). The problem is that the elements of the tuple returned from
augmented_dynamics
function have different shapes, and this causes problems withadams.py:138