sym::Factord optimization from custom Hessian Func

[mariusud]

Hi, I have a question regarding how the sym::Factord is optimized. I am creating a custom factor with the hessian functor, but I'm unsure how the actual factor is optimized when I create it this way (do you have some documentation on this?). I have calculated a new Hessian and rhs after a set of operations, but I do not want to re-calculate jacobian and residual as that seems redundant. I see that I have to include *res and *jac in the functor, but are they being used in the optimization or can i just set them to zero? I guess my question boils down to whether or not the optimizer only uses the hessian and rhs directly when optimizing a factor, or also uses the residual/jacobian somehow.

auto hessian_func = [H = Eigen::MatrixXd(H), b = Eigen::VectorXd(b), rhs_input = Eigen::VectorXd(rhs_input), J = Eigen::MatrixXd(J)]( const sym::Valuesd& state, const std::vector<sym::index_entry_t>& keys, Eigen::VectorXd* res, Eigen::MatrixXd* jac, Eigen::MatrixXd* hess, Eigen::VectorXd* rhs) { *res = Eigen::VectorXd::Zero(rhs_input.size()); *jac = Eigen::MatrixXd::Zero(H.rows(), rhs_input.size()); *hess = H; *rhs = rhs_input; return sym::Factord(hessian_func, keys, keys);

[aaron-skydio]

You do need the residual - the optimizer needs the actual cost, which is computed from the residual, for things like deciding to accept steps or not. The jacobian is unused though, so you can safely set that to zero. (It's there because it's desired in some cases, and we don't have an option to skip it completely in sym::Factord because it's generally free to compute as part of computing the Hessian)