[Question] How to correctly export a TD3 model

[Hurdle97]

❓ Question

First of all, thank you very much for the very well-implemented library. I have an already trained TD3 model that I need to export in C. Basically I want to put the model into a controller that, given the observations, gives me the correct action to take. My idea is to export the SB3 model to Pycharm and then, using MATLAB, to C. The point is that I am not able to export it into Pycharm. My model uses the following net_arch = [1000, 500, 400, 300]. I searched in previous discussions and I found some material related to the old PPO2 algorithm only where action_probability(obs) was still present. Can you help me? Thank you in advance! :)

Checklist

• I have checked that there is no similar issue in the repo
• I have read the documentation
• If code there is, it is minimal and working
• If code there is, it is formatted using the markdown code blocks for both code and stack traces.

[araffin]

Hello,
have you read the "export model" section in the doc?

what do you mean by exporting to pycharm?

[Hurdle97]

Thanks for answering. Obviously I made a mistake, I meant PyTorch. Yes, I read the entire documentation #https://readthedocs.org/projects/stable-baselines/downloads/pdf/master/ but there is nothing related to the TD3.

[Hurdle97]

To clarify, I know that I have to use get_parameters function to obtain model parameters and then construct the network
manually in PyTorch assignig the parameters correctly. I also read the old discussion hill-a/stable-baselines#372 where you implement the procedure for PPO2. I tried to reproduce it in order to fit TD3 but it doesn't work.

[araffin]

Yes, I read the entire documentation

This should be the relevant section for you: https://stable-baselines3.readthedocs.io/en/master/guide/export.html#export-to-onnx (SAC is very similar to TD3)

also relevant issue:
#1349 (comment)

onnx in matlab: https://de.mathworks.com/help/deeplearning/ref/importnetworkfromonnx.html

[Hurdle97]

Thank you very much for answering. The links that you gave me were useful. Now I managed to export the model in MATLAB.