You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The usual scenario on the python end for float16 inference is as follows:
We first train a model in float32 mode
We use a float16_transpiler to modify the program desc and generate new float16 weights in the scope
We run the inference engine in float16 mode to validate the correctness
Use can save the program desc and the float16 weights to be used next time either on the python or c++ end.
2. We convert the float32 weights into float16, and save them on disk 3. During inference, we load the float16 weights and model, run inference engine in float16 mode.
We want to have example code on the python side to demonstrate the above steps.
The text was updated successfully, but these errors were encountered:
The usual scenario on the python end for float16 inference is as follows:
2. We convert the float32 weights into float16, and save them on disk3. During inference, we load the float16 weights and model, run inference engine in float16 mode.We want to have example code on the python side to demonstrate the above steps.
The text was updated successfully, but these errors were encountered: