Listing PyTorch Models in AWS Marketplace using Torch Serve

[rickcao-aws]

Authored-by: Rick Cao [email protected]

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[aaronmarkham]

Please do not upgrade people's kernel with SageMaker v2 as this will make all other v1 notebooks fail.

If you require v2, please upgrade to it, then roll it back in the notebook's cleanup section.

Reply via ReviewNB

[aaronmarkham]

I see there's a lot of setup in here involving building a torchserve container. Why not just use one that's already built for this?
https://github.com/aws/sagemaker-pytorch-inference-toolkit

[rickcao-aws]

We use the raw torchserve because our sellers may need to write their own handlers for special models. Our notebook serves an end-to-end template for them to publish the product.

The sagemaker-pytorch-inference-toolkit is cool but it also requires sellers to dig into additional information. Our notebook set-up requires least knowledge burden on sellers. Also we have a blog for the instruction as well.

[rickcao-aws]

I see there's a lot of setup in here involving building a torchserve container. Why not just use one that's already built for this?
https://github.com/aws/sagemaker-pytorch-inference-toolkit

We use the raw torchserve because our sellers may need to write their own handlers for special models. Our notebook serves an end-to-end template for them to publish the product.

The sagemaker-pytorch-inference-toolkit is cool but it also requires sellers to dig into additional information. Our notebook set-up requires least knowledge burden on sellers. Also we have a blog for the instruction as well.

[aaronmarkham]

I see there's a lot of setup in here involving building a torchserve container. Why not just use one that's already built for this?
https://github.com/aws/sagemaker-pytorch-inference-toolkit

We use the raw torchserve because our sellers may need to write their own handlers for special models. Our notebook serves an end-to-end template for them to publish the product.

The sagemaker-pytorch-inference-toolkit is cool but it also requires sellers to dig into additional information. Our notebook set-up requires least knowledge burden on sellers. Also we have a blog for the instruction as well.

Ok, we also have pre-built Deep Learning Containers and PyTorch 1.6 and above has built in Torchserve support. You'd be able to inject custom models and handlers with those. Something to look into at least?

[aaronmarkham]

I have some code in this notebook that sets up v2 and rolls it back at the end.
#1570

[aaronmarkham]

Checking this again... still have SageMaker version issues with an upgrade call in there that will break the customer's kernel for all other notebooks.

[aaronmarkham]

please remove this and replace with pinning to a specific version that works...