change: add EI support for TFS framework #682

chuyang-deng · 2019-03-05T22:36:20Z

Issue #, if available:

Description of changes:
Update Python SDK to reflect changes in TFS container. This change should not be merged before TFS EI images get distributed.

aws/sagemaker-tensorflow-serving-container#10
aws/sagemaker-tensorflow-serving-container#12

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

I have read the CONTRIBUTING doc
I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have updated the changelog with a description of my changes (if appropriate)
I have updated any necessary documentation (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

jesterhazy · 2019-03-05T22:46:55Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

ChoiByungWook

Please update the changelog.

Update the description in your PR conversation and also reference the PR notebook example that depends on this change.

ChoiByungWook · 2019-03-06T21:58:48Z

src/sagemaker/fw_utils.py

@@ -106,7 +106,7 @@ def _accelerator_type_valid_for_framework(framework, accelerator_type=None, opti

    if framework not in VALID_EIA_FRAMEWORKS:
        raise ValueError('{} is not supported with Amazon Elastic Inference. Currently only '
-                         'Python-based TensorFlow and MXNet are supported.'.format(framework))
+                         'TensorFlow Serving, TensorFlow and MXNet are supported.'.format(framework))


I would still keep the Python-based TensorFlow wording there.

ChoiByungWook · 2019-03-06T21:59:23Z

src/sagemaker/tensorflow/deploying_python.rst

@@ -25,7 +25,7 @@ like this:

 The code block above deploys a SageMaker Endpoint with one instance of the type 'ml.c4.xlarge'.

-Python-based TensorFlow serving on SageMaker has support for `Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html>`_, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to attach an Elastic Inference accelerator to your endpoint provide the accelerator type to ``accelerator_type`` to your ``deploy`` call.
+TensorFlow serving on SageMaker has support for `Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html>`_, which allows for inference acceleration to a hosted endpoint for a fraction of the cost of using a full GPU instance. In order to attach an Elastic Inference accelerator to your endpoint provide the accelerator type to ``accelerator_type`` to your ``deploy`` call.


Since these documents are specific to the Python-based container, maybe we should keep the wording there?

ChoiByungWook · 2019-03-06T22:00:01Z

tests/integ/test_tfs.py

+                      framework_version=tf_full_version,
+                      sagemaker_session=sagemaker_session,
+                      # passing an existing image for now because ei image is not in prod account yet
+                      image='562200002688.dkr.ecr.us-west-2.amazonaws.com/sagemaker-tensorflow-serving-eia:{}-cpu'


Before this PR gets merged, this will need to change.

I've updated PR description with this information.

jesterhazy · 2019-03-06T23:59:47Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-07T00:07:18Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-14T09:15:51Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-14T09:18:24Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-15T19:02:44Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-15T20:23:46Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-15T20:42:30Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-15T20:48:34Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-15T20:52:38Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-16T03:29:27Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-16T19:27:26Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

ChoiByungWook

coolio!

ChoiByungWook · 2019-03-18T21:38:10Z

tests/integ/test_tfs.py

@@ -55,6 +70,16 @@ def test_predict(tfs_predictor, instance_type):  # pylint: disable=W0613
    assert expected_result == result


+@pytest.mark.skipif(tests.integ.test_region() not in tests.integ.EI_SUPPORTED_REGIONS,
+                    reason='EI is not supported in region {}'.format(tests.integ.test_region()))
+def test_predict_with_accelerator(tfs_predictor_with_accelerator):


this should be apart of the canary.

ChoiByungWook · 2019-03-18T21:39:59Z

src/sagemaker/tensorflow/deploying_tensorflow_serving.rst

@@ -66,6 +64,17 @@ If you already have existing model artifacts in S3, you can skip training and de

  predictor = model.deploy(initial_instance_count=1, instance_type='ml.c5.xlarge')

+You can also deploy the model with Amazon Elastic Inference, which allows you to attach low-cost GPU accelerators to your Amazon EC2 and SageMaker instances.


We should be consistent with where the EI code block is introduced in the documentation.

Either the current docs or this should change: https://github.com/aws/sagemaker-python-sdk/blob/aa7335c6556d122173030e5cf67b46d1b4b46b3c/src/sagemaker/tensorflow/deploying_python.rst#deploying-from-an-estimator

jesterhazy · 2019-03-18T23:13:09Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-19T19:48:32Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

laurenyu · 2019-03-20T17:07:16Z

now that #705 has been merged, don't forget to change the PR title to include the appropriate prefix

ChoiByungWook

Please update the PR title, so that it can work with the new build-system.

jesterhazy · 2019-03-26T00:53:36Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-26T18:40:48Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-26T19:29:20Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-26T21:28:56Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jesterhazy · 2019-03-26T23:03:01Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

chuyang-deng requested a review from ChoiByungWook March 5, 2019 22:36

ChoiByungWook suggested changes Mar 6, 2019

View reviewed changes

chuyang-deng mentioned this pull request Mar 6, 2019

Update TFS EI example. aws/amazon-sagemaker-examples#660

Merged

ChoiByungWook reviewed Mar 18, 2019

View reviewed changes

Chuyang Deng and others added 4 commits March 25, 2019 16:35

Add EI support for TFS.

4ed3998

Add integ test to canary, update tfs readme.

945e4c5

Resolve conflicts.

9f93237

Remove outdated CHANGELOG item.

aee6e3b

ChoiByungWook self-requested a review March 26, 2019 00:11

ChoiByungWook approved these changes Mar 26, 2019

View reviewed changes

chuyang-deng changed the title ~~Add EI support for TFS framework.~~ change: add EI support for TFS framework Mar 26, 2019

Merge branch 'master' into ei_support

511f6aa

Merge branch 'master' into ei_support

38e0135

chuyang-deng merged commit 475e051 into aws:master Mar 26, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

change: add EI support for TFS framework #682

change: add EI support for TFS framework #682

chuyang-deng commented Mar 5, 2019 •

edited

Loading

jesterhazy commented Mar 5, 2019

ChoiByungWook left a comment •

edited

Loading

ChoiByungWook Mar 6, 2019

ChoiByungWook Mar 6, 2019

ChoiByungWook Mar 6, 2019

chuyang-deng Mar 6, 2019

jesterhazy commented Mar 6, 2019

jesterhazy commented Mar 7, 2019

jesterhazy commented Mar 14, 2019

jesterhazy commented Mar 14, 2019

jesterhazy commented Mar 15, 2019

jesterhazy commented Mar 15, 2019

jesterhazy commented Mar 15, 2019

jesterhazy commented Mar 15, 2019

jesterhazy commented Mar 15, 2019

jesterhazy commented Mar 16, 2019

jesterhazy commented Mar 16, 2019

ChoiByungWook left a comment

ChoiByungWook Mar 18, 2019

ChoiByungWook Mar 18, 2019

jesterhazy commented Mar 18, 2019

jesterhazy commented Mar 19, 2019

laurenyu commented Mar 20, 2019

ChoiByungWook left a comment

jesterhazy commented Mar 26, 2019

jesterhazy commented Mar 26, 2019

jesterhazy commented Mar 26, 2019

jesterhazy commented Mar 26, 2019

jesterhazy commented Mar 26, 2019

		@@ -66,6 +64,17 @@ If you already have existing model artifacts in S3, you can skip training and de

		predictor = model.deploy(initial_instance_count=1, instance_type='ml.c5.xlarge')

		You can also deploy the model with Amazon Elastic Inference, which allows you to attach low-cost GPU accelerators to your Amazon EC2 and SageMaker instances.

change: add EI support for TFS framework #682

change: add EI support for TFS framework #682

Conversation

chuyang-deng commented Mar 5, 2019 • edited Loading

Merge Checklist

jesterhazy commented Mar 5, 2019

AWS CodeBuild CI Report

ChoiByungWook left a comment • edited Loading

Choose a reason for hiding this comment

ChoiByungWook Mar 6, 2019

Choose a reason for hiding this comment

ChoiByungWook Mar 6, 2019

Choose a reason for hiding this comment

ChoiByungWook Mar 6, 2019

Choose a reason for hiding this comment

chuyang-deng Mar 6, 2019

Choose a reason for hiding this comment

jesterhazy commented Mar 6, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 7, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 14, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 14, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 15, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 15, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 15, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 15, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 15, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 16, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 16, 2019

AWS CodeBuild CI Report

ChoiByungWook left a comment

Choose a reason for hiding this comment

ChoiByungWook Mar 18, 2019

Choose a reason for hiding this comment

ChoiByungWook Mar 18, 2019

Choose a reason for hiding this comment

jesterhazy commented Mar 18, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 19, 2019

AWS CodeBuild CI Report

laurenyu commented Mar 20, 2019

ChoiByungWook left a comment

Choose a reason for hiding this comment

jesterhazy commented Mar 26, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 26, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 26, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 26, 2019

AWS CodeBuild CI Report

jesterhazy commented Mar 26, 2019

AWS CodeBuild CI Report

chuyang-deng commented Mar 5, 2019 •

edited

Loading

ChoiByungWook left a comment •

edited

Loading