Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stepfunctions-tasks): FastFile mode for SageMaker Training Job #26675

Merged
merged 12 commits into from
Aug 23, 2023

Conversation

tmyoda
Copy link
Contributor

@tmyoda tmyoda commented Aug 9, 2023

According to the AWS documentation, the TrainingInputMode for a SageMaker Training Job can be one of the following: Pipe | File | FastFile

https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_Channel.html#sagemaker-Type-Channel-InputMode

https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_AlgorithmSpecification.html#API_AlgorithmSpecification_Contents

I have just added FastFile below to align with the official documentation.
https://github.com/aws/aws-cdk/blob/v2.90.0/packages/aws-cdk-lib/aws-stepfunctions-tasks/lib/sagemaker/base-types.ts#L458

Closes #26653.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team August 9, 2023 00:11
@github-actions github-actions bot added beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2 labels Aug 9, 2023
@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 9, 2023
Copy link
Contributor

@lpizzinidev lpizzinidev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall.

I think that since we are here we should add validation for the algorithmName parameter in the SageMakerCreateTrainingJob constructor following this spec, and the related unit tests.

@tmyoda
Copy link
Contributor Author

tmyoda commented Aug 9, 2023

Thank you for the review.
I will add validation for algorithmName in this PR.

@tmyoda tmyoda requested a review from lpizzinidev August 12, 2023 02:19
@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 12, 2023
Copy link
Contributor

@lpizzinidev lpizzinidev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up work 💪
I left you a couple of comments on things that I think should be fixed

@tmyoda tmyoda requested a review from lpizzinidev August 12, 2023 22:38
Copy link
Contributor

@lpizzinidev lpizzinidev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you 👍
I left you a comment on the enum value name (sorry, I missed that in the previous review).
If you could fix that then this will be good to go for me.

@tmyoda tmyoda requested a review from lpizzinidev August 14, 2023 18:28
@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Aug 15, 2023
@kaizencc kaizencc changed the title fix(stepfunctions-tasks): FastFile mode for SageMaker Training Job feat(stepfunctions-tasks): FastFile mode for SageMaker Training Job Aug 16, 2023
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Aug 16, 2023
Copy link
Contributor

@kaizencc kaizencc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @tmyoda! Thanks for this PR. Here is my additional feedback:

  • there looks to be two things here, fastFile mode and validation for algorithm names. they belong in separate PRs
  • fastFile mode is a feature, not a fix, which also means that I would like to see an update to the README section detailing how to use fast file (and what it is haha).

@@ -324,6 +332,21 @@ export class SageMakerCreateTrainingJob extends sfn.TaskStateBase implements iam
: {};
}

private validateAlgorithmName(algorithmName?: string): void {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't have anything to do with FastFile mode, right? Can it be included in a separate PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, will create another PR for the algorithm name validation

@aws-cdk-automation aws-cdk-automation dismissed their stale review August 19, 2023 19:02

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@tmyoda tmyoda requested a review from kaizencc August 19, 2023 19:35
@tmyoda
Copy link
Contributor Author

tmyoda commented Aug 19, 2023

I plan to create another PR for algorithm name validation once it's merged

@mergify
Copy link
Contributor

mergify bot commented Aug 23, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: baf02cb
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented Aug 23, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 5fef403 into aws:main Aug 23, 2023
mergify bot pushed a commit that referenced this pull request Aug 29, 2023
…reateTrainingJob` (#26877)

Referencing PR #26675, I have added validation for the `algorithmName` parameter in `SageMakerCreateTrainingJob`.
However, it was suggested that changes for validation should be separated.  So, I have created this PR.

Docs for `algorithmName`:
https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_AlgorithmSpecification.html#API_AlgorithmSpecification_Contents


Exemption Request:
This change does not alter the behavior.
I believe the unit test `create-training-job.test.ts` that I have added is sufficient to test this change.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(stepfunctions-tasks): InputMode for SageMaker does not support FastFile
4 participants