Adds Huggingface GenAI container build specs into pytorch/Dockerfile and pytorch/docker-compose.yaml

[HarshaRamayanam]

Description

This PR adds a new entry into pytorch/Dockerfile for building a Huggingface GenAI container that can run a typical Generative AI model from Hugging Face. (like run_clm.py script from transformers)

It also adds a new service entry in pytorch/docker-compose.yaml file named hf-genai (can be renamed in future) to build the container

Additional files added are

• pytorch/generate_ssh_keys.sh
• pytorch/hf-genai-requirements.txt

Related Issue

None

Changes Made

• pytorch/Dockerfile (Modified)

• pytorch/docker-compose.yaml (Modified)

• pytorch/generate_ssh_keys.sh (New file)

• pytorch/hf-genai-requirements.txt (New file)

• The code follows the project's coding standards.

• No Intel Internal IP is present within the changes.

• The documentation has been updated to reflect any changes in functionality.

Validation

• I have tested any changes in container groups locally with test_runner.py with all existing tests passing, and I have added new tests where applicable.

[github-actions]

Dependency Review

The following issues were found:

• ✅ 0 vulnerable package(s)
• ✅ 0 package(s) with incompatible licenses
• ✅ 0 package(s) with invalid SPDX license definitions
• ⚠️ 1 package(s) with unknown licenses.

See the Details below.

License Issues

pytorch/hf-genai-requirements.txt

Package	Version	License	Issue Type
rouge_score	0.1.2	Null	Unknown License

OpenSSF Scorecard

Scorecard details

Package	Version	Score	Details
pip/SentencePiece	0.2.0	🟢 7.5	Details Check Score Reason Code-Review 🟢 5 Found 6/11 approved changesets -- score normalized to 5 Maintained 🟢 10 22 commit(s) and 16 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Signed-Releases 🟢 4 2 out of the last 5 releases have a total of 2 signed artifacts. Packaging ⚠️ -1 packaging workflow not detected Branch-Protection ⚠️ 0 branch protection not enabled on development/release branches Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Token-Permissions 🟢 10 GitHub workflow tokens follow principle of least privilege Binary-Artifacts 🟢 10 no binaries found in the repo SAST 🟢 5 SAST tool is not run on all commits -- score normalized to 5 Fuzzing 🟢 10 project is fuzzed Security-Policy 🟢 10 security policy file detected Pinned-Dependencies 🟢 8 dependency not pinned by hash detected -- score normalized to 8 Vulnerabilities 🟢 10 0 existing vulnerabilities detected
pip/accelerate	0.28.0	🟢 6.2	Details Check Score Reason Code-Review 🟢 9 Found 28/30 approved changesets -- score normalized to 9 Maintained 🟢 10 30 commit(s) and 16 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Signed-Releases ⚠️ -1 no releases found Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Binary-Artifacts 🟢 10 no binaries found in the repo Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Fuzzing ⚠️ 0 project is not fuzzed Security-Policy ⚠️ 0 security policy file not detected Vulnerabilities 🟢 10 0 existing vulnerabilities detected Packaging 🟢 10 packaging workflow detected SAST 🟢 3 SAST tool is not run on all commits -- score normalized to 3
pip/datasets	2.19.0	🟢 6	Details Check Score Reason Code-Review 🟢 5 Found 16/30 approved changesets -- score normalized to 5 Maintained 🟢 10 30 commit(s) and 9 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Security-Policy 🟢 10 security policy file detected Packaging ⚠️ -1 packaging workflow not detected Binary-Artifacts 🟢 10 no binaries found in the repo Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Vulnerabilities 🟢 10 0 existing vulnerabilities detected Fuzzing ⚠️ 0 project is not fuzzed Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0
pip/einops	0.7.0	🟢 5	Details Check Score Reason Code-Review ⚠️ 2 Found 4/20 approved changesets -- score normalized to 2 Maintained 🟢 10 8 commit(s) and 8 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Packaging ⚠️ -1 packaging workflow not detected Binary-Artifacts 🟢 10 no binaries found in the repo Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Signed-Releases ⚠️ -1 no releases found Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Vulnerabilities 🟢 10 0 existing vulnerabilities detected Fuzzing ⚠️ 0 project is not fuzzed Security-Policy ⚠️ 0 security policy file not detected SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0
pip/evaluate	0.4.1	🟢 5.4	Details Check Score Reason Code-Review 🟢 9 Found 29/30 approved changesets -- score normalized to 9 Maintained 🟢 5 6 commit(s) and 1 issue activity found in the last 90 days -- score normalized to 5 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 packaging workflow not detected Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Binary-Artifacts 🟢 10 no binaries found in the repo Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Fuzzing ⚠️ 0 project is not fuzzed Security-Policy ⚠️ 0 security policy file not detected Vulnerabilities 🟢 10 0 existing vulnerabilities detected SAST 🟢 3 SAST tool is not run on all commits -- score normalized to 3
pip/nltk	3.8.1	🟢 4.9	Details Check Score Reason Code-Review 🟢 10 all changesets reviewed Maintained ⚠️ 0 1 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Security-Policy 🟢 9 security policy file detected Branch-Protection ⚠️ 0 branch protection not enabled on development/release branches Packaging ⚠️ -1 packaging workflow not detected Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Binary-Artifacts 🟢 10 no binaries found in the repo Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Vulnerabilities 🟢 10 0 existing vulnerabilities detected Fuzzing ⚠️ 0 project is not fuzzed SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0
pip/onnxruntime	1.17.3	🟢 6.8	Details Check Score Reason Code-Review 🟢 10 all last 30 commits are reviewed through GitHub Maintained 🟢 10 30 commit(s) out of 30 and 8 issue activity out of 30 found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no badge detected Vulnerabilities 🟢 10 no vulnerabilities detected Signed-Releases ⚠️ 0 0 out of 5 artifacts are signed or have provenance Branch-Protection 🟢 8 branch protection is not maximal on development and all release branches Security-Policy 🟢 10 security policy file detected Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 no published package detected License 🟢 10 license file detected Token-Permissions ⚠️ 0 non read-only tokens detected in GitHub workflows Dependency-Update-Tool 🟢 10 update tool detected Binary-Artifacts 🟢 10 no binaries found in the repo Fuzzing ⚠️ 0 project is not fuzzed Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0
pip/onnxruntime-extensions	0.10.1	🟢 6.1	Details Check Score Reason Code-Review 🟢 9 Found 29/30 approved changesets -- score normalized to 9 Maintained 🟢 10 30 commit(s) and 11 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 packaging workflow not detected Security-Policy 🟢 10 security policy file detected Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0 Fuzzing ⚠️ 0 project is not fuzzed Vulnerabilities 🟢 10 0 existing vulnerabilities detected Binary-Artifacts 🟢 7 binaries present in source code Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0
pip/peft	0.10.0	Unknown	Unknown
pip/protobuf	4.24.4	🟢 7.1	Details Check Score Reason Binary-Artifacts 🟢 10 no binaries found in the repo Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration CI-Tests 🟢 10 23 out of 23 merged PRs checked by a CI test -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected Code-Review ⚠️ 2 found 24 unreviewed changesets out of 30 -- score normalized to 2 Contributors 🟢 10 13 different organizations found -- score normalized to 10 Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Dependency-Update-Tool 🟢 10 update tool detected Fuzzing 🟢 10 project is fuzzed License 🟢 9 license file detected Maintained 🟢 10 30 commit(s) out of 30 and 2 issue activity out of 30 found in the last 90 days -- score normalized to 10 Packaging ⚠️ -1 no published package detected Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 SAST 🟢 3 SAST tool is not run on all commits -- score normalized to 3 Security-Policy 🟢 10 security policy file detected Signed-Releases ⚠️ 0 0 out of 5 artifacts are signed or have provenance Token-Permissions 🟢 10 GitHub workflow tokens follow principle of least privilege Vulnerabilities 🟢 7 3 existing vulnerabilities detected
pip/py-cpuinfo	9.0.0	🟢 3.8	Details Check Score Reason Code-Review 🟢 4 Found 7/17 approved changesets -- score normalized to 4 Maintained ⚠️ 0 0 commit(s) and 1 issue activity found in the last 90 days -- score normalized to 0 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Signed-Releases ⚠️ -1 no releases found Branch-Protection ⚠️ 0 branch protection not enabled on development/release branches Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Packaging ⚠️ -1 packaging workflow not detected Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Binary-Artifacts 🟢 10 no binaries found in the repo Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Vulnerabilities 🟢 10 0 existing vulnerabilities detected Fuzzing ⚠️ 0 project is not fuzzed Security-Policy ⚠️ 0 security policy file not detected SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0
pip/rouge_score	0.1.2	Unknown	Unknown
pip/scikit-learn	1.5.0	🟢 9.5	Details Check Score Reason Code-Review 🟢 10 all changesets reviewed Maintained 🟢 10 30 commit(s) and 23 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Signed-Releases ⚠️ -1 no releases found Dangerous-Workflow ⚠️ -1 no workflows found Security-Policy 🟢 10 security policy file detected Packaging ⚠️ -1 packaging workflow not detected Token-Permissions ⚠️ -1 No tokens found Vulnerabilities 🟢 10 0 existing vulnerabilities detected Binary-Artifacts 🟢 10 no binaries found in the repo Pinned-Dependencies ⚠️ -1 no dependencies found Fuzzing 🟢 10 project is fuzzed SAST 🟢 10 SAST tool is run on all commits
pip/tokenizers	0.19.1	🟢 5.5	Details Check Score Reason Code-Review 🟢 7 Found 20/28 approved changesets -- score normalized to 7 Maintained 🟢 10 18 commit(s) and 24 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Signed-Releases ⚠️ -1 no releases found Dangerous-Workflow 🟢 10 no dangerous workflow patterns detected Binary-Artifacts 🟢 10 no binaries found in the repo Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Fuzzing ⚠️ 0 project is not fuzzed Security-Policy ⚠️ 0 security policy file not detected Packaging 🟢 10 packaging workflow detected SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0 Vulnerabilities 🟢 6 4 existing vulnerabilities detected
pip/transformers	4.41.2	🟢 4.5	Details Check Score Reason Code-Review 🟢 10 all changesets reviewed Maintained 🟢 10 30 commit(s) and 17 issue activity found in the last 90 days -- score normalized to 10 CII-Best-Practices ⚠️ 0 no effort to earn an OpenSSF best practices badge detected License 🟢 10 license file detected Branch-Protection ⚠️ -1 internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration Signed-Releases ⚠️ -1 no releases found Security-Policy 🟢 10 security policy file detected Dangerous-Workflow ⚠️ 0 dangerous workflow patterns detected Token-Permissions ⚠️ 0 detected GitHub workflow tokens with excessive permissions Binary-Artifacts 🟢 10 no binaries found in the repo Fuzzing ⚠️ 0 project is not fuzzed Packaging 🟢 10 packaging workflow detected SAST ⚠️ 0 SAST tool is not run on all commits -- score normalized to 0 Pinned-Dependencies ⚠️ 0 dependency not pinned by hash detected -- score normalized to 0 Vulnerabilities ⚠️ 0 465 existing vulnerabilities detected

Scanned Manifest Files

pytorch/hf-genai-requirements.txt

• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]
• [email protected]

[tylertitsworth]

• What is the public name of this container going to be? intel/intel-optimized-pytorch:2.3.0-pip-hf-genai?
• What happens to workflows/charts/huggingface-llm? Does that container go away in favor of this one or are they different?

Please update the README.md file appropriately, this will go to Dockerhub.

[dmsuehir]

• What happens to workflows/charts/huggingface-llm? Does that container go away in favor of this one or are they different?

@tylertitsworth The huggingface-llm container will switch to use this Gen AI container as it's base and I think the only thing it'll need is a copy of the fine tuning python script

[tylertitsworth]

@tylertitsworth The huggingface-llm container will switch to use this Gen AI container as it's base and I think the only thing it'll need is a copy of the fine tuning python script

@dmsuehir Good to know, then the only potential conflict with this PR is #124.

[HarshaRamayanam]

• intel/intel-optimized-pytorch:2.3.0-pip-hf-genai

@ashahba @dmsuehir What do you think about public name Tyler mentioned? I'm okay with it.

[dmsuehir]

• intel/intel-optimized-pytorch:2.3.0-pip-hf-genai

@ashahba @dmsuehir What do you think about public name Tyler mentioned? I'm okay with it.

I'm ok with this, or maybe also including the HF version. Either way

[tylertitsworth]

Recipe looks good, there are a few kinks to work out that have been helpfully suggested by your coworkers. Please write some tests in pytorch/tests/tests.yaml that validate this solution. Make sure no test runs for more than 5-10 minutes.

Next part is the docs, which you can choose to split into another PR if you want. Let me know what you decide.

Most important thing is that all of the CI is green.

[HarshaRamayanam]

Recipe looks good, there are a few kinks to work out that have been helpfully suggested by your coworkers. Please write some tests in pytorch/tests/tests.yaml that validate this solution. Make sure no test runs for more than 5-10 minutes.

Next part is the docs, which you can choose to split into another PR if you want. Let me know what you decide.

Most important thing is that all of the CI is green.

I'll do the docs too in this PR. I'll update you

[tylertitsworth]

All of your changes look fine, I have a couple comments, but since this is dependent on #124 we need to wait on merging this.

[tylertitsworth]

@HarshaRamayanam I had to squash your commits because the history was getting complex, feel free to re-contribute those commits however you want.

[tylertitsworth]

A few minor things, I have locally validated your image and if you're happy with it we can merge today.