Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(ai.triton.server): Add Model Encryption support for Triton Server Service #3986

Merged
merged 91 commits into from
May 24, 2022
Merged

feat(ai.triton.server): Add Model Encryption support for Triton Server Service #3986

merged 91 commits into from
May 24, 2022

Conversation

mattdibi
Copy link
Contributor

Add Model Encryption support for Triton Server Service

Description of the solution adopted:

If a password was specified during configuration we assume all models in the path are encrypted with the same algorithm and can be decrypted with said password.

Therefore Kura, upon applying the TritonServerImpl configuration, will perform the following:

  • Create a new folder in /tmp with the required permission to avoid allowing access from users that have no root privileges or need that (i.e. Triton, Kura). For this example we'll call this folder /tmp/decypted_models

  • Run the Inference Server using this new folder as its Model Repository

tritonserver --model-repository=/tmp/decrypted-models

  • For each model specified in the "Model" property:

    • Will look for it in the "Model Repository Path" passed in the configuration

    • Decrypt it and store it in the /tmp/decrypted_models folder (for example /tmp/decrypted_models/autoencoder_fp32)

    • Tell Triton to load it from the /tmp/decrypted_models folder

    • Wait for Triton to signal correct loading of the decrypted model

    • Wipe the decrypted model folder /tmp/decrypted_models/autoencoder_fp32 from the filesystem as soon as Triton signals model load completion (see "Model Ready" and here)

Encryption procedure

Given a trained model inside the folder tf_autoencoder_fp32

tf_autoencoder_fp32
├── 1
│   └── model.savedmodel
│       ├── assets
│       ├── keras_metadata.pb
│       ├── saved_model.pb
│       └── variables
│           ├── variables.data-00000-of-00001
│           └── variables.index
└── config.pbtxt

4 directories, 5 files

We'll need to archive it with:

zip -vr tf_autoencoder_fp32.zip tf_autoencoder_fp32/

and encrypt the archive with:

gpg --armor --symmetric --cipher-algo AES256 tf_autoencoder_fp32.zip

The resulting archive tf_autoencoder_fp32.zip.asc can be decrypted by this method.

mattdibi added 30 commits May 10, 2022 11:04
@mattdibi
feat(EncryptionUtils): add class and methods skeleton
860ed55 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): add test skeleton
dce2ea5 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): add basic tests for createDecryptionFolder method
c2050b5 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): refactor to Gherkin style
d576c2c 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): removed comments, renamed method
507bfda 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): rename method
1e28418 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
feature(EncryptionUtils): add createDecryptionFolder method implement…
b63ca55 
…ation

Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): improve test with workdir setup/teardown
bab58b0 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): renamed method cleanup -> teardown
417a358 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(resources): add encrypted files for testing
9c4973e 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): add basic tests for decryptModel method
b1e5d85 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): simplify teardown method
fb68e3f
@mattdibi
test(EncryptionUtils): move test WORKDIR in java.io.tmpdir
1648678 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EcryptionUtils): add Gherkin style comments
6fecb20 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): add test for ASCII-armored file
eaa23c3 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): stricter checks on exception throw
89a13d7 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
docs: add copyright notice
aadcb6a 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test: fix expectation param order
d56d796 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
build: add org.bouncycastle.openpgp as local dependency
ea2aff3 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
feat(EncryptionUtils): add first implementation of decryptModel method
064aa01 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
feat(EncryptionUtils): fix BouncyCastle provider addition
baf6d06 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
feat(EncryptionUtils): add preconditions check
7c17f30 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
refactor: improve var names
55e91f4 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
refactor: remove unnecessary toString
3799576 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
feat(EncryptionUtils): move stream instantiation in decryptFile method
1685b8d 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
refactor: move stream instantiation close to use
c47e921 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
refactor: improve var names
0518c2a 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
feat(EncryptionUtils): rethrow PGPException on decryption failure
ffe0507 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(resources): rename resourse files
658723d 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi
test(EncryptionUtils): add zip file test
513c78f 
Signed-off-by: Mattia Dal Ben <[email protected]>
@mattdibi mattdibi changed the title feat(ai): Add Model Encryption support for Triton Server Service feat(ai.triton): Add Model Encryption support for Triton Server Service May 18, 2022
@mattdibi mattdibi marked this pull request as ready for review May 18, 2022 12:47
@mattdibi mattdibi changed the title feat(ai.triton): Add Model Encryption support for Triton Server Service feat(ai.triton.server): Add Model Encryption support for Triton Server Service May 18, 2022
@MMaiero MMaiero requested a review from nicolatimeus May 18, 2022 15:14
@mattdibi mattdibi mentioned this pull request May 20, 2022
Merged
nicolatimeus
nicolatimeus reviewed May 24, 2022
View reviewed changes
...iton.server/src/main/java/org/eclipse/kura/ai/triton/server/TritonServerEncryptionUtils.java Outdated
try {
FileUtils.cleanDirectory(new File(modelRootPath));
} catch (IOException e) {
logger.warn("Cannot clean directory at path {}", modelRootPath);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please log also exception stacktraces on failure.

nicolatimeus
nicolatimeus reviewed May 24, 2022
View reviewed changes
...iton.server/src/main/java/org/eclipse/kura/ai/triton/server/TritonServerEncryptionUtils.java Outdated
try {
FileUtils.cleanDirectory(new File(modelRootPath));
} catch (IOException e) {
logger.warn("Cannot clean directory at path {}", modelRootPath);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please log also exception stacktraces on failure.

@nicolatimeus nicolatimeus merged commit e0aa32f into eclipse-kura:develop May 24, 2022
@mattdibi mattdibi deleted the feature/model_encryption branch May 24, 2022 09:18
@nicolatimeus
Copy link
Contributor

nicolatimeus commented May 24, 2022 •
edited
Loading

This PR adds the org.bouncycastle/bcpg-jdk15on/1.68 dependency:
https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/52
MicrosoftTeams-image (5)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MMaiero MMaiero MMaiero left review comments

@nicolatimeus nicolatimeus nicolatimeus approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mattdibi @MMaiero @nicolatimeus