Jupyter Notebooks to illustrate the inference-contract family. #31

prakashngit · 2024-03-19T23:08:00Z

The Notebooks' structure, installation, execution patterns are same as those used by the Noteboks provided for the Exchange contract family. Please note that the inference guardian service must be started prior to executing the notebooks. Please see inference-contract/README.md for instructions.

Some of the additional features offered by this PR taking advantage of recent commits to the contracts repo:

It is possible to use the inference contract jupyter notebook via docker (using features from PR 23)
When using docker, the inference guardian will run in a separate container (using features from PR 27)
The notebooks are constructed using jupytext python modules (using features from PR 29)

g2flyer

After a few hurdles, i managed to build, run & execute all notebooks. A few comments though ..

g2flyer · 2024-03-20T23:51:46Z

inference-contract/test/script_test.sh

@@ -150,6 +150,8 @@ try ${PDO_HOME}/contracts/inference/scripts/ss_start.sh -c -o ${PDO_HOME}/logs -
    --config-dir ${PDO_HOME}/etc/contracts \
    --identity guardian_sservice

+sleep 3


i guess this sleep is to make sure that storage service is running before guardian is started? How robust is that 3 seconds and in case for whatever reason the sleep happened to be too short, how easy would it be to diagnose the issue? Couldn't we do some similar synchronization we do as when we wait for CCF or alike?

in ccf we wait for a file to be created. in pdo we curl a url found in the configuration file. i don't think either approach would work here. there is no guarantee that the db file (the obvious example) was removed between runs and the configuration file has several urls in it that make the grep approach "brittle" at best. maybe a better way for this service AND for the ones in pdo is to explicitly write a URL into a file (like what we do with the pid) and then use that to test for a service listening at the interface.

@g2flyer curious about your thoughts on hyperledger-labs/private-data-objects#476. This would be a clean way to address your concerns as well.

inference-contract/docs/notebooks/documents/overview.md

g2flyer · 2024-03-21T16:59:36Z

inference-contract/test/guardian_frontend.sh

+    --config-dir ${PDO_HOME}/etc/contracts \
+    --identity guardian_sservice
+
+sleep 3


Same comment as for sleep in script_test.sh

g2flyer · 2024-03-27T18:14:35Z

inference-contract/docs/notebooks/templates/token-issuer.py

+# token_metadata : additional information about the token
+# count : the number of tokens to mint for the asset
+
+# Note that the notebook assumes that there is a key file for the identity of the form: `${keys}/${identity}_private.pem`.


Maybe mention that User1 to User10 exist but more could be created via /documents/key_manager.ipynb notebook if

There is an improvement to this coming (see key-manager.py in #32 ). I don't think we should rely on the user keys... they should be removed from the PDO client install by default (fine for tests, but not for a real client).

inference-contract/docs/notebooks/templates/token-issuer.py

inference-contract/docs/notebooks/factories/token-issuer.py

g2flyer · 2024-03-27T21:23:42Z

inference-contract/docs/notebooks/templates/token-issuer.py

+# Create a token notebook for each of the minted tokens.
+
+# %%
+# %%skip True


I know these (undocumented!) skips are inherited but i still found them quite annoying ...

actually it is documented. exchange/pdo/contracts/jupyter.py (this is CODE so the question is whether code documents it). and also in docs/notebooks/README.md

g2flyer · 2024-03-27T21:27:19Z

inference-contract/docs/notebooks/templates/token.py

+# ```
+#
+
+# %% tags=["parameters"]


Same comment as with token-issuer, duplication of (slightly re-ordered) fields in two code blocks, one of them auto-generated is highly confusing. Seems to me relying just on the auto-generated one should be completely sufficient ...

can't do it. the parameters field is necessary for papermill to inject the parameters into the file.

I've verified that the parameters block must have a superset of the variables that are inherited.

inference-contract/docs/notebooks/templates/token.py

cmickeyb

Several things I would like to see addressed.

cmickeyb · 2024-03-29T22:41:22Z

inference-contract/docs/notebooks/factories/token-issuer.py

+# %% tags=["parameters"]
+identity = input('Identity of the token issuer: ')
+token_class = input('Name of the class of tokens to issue: ')
+service_host = input('Service host [localhost]: ')


suggest you pick up the actual defaults... service_host = input('...') or 'localhost'

cmickeyb · 2024-03-29T22:43:28Z

inference-contract/docs/notebooks/templates/token-issuer.py

+token_metadata = 'created by {}'.format(token_owner)
+count = 1
+service_host = 'localhost'
+instance_identifier = ''


there is no description for instance_identifyer; More interestingly, though, the "magic" second "parameter" code block (see below) actually redefines this variable anyway? So drop it here?

this is a property of the papermill templating modules. these are "placeholders".

cmickeyb · 2024-03-29T22:44:47Z

inference-contract/docs/notebooks/templates/token-issuer.py

+# service host by running the following:
+
+# %%
+# %%skip True


Why is skip default True here? Seems to contract text from above? In any case above we should say when to skip or not and that one correspondingly has to change the code block? Given that we have creation conditional on existence of toml file and so re-eval block is safe, why even bother with %%skip at alll?

this only need to be run once per "session". note it will be changing completely soon.

cmickeyb · 2024-03-29T22:46:21Z

inference-contract/docs/notebooks/templates/token-issuer.py

+
+context = pc_jupyter.ml_jupyter.initialize_token_context(state, bindings, context_file, token_path, **context_bindings)
+pc_jupyter.pbuilder.Context.SaveContextFile(state, context_file, prefix=token_path)
+print('context initialized')


this particular message is not an error and the context comes back successfully. that being said... we are trusting to exceptions (the python way) to be thrown when something break. more or less.

cmickeyb · 2024-03-29T22:48:16Z

inference-contract/docs/notebooks/templates/token.py

+# ### Initialize the PDO Environment
+#
+# Initialize the PDO environment. This assumes that a functional PDO configuration
+# is in place and that the PDO virtual environment has been activated. In particular,
+# ensure that the groups file and eservice database have been configured correctly.
+This can be done most easily by running the following in a shell:
+#
+# `$PDO_HOME/bin/pdo-create-service-groups.psh --service_host <service_host>`
+#


because you might have come through this factory first. and... #32 will change the way this is handled completely. suggest we not push for changes at this time. this code is one of the prime motivations for #32.

cmickeyb · 2024-03-29T22:52:20Z

inference-contract/docs/notebooks/templates/token.py

+    state, token_context, image=image_name, search_path=[path])
+
+# %% [markdown]
+# ### New User Performs Operations on the Asset


This is pretty much a TEST not part of the factory or template. This file is supposed to be a (primitive) interface to the token, not a test file.

cmickeyb · 2024-03-29T22:53:49Z

inference-contract/test/guardian_frontend.sh

+    --config-dir ${PDO_HOME}/etc/contracts \
+    --identity guardian_sservice
+
+sleep 3


what is the purpose of this file?

this file is necessary only if we do not use docker for the guardian (to be started before starting the jupyter server). Question is do we even need to support such a mode ? My suggestion is that we always expect the guardian to be deployed via the docker container. Please comment.

thanks, have removed this file completely. As part of documentation, have given commands using ss_start and gs_start as to how to deploy guardian directly, if needed for bare-metal.

…t family use-case. The Notebooks' structure, installation, execution patterns are same as those used by the Noteboks provided for the Exchange contract family. Please note that the inference guardian service must be started prior to executing the notebooks. Please see inference-contract/README.md for instructions. Some of the additional features offered by this PR taking advantage of recent commits to the contracts repo: 1. It is possible to use the inference contract jupyter notebook via docker (using features from PR 23) 2. When using docker, the inference guardian will run in a separate container (using features from PR 27) 3. The notebooks are constructed using jupytext python modules (using features from PR 29) Signed-off-by: Prakash Narayana Moorthy <[email protected]>

g2flyer

Changes seem to address the easy addressable concerns and others seem currently general limitations (and matter of taste, e.g., on the skip ;-)

Couldn't test it though as my ICX server has some boot problems (while trying to get DCAP working on it) and the other test machines i have cannot run the code: After a bit of annoying debugging, turns out that the tensorflow libraries we use do not work if you don't have AVX (as for these gemini lake) and the error message is not visible in docker lock. Two suggestions:

make before running anything involving tensorflow we might to do something like $(lscpu | grep -q avx) || die "AVX not avilable"
more generally for docker, might be useful to redirect error messages not only in a file but also to /dev/tty? That said, i found now also a found a reasonably convenient way to investigate failures after docker-compose stopped (you will though have to run docker-compose without --abort-on-container-exit as currently used in Makefile!): https://stackoverflow.com/questions/65953634/how-do-i-inspect-the-stopped-docker-container-files

cmickeyb

A couple cosmetic changes in the additional commits. I need to test this with the changes to the papermill parameters.

cmickeyb · 2024-04-15T20:43:34Z

inference-contract/docs/notebooks/templates/token-issuer.py

-service_host = 'localhost'
-instance_identifier = ''
+# rest of the parameters such as token_owner, token_class, service_host, and instance_identifier will get auto filled from factory via papermill
+# note that papermill places the parameters cell after first paramters cell in this notebook



have you verified that this works? papermill generally requires that the template contain complete list of parameters. it isn't pretty, but if you're setting through the templating mechanism it has to be there.

Verified that papermill generates error/warning messages for missing variables. Just add the parameters you expect with reasonable defaults. You aren't testing for missing variables anyway.

yes, tested, paperill does not need repeating all parameters. per papermill documentation, papermill populates the instance notebook with parameters after the first cell in the target notebook that has parameters. (that's why we currently see new parameters appearing after the default set). there is no need for the default set to have anything that papaermill will add. Yes, agreed, warnings will be generated while running the factory notebook.

it actually doesn't work. an error is generated. lets just add the variables that are necessary. to be clear... you can think about the papermill parameters as being the parameters specified to a script. its the "argparse" for notebooks. while @g2flyer doesn't like the duplication, it is a standard way of extending notebooks. your parameter block in the template should contain EVERY variable that you currently (or may reasonably in the future) specify when creating the contracts. for example, you don't currently have the ability to specify the metadata. in the future, that should absolutely be a parameter that could be specified.

thanks, added back all parameters that are being parametrized by papermill.

cmickeyb

See message about missing papermill parameters.

marcelamelara

I mostly have nitpicky formatting comments, though there are a couple of places that adding links/references to other documentation would be really helpful.

marcelamelara · 2024-04-15T23:25:09Z

docs/notebooks/index.md

@@ -51,9 +51,10 @@ The Digital Assets contract family implements a basic digital asset for bitmap i
 ### Inference


-The Inference contract family provides contracts for creating a confidentiality preserving policy-wrapper around the usage of a machine learning (ML) model. At its core, the implementation uses a token contract to  specify and enforce policies to be followed while using the ML model for inferecing operations. 
+The Inference contract family provides contracts for creating a confidentiality
+preserving policy-wrapper around the usage of a machine learning (ML) model.


What is a policy-wrapper?

Thank you, have clarified about both policy-wrapper and token object : "The policy-wrapper specifies and checks the
policies to be followed while using the ML model for inferencing
operations, and is implemented as a PDO contract that we refer to as the Token Object."

marcelamelara · 2024-04-15T23:29:59Z

docs/notebooks/index.md

-The Inference contract family provides contracts for creating a confidentiality preserving policy-wrapper around the usage of a machine learning (ML) model. At its core, the implementation uses a token contract to  specify and enforce policies to be followed while using the ML model for inferecing operations. 
+The Inference contract family provides contracts for creating a confidentiality
+preserving policy-wrapper around the usage of a machine learning (ML) model.
+At its core, the implementation uses a token contract to  specify and enforce


We shouldn't assume the reader knows what a "token contract" is. It would be helpful to add a link to "token contract" to documentation where the reader can find more information on the underlying implementation.

thanks, please see comment above for fix.

marcelamelara · 2024-04-15T23:30:31Z

docs/notebooks/index.md

+The Inference contract family provides contracts for creating a confidentiality
+preserving policy-wrapper around the usage of a machine learning (ML) model.
+At its core, the implementation uses a token contract to  specify and enforce
+policies to be followed while using the ML model for inferecing operations.


Suggested change

policies to be followed while using the ML model for inferecing operations.

policies to be followed while using the ML model for inferencing operations.

thanks, done.

marcelamelara · 2024-04-15T23:33:17Z

inference-contract/README.md

+even though the Jupyter server is deployed on bare-metal. Please consult the
+`https://github.com/hyperledger-labs/private-data-objects/tree/main` repo for deploypment
+options for pdo ledger and services. Regarding deployment of the model guardian,


This is a bit cleaner.

Suggested change

even though the Jupyter server is deployed on bare-metal. Please consult the

`https://github.com/hyperledger-labs/private-data-objects/tree/main` repo for deploypment

options for pdo ledger and services. Regarding deployment of the model guardian,

even though the Jupyter server is deployed on bare-metal. Please consult the [PDO repo](https://github.com/hyperledger-labs/private-data-objects/tree/main) for deployment

options for pdo ledger and services. Regarding deployment of the model guardian,

thanks, done

marcelamelara · 2024-04-15T23:33:42Z

inference-contract/README.md

+We provide [Jupyter Notebooks](./docs/notebooks/README.md) that can be executed
+in an interactive manner to illustrate the functionality of the inference
+contract family. We support Jupyter notebook deployment via both docker containers
+as well as bare-metal. For docker, please invoke the `make test_jupyter` command


Para break.

Suggested change

as well as bare-metal. For docker, please invoke the `make test_jupyter` command

as well as bare-metal.

For docker, please invoke the `make test_jupyter` command

thanks, done

marcelamelara · 2024-04-15T23:39:49Z

inference-contract/docs/notebooks/documents/overview.md

+Object and the Asset Guardian Service.
+
+The asset guardian service here is implemented as a web service
+containing a frontend and backend portions.The backend of the guardian


Suggested change

containing a frontend and backend portions.The backend of the guardian

containing a frontend and backend portions. The backend of the guardian

thanks, done

marcelamelara · 2024-04-15T23:40:24Z

inference-contract/README.md

+contract family. We support Jupyter notebook deployment via both docker containers
+as well as bare-metal. For docker, please invoke the `make test_jupyter` command
+from within the docker folder contained within this repo. This command deploys the
+pdo ledger, pdo services, guardian frontend as well as the openvino containers in


Please ensure consistent capitalization here an in other docs.

Suggested change

pdo ledger, pdo services, guardian frontend as well as the openvino containers in

pdo ledger, pdo services, guardian frontend as well as the openVINO containers in

thanks, done

marcelamelara · 2024-04-15T23:42:14Z

inference-contract/docs/notebooks/documents/overview.md

+functions: 1. Processes the capability package generated by the token
+object and generates/invokes the API for the backend. The capability
+package may be thought of the asset-usage approval generated by the
+token object smart contract. 2. Implements any input preprocessing and
+output postprocessing (collectively referred to as model scoring) as
+determined necessary by the original asset owner.


This will ensure these are rendered as numbered bullets.

Suggested change

functions: 1. Processes the capability package generated by the token

object and generates/invokes the API for the backend. The capability

package may be thought of the asset-usage approval generated by the

token object smart contract. 2. Implements any input preprocessing and

output postprocessing (collectively referred to as model scoring) as

determined necessary by the original asset owner.

functions:

1. Processes the capability package generated by the token

object and generates/invokes the API for the backend. The capability

package may be thought of the asset-usage approval generated by the

token object smart contract.

2. Implements any input preprocessing and

output postprocessing (collectively referred to as model scoring) as

determined necessary by the original asset owner.

thanks, done

marcelamelara · 2024-04-15T23:43:47Z

inference-contract/docs/notebooks/documents/overview.md

+script, and thus ensuring tight binding between the token object
+contract and the model usage.
+
+We note that while the Token Object smart contract is executed within


As above, it may be good to add a link to "token object smart contract" here for reference.

thanks, hope the clarification of token object addresses this.;

marcelamelara · 2024-04-15T23:44:06Z

inference-contract/docs/notebooks/documents/overview.md

+
+We note that while the Token Object smart contract is executed within
+Intel SGX enclaves (via PDO), the asset guardian service in this
+impelementation is not protected by TEEs. A prospective asset-user


Suggested change

impelementation is not protected by TEEs. A prospective asset-user

implementation is not protected by TEEs. A prospective asset-user

thanks, done.

Signed-off-by: Prakash Narayana Moorthy <[email protected]>

prakashngit requested review from cmickeyb, g2flyer, marcelamelara and bvavala March 19, 2024 23:08

prakashngit assigned cmickeyb Mar 19, 2024

g2flyer requested changes Mar 27, 2024

View reviewed changes

cmickeyb reviewed Mar 29, 2024

View reviewed changes

prakashngit force-pushed the Prakash.Jupyter_Notebooks_inference_contract_family branch from 3e1fcc3 to 676abb5 Compare April 12, 2024 20:09

g2flyer self-requested a review April 13, 2024 03:46

g2flyer approved these changes Apr 13, 2024

View reviewed changes

cmickeyb reviewed Apr 15, 2024

View reviewed changes

cmickeyb requested changes Apr 15, 2024

View reviewed changes

marcelamelara reviewed Apr 15, 2024

View reviewed changes

address review comments on PR

5e367ba

Signed-off-by: Prakash Narayana Moorthy <[email protected]>

prakashngit force-pushed the Prakash.Jupyter_Notebooks_inference_contract_family branch from d72723f to 5e367ba Compare April 16, 2024 22:24

cmickeyb approved these changes Apr 16, 2024

View reviewed changes

cmickeyb merged commit a27b504 into hyperledger-labs:main Apr 17, 2024
2 checks passed

	policies to be followed while using the ML model for inferecing operations.
	policies to be followed while using the ML model for inferencing operations.

	containing a frontend and backend portions.The backend of the guardian
	containing a frontend and backend portions. The backend of the guardian

	pdo ledger, pdo services, guardian frontend as well as the openvino containers in
	pdo ledger, pdo services, guardian frontend as well as the openVINO containers in

	impelementation is not protected by TEEs. A prospective asset-user
	implementation is not protected by TEEs. A prospective asset-user

Jupyter Notebooks to illustrate the inference-contract family. #31

Jupyter Notebooks to illustrate the inference-contract family. #31

Conversation

prakashngit commented Mar 19, 2024

g2flyer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmickeyb Mar 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmickeyb Mar 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmickeyb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

g2flyer left a comment • edited Loading

Choose a reason for hiding this comment

cmickeyb left a comment

Choose a reason for hiding this comment

cmickeyb Apr 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmickeyb Apr 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmickeyb left a comment

Choose a reason for hiding this comment

marcelamelara left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmickeyb Mar 29, 2024 •

edited

Loading

cmickeyb Mar 29, 2024 •

edited

Loading

g2flyer left a comment •

edited

Loading

cmickeyb Apr 15, 2024 •

edited

Loading

cmickeyb Apr 16, 2024 •

edited

Loading