-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixes #35: Adding support for loading configs from stdin #36
Conversation
Testing locally, I no longer get a file extension error trying to pass stdin to This works as expected w/ the smallfile-example.yaml file included in the container. $ podman run --rm smallfile-test -f smallfile-example.yaml But I get a typing error traceback with the stdin aproach. $ cat smallfile-example.yaml | podman run --rm smallfile-test -f - --debug
Invalid input encountered while executing step 'workload' from file '-':
Validation failed: Must be a dict, got NoneType
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/arcaflow_plugin_sdk/schema.py", line 1644, in _unserialize_input
return step.input.unserialize(data)
File "/usr/local/lib/python3.9/site-packages/arcaflow_plugin_sdk/schema.py", line 1193, in unserialize
raise ConstraintException(path, "Must be a dict, got {}".format(type(data).__name__))
arcaflow_plugin_sdk.schema.ConstraintException: Validation failed: Must be a dict, got NoneType
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/arcaflow_plugin_sdk/plugin.py", line 648, in _execute_file
output_id, output_data = s(step_id, data)
File "/usr/local/lib/python3.9/site-packages/arcaflow_plugin_sdk/schema.py", line 1708, in __call__
input_param = self._unserialize_input(step, data)
File "/usr/local/lib/python3.9/site-packages/arcaflow_plugin_sdk/schema.py", line 1646, in _unserialize_input
raise InvalidInputException(e) from e
arcaflow_plugin_sdk.schema.InvalidInputException: Validation failed: Must be a dict, got NoneType Params file for reference: smallfile_params:
top: "/var/tmp"
operation: "create"
threads: 5
file-size: 64
files: 100
auto-pause: 'N'
fsync: 'N'
files-per-dir: 100
dirs-per-dir: 10
stonewall: 'Y'
finish: 'Y'
prefix: 'foo'
suffix: 'bar'
hash-into-dirs: 'N'
same-dir: 'N'
verify-read: 'Y'
incompressible: 'N'
cleanup-delay-usec-per-file: 0
|
I've updated the integration to use |
e69bebd
to
3773309
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested with both podman run -i
and docker run -i
, using both direct echo
of json and cat
of both yaml and json input files, and all conditions passed.
Updated readme for containerized run Now possible per arcalot/arcaflow-plugin-sdk-python#36 update plugin directory
Updated readme for containerized run Now possible per arcalot/arcaflow-plugin-sdk-python#36 update plugin directory
Updated readme for containerized run Now possible per arcalot/arcaflow-plugin-sdk-python#36 update plugin directory
Updated readme for containerized run Now possible per arcalot/arcaflow-plugin-sdk-python#36 update plugin directory
git rid of samples parameter enable tests in dockerfile convert operations parameters to enum enable yes/no enum for compatibility with smallfile boolean params significantly expanded supported input params comment cleanup stagify dockerfile variable format standardization autopep8 formatting remove .gitignore housekeeping fixes rsptimes serialization test fix add input params documentation Updated readme for containerized run Now possible per arcalot/arcaflow-plugin-sdk-python#36 update plugin directory separate schema module for better clarity document output schema handle errors for opening rsptimes csv file cleanup autopep8 remove per-plugin license file
add python plugin for smallfile git rid of samples parameter enable tests in dockerfile convert operations parameters to enum enable yes/no enum for compatibility with smallfile boolean params significantly expanded supported input params comment cleanup stagify dockerfile variable format standardization autopep8 formatting remove .gitignore housekeeping fixes rsptimes serialization test fix add input params documentation Updated readme for containerized run Now possible per arcalot/arcaflow-plugin-sdk-python#36 update plugin directory separate schema module for better clarity document output schema handle errors for opening rsptimes csv file cleanup autopep8 remove per-plugin license file
Changes introduced with this PR
This PR adds support for loading config files from the stdin by specifying
-f -
in plugins.By contributing to this repository, I agree to the contribution guidelines.