Model updates with side inputs #25174

AnandInguva · 2023-01-25T23:02:34Z

Add side input to the RunInference base transform.

Side inputs passed to RunInference DoFn must be singleton, and should be wrapped around beam.ml.inference.base.ModelMetadata.

TODO:

Add documentation
Add feature to change log
Follow up PR on WatchFileTransform PTransform and an example on how to use side inputs with RunInference

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

fix up parentheses Add time.sleep Update PredictionResult Update Pcollection type hint Add model_path param in the ModelHandler

refactor name

Update example

Fix windowing

Update model path in pytorch_inference.py Add model path to sklearn model Add model id to sklearn inference PredictionResult clean up base.py Update utils update base.py Update base.py

Add default value Add prefix/human readable id to ModelMetadata

github-actions · 2023-01-26T00:16:15Z

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

AnandInguva · 2023-01-26T04:39:40Z

Run Python 3.8 PostCommit

AnandInguva · 2023-01-26T04:55:11Z

Run Python 3.8 PostCommit

AnandInguva · 2023-01-26T04:59:24Z

Run Python 3.8 PostCommit

codecov · 2023-01-26T05:18:48Z

Codecov Report

Merging #25174 (99e14f8) into master (428ec97) will decrease coverage by 0.06%.
The diff coverage is 65.93%.

❗ Current head 99e14f8 differs from pull request most recent head 5e7ca32. Consider uploading reports for the commit 5e7ca32 to get more accurate results

@@            Coverage Diff             @@
##           master   #25174      +/-   ##
==========================================
- Coverage   73.12%   73.07%   -0.06%     
==========================================
  Files         735      741       +6     
  Lines       98161    98822     +661     
==========================================
+ Hits        71779    72212     +433     
- Misses      25019    25247     +228     
  Partials     1363     1363

Flag	Coverage Δ
python	`82.50% <65.93%> (-0.17%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...thon/apache_beam/ml/inference/pytorch_inference.py	`0.00% <0.00%> (ø)`
...hon/apache_beam/ml/inference/tensorrt_inference.py	`0.00% <0.00%> (ø)`
sdks/python/apache_beam/ml/inference/base.py	`91.37% <76.92%> (-4.39%)`	⬇️
...thon/apache_beam/ml/inference/sklearn_inference.py	`96.25% <100.00%> (-0.05%)`	⬇️
sdks/python/apache_beam/ml/inference/utils.py	`100.00% <100.00%> (ø)`
sdks/python/apache_beam/utils/python_callable.py	`86.88% <0.00%> (-11.48%)`	⬇️
.../python/apache_beam/typehints/trivial_inference.py	`95.89% <0.00%> (-0.30%)`	⬇️
sdks/python/apache_beam/transforms/util.py	`96.24% <0.00%> (-0.16%)`	⬇️
sdks/python/apache_beam/transforms/external.py	`78.74% <0.00%> (-0.14%)`	⬇️
...ache/beam/model/fn_execution/v1/beam_fn_api_pb2.py	`100.00% <0.00%> (ø)`
... and 19 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

AnandInguva · 2023-01-26T05:22:15Z

Run Python 3.8 PostCommit

github-actions · 2023-01-26T06:07:57Z

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @tvalentyn for label python.
R: @Abacn for label build.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

AnandInguva · 2023-01-26T14:19:19Z

stop reviewer notifications

github-actions · 2023-01-26T14:20:43Z

Stopping reviewer notifications for this pull request: requested by reviewer

AnandInguva · 2023-01-26T14:26:03Z

Run Python 3.8 PostCommit

AnandInguva · 2023-01-26T19:49:02Z

Some mypy related errors I am facing. Seems like the Generic NamedTuples are not supported in the mypy version 0.782.

If I update mypy, it is causing unrelated errors.

apache_beam/ml/inference/utils.py:45: error: Argument 2 to "PredictionResult" has incompatible type "Dict[Any, Any]"; expected "_OUTPUT_TYPE"  [arg-type]
apache_beam/ml/inference/base_test.py:91: error: Argument "example" to "PredictionResult" has incompatible type "int"; expected "_INPUT_TYPE"  [arg-type]
apache_beam/ml/inference/base_test.py:92: error: Argument "inference" to "PredictionResult" has incompatible type "int"; expected "_OUTPUT_TYPE"  [arg-type]
apache_beam/ml/inference/tensorrt_inference_test.py:55: error: Argument 1 to "PredictionResult" has incompatible type "ndarray[Any, Any]"; expected "_INPUT_TYPE"  [arg-type]
apache_beam/ml/inference/tensorrt_inference_test.py:55: error: Argument 2 to "PredictionResult" has incompatible type "List[ndarray[Any, Any]]"; expected "_OUTPUT_TYPE"  [arg-type]
apache_beam/ml/inference/tensorrt_inference_test.py:63: error: Argument 1 to "PredictionResult" has incompatible type "ndarray[Any, Any]"; expected "_INPUT_TYPE"  [arg-type]
apache_beam/ml/inference/tensorrt_inference_test.py:63: error: Argument 2 to "PredictionResult" has incompatible type "List[ndarray[Any, Any]]"; expected "_OUTPUT_TYPE"  [arg-type]
apache_beam/ml/inference/tensorrt_inference_test.py:78: error: Argument 1 to "PredictionResult" has incompatible type "ndarray[Any, Any]"; expected "_INPUT_TYPE"  [arg-type]
apache_beam/ml/inference/tensorrt_inference_test.py:78: error: Argument 2 to "PredictionResult" has incompatible type "List[ndarray[Any, Any]]"; expected "_OUTPUT_TYPE"  [arg-type]
apache_beam/ml/inference/pytorch_inference_test.py:72: error: Argument 2 to "PredictionResult" has incompatible type "Dict[str, _OUTPUT_TYPE]"; expected "_OUTPUT_TYPE"  [arg-type]
apache_beam/ml/inference/pytorch_inference_test.py:97: error: Argument 1 to "PredictionResult" has incompatible type "Dict[str, Any]"; expected "_INPUT_TYPE"  [arg-type]
apache_beam/ml/inference/pytorch_inference_test.py:105: error: Argument 1 to "PredictionResult" has incompatible type "Dict[str, Any]"; expected "_INPUT_TYPE"  [arg-type]
apache_beam/ml/inference/pytorch_inference_test.py:115: error: Argument 2 to "PredictionResult" has incompatible type "Dict[str, _OUTPUT_TYPE]"; expected "_OUTPUT_TYPE"  [arg-type]

So modifying new class of NamedTuple to accept default value for model_id

AnandInguva · 2023-01-26T22:31:42Z

Run Python 3.8 PostCommit

AnandInguva · 2023-01-26T22:50:30Z

https://ci-beam.apache.org/job/beam_PostCommit_Python38_PR/688/console

Successful test in post commit

AnandInguva · 2023-01-27T04:55:48Z

Closing because of #25200

AnandInguva added 19 commits January 20, 2023 20:25

Add model pcoll param to the RunInference Ptransform

74d72cf

fix up parentheses Add time.sleep Update PredictionResult Update Pcollection type hint Add model_path param in the ModelHandler

Add sklearn side input example

36a7db4

refactor name

Add WatchFilePattern PTransform

40b3d19

Add ModeMetadata and some refactoring

857766d

Update example

Add GetLatestFileByTimeStamp

9d59040

Fix windowing

refactor _convert_to_result and add it to the utils.py

3aa4c8c

Update model path in pytorch_inference.py Add model path to sklearn model Add model id to sklearn inference PredictionResult clean up base.py Update utils update base.py Update base.py

Add tag to the RunInference DoFn

f87ceb9

Add enable_side_input_loading flag

f82238b

Add helper functions

485bfd1

Add default value Add prefix/human readable id to ModelMetadata

Add doc string, refactor utils code

02a2386

Fix pytorch inference tests

3f2bb77

Fix up sklearn inference

1e41460

Remove logging

6dc0a18

Add thread Lock when there is an update to side input

57fbd5c

Check if side input is EmptySideInput

9768aac

Change interval on processing time

ddb4663

Add unit test for side input loading

8fde482

Remove examples

3eeae3c

Add log when side input path is updated

193e561

github-actions bot added the python label Jan 25, 2023

AnandInguva force-pushed the model_update_api branch from 73d250a to fde80b4 Compare January 26, 2023 04:54

github-actions bot added the build label Jan 26, 2023

Add test to Dataflow

f709338

AnandInguva force-pushed the model_update_api branch from fde80b4 to f709338 Compare January 26, 2023 04:58

github-actions bot added the Next Action: Reviewers label Jan 26, 2023

AnandInguva marked this pull request as draft January 26, 2023 14:19

AnandInguva added 2 commits January 26, 2023 10:18

Fix snippets tests

dfde454

revert build settings

d678794

github-actions bot added examples and removed build labels Jan 26, 2023

AnandInguva force-pushed the model_update_api branch from bbc44ad to f8480d2 Compare January 26, 2023 15:46

Fix lint

20d8a4a

AnandInguva force-pushed the model_update_api branch from f8480d2 to 20d8a4a Compare January 26, 2023 15:58

AnandInguva added 2 commits January 26, 2023 17:16

Refactor side input loading code

18b33a2

Add model_id to PredictionResult

99e14f8

github-actions bot added the build label Jan 26, 2023

AnandInguva force-pushed the model_update_api branch from 57cd43a to 99e14f8 Compare January 26, 2023 22:50

github-actions bot removed the build label Jan 26, 2023

Add documentation, changelog

5e7ca32

github-actions bot added the website label Jan 27, 2023

AnandInguva closed this Jan 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Model updates with side inputs #25174

Model updates with side inputs #25174

AnandInguva commented Jan 25, 2023 •

edited

Loading

github-actions bot commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

codecov bot commented Jan 26, 2023 •

edited

Loading

AnandInguva commented Jan 26, 2023

github-actions bot commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

github-actions bot commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

AnandInguva commented Jan 26, 2023 •

edited

Loading

AnandInguva commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

AnandInguva commented Jan 27, 2023

Model updates with side inputs #25174

Model updates with side inputs #25174

Conversation

AnandInguva commented Jan 25, 2023 • edited Loading

GitHub Actions Tests Status (on master branch)

github-actions bot commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

codecov bot commented Jan 26, 2023 • edited Loading

Codecov Report

AnandInguva commented Jan 26, 2023

github-actions bot commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

github-actions bot commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

AnandInguva commented Jan 26, 2023 • edited Loading

AnandInguva commented Jan 26, 2023

AnandInguva commented Jan 26, 2023

AnandInguva commented Jan 27, 2023

AnandInguva commented Jan 25, 2023 •

edited

Loading

codecov bot commented Jan 26, 2023 •

edited

Loading

AnandInguva commented Jan 26, 2023 •

edited

Loading