Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenVINO EP Rel 1.18 Changes #20337

Merged
merged 27 commits into from
Apr 19, 2024
Merged

OpenVINO EP Rel 1.18 Changes #20337

merged 27 commits into from
Apr 19, 2024

Conversation

sfatimar
Copy link
Contributor

Description

These changes include
Support to OpenVINO 2024.1
Import PreCompiled Blobs with EPContext Blob
Separate Device/Precision as input
Deprecate CPU_FP32 , GPU_FP32 terminology , introduce CPU, GPU
AUTO GPU, CPU will only create GPU Blob and not CPU Blob.

Motivation and Context

  • OpenVINO 2024.1 will be out soon
  • Import Precompiled Blob can greatly reduce FEIL/FIL Time.
  • Separating Device/Precision will make the input cleaner

sfatimar and others added 13 commits April 16, 2024 13:49
@sfatimar
OpenVINO Commit 2024.1
b8163ee
@sspintel @preetha-intel
Add support to work with EPContext Models (with embedded compiled blo…
72a5897 
…bs) in OVEP
@sspintel @preetha-intel
Add provider option for exporting compiled blob
eceb1e9
@sspintel @preetha-intel
Apply lintrunner patch
03bf667
@sspintel @preetha-intel
Dump model with original model name + "-ov_blob.onnx" suffix
469abd4
@preetha-intel
Preetha/device precision change (#358)
307af50 
* Remove support for device_id in OVEP provider options

* Add precision as seperate OVEP cli option

* Fix lint issues
@sspintel @preetha-intel
Add model_priority hint as a provider option
753f673
@sspintel @preetha-intel
Fix build issue on Windows
0842263
@sspintel @preetha-intel
CPU Plugin doesn't support model priority hint
18f9d70
@sspintel @preetha-intel
Fix equality op
53a652a
@preetha-intel
Add deprecation notice for device_id session option (#360)
0f6ce55
@sfatimar
Merge pull request #363 from intel/sahar/openvino_2024.1_1.18
d9c6e67 
OpenVINO Commit 2024.1
@preetha-intel
Add custom blob generation for OV AUTO device (#364)
972d50d 
* Add custom blob generation for OV AUTO device

* Update build command in docker
@sfatimar
Copy link
Contributor Author

@jywu-msft

@jywu-msft
Copy link
Member

/azp run Linux OpenVINO CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jywu-msft
Copy link
Member

lint caught a couple typos,
also python lint/format required pipeline is failing. please take a look.

@jywu-msft
Copy link
Member

this test hit exception and was aborted.
1: [----------] 1 test from ParallelExecutor
1: [ RUN ] ParallelExecutor.TestStatusPropagation
1/7 Test #1: onnxruntime_test_all ....................Subprocess aborted***Exception: 279.98 sec

@jywu-msft
Copy link
Member

/azp run Linux CPU CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jywu-msft
Copy link
Member

/azp run Windows CPU CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@jywu-msft
Copy link
Member

/azp run Linux CPU CI Pipeline, Linux OpenVINO CI Pipeline, Windows CPU CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 3 pipeline(s).

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 8 pipeline(s).

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 9 pipeline(s).

@sspintel
Copy link
Contributor

@jywu-msft We had to pull in a critical fix so the pipelines need to be restarted (earlier macOS pipeline was failing due to an unrelated reason)

We don't expect lint failures, and so can you please help run all pipelines?

@preetha-intel
Copy link
Contributor

@jywu-msft There was a bug fix for one of the feature wrt NPU device. We have fixed it. Kindly help re-run the pipelines. We have fixed the lint and unit test cases failures. Anticipating no more changes to this PR.

@jywu-msft
Copy link
Member

@jywu-msft There was a bug fix for one of the feature wrt NPU device. We have fixed it. Kindly help re-run the pipelines. We have fixed the lint and unit test cases failures. Anticipating no more changes to this PR.

can you mark it ready for review (if no more expected code changes)
I'll kick off the pipelines

@jywu-msft
Copy link
Member

/azp run Windows ARM64 QNN CI Pipeline,Windows x64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,ONNX Runtime Web CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline

@jywu-msft
Copy link
Member

/azp run Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,onnxruntime-binary-size-checks-ci-pipeline,Big Models,Android CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 10 pipeline(s).

1 similar comment
@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 10 pipeline(s).

@sfatimar sfatimar marked this pull request as ready for review April 18, 2024 18:20
@sfatimar sfatimar requested a review from a team as a code owner April 18, 2024 18:20
@sfatimar
Copy link
Contributor Author

@jywu-msft PR is ready for review

@jywu-msft
Copy link
Member

MacOS failure is under investigation. (unrelated to this PR)

@jywu-msft
Copy link
Member

can you merge latest main?

@sspintel
Copy link
Contributor

@jywu-msft Merged latest main.

@jywu-msft
Copy link
Member

/azp run Windows ARM64 QNN CI Pipeline,Windows x64 QNN CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,ONNX Runtime Web CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline

@jywu-msft
Copy link
Member

/azp run Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,onnxruntime-binary-size-checks-ci-pipeline,Big Models,Android CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 8 pipeline(s).

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 10 pipeline(s).

@jywu-msft
Copy link
Member

/azp run Linux CPU CI Pipeline, Windows CPU CI Pipeline

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@jywu-msft jywu-msft merged commit 4d1963c into microsoft:main Apr 19, 2024
74 of 77 checks passed
TedThemistokleous pushed a commit to TedThemistokleous/onnxruntime that referenced this pull request May 7, 2024
@sfatimar @sspintel @preetha-intel
OpenVINO EP Rel 1.18 Changes (microsoft#20337)
78d1ccf 
### Description
These changes include
Support to OpenVINO 2024.1 
Import PreCompiled Blobs with EPContext Blob 
Separate Device/Precision as input
Deprecate CPU_FP32 , GPU_FP32 terminology , introduce CPU, GPU 
AUTO GPU, CPU will only create GPU Blob and not CPU Blob. 



### Motivation and Context
- OpenVINO 2024.1 will be out soon
- Import Precompiled Blob can greatly reduce FEIL/FIL Time. 
- Separating Device/Precision will make the input cleaner
-

---------

Co-authored-by: Suryaprakash Shanmugam <[email protected]>
Co-authored-by: Preetha Veeramalai <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jywu-msft jywu-msft jywu-msft approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sfatimar @jywu-msft @sspintel @preetha-intel