Skip to content

Commit

Permalink
Check whether required tests are executed. (microsoft#19884)
Browse files Browse the repository at this point in the history
### Description
Check the onnx node tests and model tests worked

### Motivation and Context
onnx node test data and model data are mount in one dir.
And onnxruntime_test_all search the dir and load the data.
If the dir does exist or there's some change in onnxruntime_test_all,
those tests may not be executed.
For example, all onnx node test data is 32M. It's hardly for us aware of
the regression.
So I add the simple check to ensure those tests are executed.

---------

Co-authored-by: Yi Zhang <[email protected]>
  • Loading branch information
mszhanyi and Yi Zhang authored Mar 13, 2024
1 parent 7313aa4 commit 9e0a0f0
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 2 deletions.
9 changes: 7 additions & 2 deletions tools/ci_build/github/azure-pipelines/linux-ci-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ stages:
-e NIGHTLY_BUILD \
-e BUILD_BUILDNUMBER \
onnxruntimecpubuildcentos8x64 \
/bin/bash -c "
/bin/bash -c '
set -ex; \
python3.9 /onnxruntime_src/tools/ci_build/build.py \
--build_dir /build --cmake_generator 'Ninja' \
Expand All @@ -105,7 +105,8 @@ stages:
--parallel --use_binskim_compliant_compile_flags \
--build_csharp \
--enable_onnx_tests --enable_address_sanitizer \
--test;"
--test;
'
workingDirectory: $(Build.SourcesDirectory)

- task: PublishTestResults@2
Expand Down Expand Up @@ -285,6 +286,10 @@ stages:
"
displayName: 'Run Release tests and symbolic shape infer test'
- template: templates/check_test_result.yml
parameters:
FileName: '$(Build.BinariesDirectory)/Release/onnxruntime_test_all.Release.results.xml'

- task: PublishTestResults@2
displayName: 'Publish unit test results'
inputs:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -265,4 +265,8 @@ stages:
'
displayName: 'Run Tests'

- template: templates/check_test_result.yml
parameters:
FileName: '$(Build.BinariesDirectory)/Release/onnxruntime_test_all.Release.results.xml'

- template: templates/clean-agent-build-directory-step.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
parameters:
- name: FileName
type: string

steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.x'
addToPath: true
architecture: 'x64'

- task: PythonScript@0
displayName: 'Check test result yml'
inputs:
scriptSource: 'inline'
script: |
with open('${{parameters.FileName}}', 'r') as file:
content = file.read()
assert 'data_onnx_opset' in content, "operator test not found in test result file"
assert 'models_zoo_opset' in content, "models_zoo model not found in test reuslt file"

0 comments on commit 9e0a0f0

Please sign in to comment.