Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Test and build package using Azure Pipelines #1406

Merged
merged 7 commits into from
Feb 10, 2021
Merged

[ci] Test and build package using Azure Pipelines #1406

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
7543cc7
[ci] Test and build package using Azure Pipelines
jleveque Feb 6, 2021
20cd8c1
Call pip3 using sudo
jleveque Feb 6, 2021
5de2978
Attempt to run sonic-slave-buster container in privileged mode
jleveque Feb 9, 2021
e2e30e1
Omit more files from pytest coverage
jleveque Feb 9, 2021
5c08e61
Add coverage report types back to pytest.ini
jleveque Feb 9, 2021
a3728c7
Modify coverage
jleveque Feb 9, 2021
9bf31a3
Remove 'omit' section from .coveragerc
jleveque Feb 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .artifactignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
**/*
!dist/*.whl
29 changes: 29 additions & 0 deletions .coveragerc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
[run]
branch = True
source =
acl_loader
clear
config
connect
consutil
counterpoll
crm
debug
fdbutil
fwutil
pcieutil
pddf_fanutil
pddf_ledutil
pddf_psuutil
pddf_thermalutil
pfc
pfcwd
psuutil
scripts
sfputil
show
sonic_installer
ssdutil
undebug
utilities_common
watchdogutil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have to remember all the folder and update the folder when we have new one. is there way to include all of them?

Copy link
Contributor Author

@jleveque jleveque Feb 9, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this by using the omit = parameter here instead to skip directories we don't want to test. However, with this approach it appeared to only report coverage for directories which had > 0% coverage (i.e., directories which do not yet have any unit tests were not reported as having 0% coverage). This is no worse than the pervious solution, however, the list of directories is more organized and easy to read in .coveragerc than it was is pytest.ini.

I will continue looking into a way to make this better.

70 changes: 65 additions & 5 deletions azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,71 @@ trigger:
pool:
vmImage: ubuntu-20.04

container:
image: sonicdev-microsoft.azurecr.io:443/sonic-slave-buster:latest
options: --privileged
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should not do this, but I will accept it as a workaround.


steps:
- script: echo Hello, world!
displayName: 'Run a one-line script'
- task: DownloadPipelineArtifact@2
inputs:
source: specific
project: build
pipeline: 1
artifact: sonic-buildimage.kvm
runVersion: 'latestFromBranch'
runBranch: 'refs/heads/master'
displayName: "Download artifacts from latest sonic-buildimage build"

- script: |
set -xe
sudo dpkg -i libnl-3-200_*.deb
sudo dpkg -i libnl-genl-3-200_*.deb
sudo dpkg -i libnl-route-3-200_*.deb
sudo dpkg -i libnl-nf-3-200_*.deb
sudo dpkg -i libhiredis0.14_*.deb
sudo dpkg -i libswsscommon_1.0.0_amd64.deb
sudo dpkg -i python3-swsscommon_1.0.0_amd64.deb
sudo dpkg -i libyang_1.0.73_amd64.deb
sudo dpkg -i libyang-cpp_1.0.73_amd64.deb
sudo dpkg -i python3-yang_1.0.73_amd64.deb
workingDirectory: $(Pipeline.Workspace)/target/debs/buster/
displayName: 'Install Debian dependencies'

- script: |
set -xe
sudo pip3 install swsssdk-2.0.1-py3-none-any.whl
sudo pip3 install sonic_py_common-1.0-py3-none-any.whl
sudo pip3 install sonic_config_engine-1.0-py3-none-any.whl
sudo pip3 install sonic_platform_common-1.0-py3-none-any.whl
sudo pip3 install sonic_yang_mgmt-1.0-py3-none-any.whl
sudo pip3 install sonic_yang_models-1.0-py3-none-any.whl
workingDirectory: $(Pipeline.Workspace)/target/python-wheels/
displayName: 'Install Python dependencies'

- script: |
echo Add other tasks to build, test, and deploy your project.
echo See https://aka.ms/yaml
displayName: 'Run a multi-line script'
python3 setup.py test
displayName: 'Test Python 3'

- task: PublishTestResults@2
inputs:
testResultsFiles: '$(System.DefaultWorkingDirectory)/test-results.xml'
testRunTitle: Python 3
failTaskOnFailedTests: true
condition: succeededOrFailed()
displayName: 'Publish Python 3 test results'

- task: PublishCodeCoverageResults@1
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(System.DefaultWorkingDirectory)/coverage.xml'
reportDirectory: '$(System.DefaultWorkingDirectory)/htmlcov/'
displayName: 'Publish Python 3 test coverage'

- script: |
set -e
python3 setup.py bdist_wheel
displayName: 'Build Python 3 wheel'

- publish: '$(System.DefaultWorkingDirectory)/dist/'
artifact: wheels
displayName: "Publish Python wheels"
4 changes: 1 addition & 3 deletions pytest.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,2 @@
[pytest]
filterwarnings =
ignore::DeprecationWarning
addopts = --cov=acl_loader --cov=clear --cov=config --cov=connect --cov=consutil --cov=counterpoll --cov=crm --cov=debug --cov=fdbutil --cov=fwutil --cov=pcieutil --cov=pfcwd --cov=psuutil --cov=pddf_fanutil --cov=pddf_ledutil --cov=pddf_psuutil --cov=pddf_thermalutil --cov=scripts --cov=sfputil --cov=show --cov=sonic_installer --cov=ssdutil --cov=utilities_common --cov=watchdogutil --cov-report html --cov-report term --cov-report xml
addopts = --cov-config=.coveragerc --cov --cov-report html --cov-report term --cov-report xml --junitxml=test-results.xml -v