Add Docker image for Python Testing #24235

khalludi · 2021-11-04T10:09:48Z

Separates Docker test image code from PR #24034 . This allows us to test that the Docker test image is being built before it is used in the referenced PR.

This PR also partially takes care of the root issue #23830.

(@chaodaiG, please review at your convenience)

k8s-ci-robot · 2021-11-04T10:09:56Z

Hi @khalludi. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

chaodaiG

/ok-to-test

chaodaiG · 2021-11-04T13:49:30Z

config/jobs/image-pushing/k8s-staging-test-infra.yaml

+      annotations:
+        testgrid-dashboards: sig-testing-images, sig-k8s-infra-gcb
+        testgrid-tab-name: "pytest"
+        testgrid-alert-email: [email protected], [email protected]


Let's not alert until we know it works

Please delete this line

chaodaiG · 2021-11-04T13:49:54Z

config/jobs/image-pushing/k8s-staging-test-infra.yaml

+          - --scratch-bucket=gs://k8s-staging-test-infra-gcb
+          - --project=k8s-staging-test-infra
+          - --build-dir=.
+          - images/pytest


add a new line at the end of the file, please also do this to the other files

chaodaiG · 2021-11-04T13:52:17Z

images/pytest/Dockerfile

+COPY requirements3.txt /app/requirements3.txt
+
+RUN /bin/bash -c "apt-get update && apt-get install -y make wget"
+RUN /bin/bash -c "pip3 install -r requirements3.txt"


When a PR updates requirements3.txt, do we expect make pytests also run pip3 install -r requirements3.txt?

chaodaiG · 2021-11-04T13:58:50Z

config/jobs/image-pushing/k8s-staging-test-infra.yaml

@@ -273,3 +273,27 @@ postsubmits:
          - --project=k8s-staging-test-infra
          - --env-passthrough=PULL_BASE_REF
          - triage/
+    -name: post-test-infra-push-pytest


test failed, add a space between - and name

khalludi · 2021-11-04T14:34:05Z

it seems like the 2 tests are failing because of py3_deps. Is there a way we can outright ignore this since we are deprecating it?

chaodaiG · 2021-11-04T14:36:05Z

We might not want to ignore them in this PR, at least not yet. I think the reason why the tests failed is because the changes in requirements3.txt file, one idea is revert the change in the file, and do pip3 install in make rules for now. What do you think?

khalludi · 2021-11-05T05:31:21Z

It looks like all the tests are passing this time. Are we green to merge?

chaodaiG

Please take a look at my comments

chaodaiG · 2021-11-05T13:39:25Z

config/jobs/image-pushing/k8s-staging-test-infra.yaml

+      annotations:
+        testgrid-dashboards: sig-testing-images, sig-k8s-infra-gcb
+        testgrid-tab-name: "pytest"
+        testgrid-alert-email: [email protected], [email protected]


Please delete this line

chaodaiG · 2021-11-05T13:39:39Z

images/pytest/Dockerfile

+COPY requirements3.txt /app/requirements3.txt
+
+RUN /bin/bash -c "apt-get update && apt-get install -y make wget"
+RUN /bin/bash -c "pip3 install -r requirements3.txt"


chaodaiG · 2021-11-10T14:50:30Z

images/pytest/Dockerfile

+RUN /bin/bash -c "pip3 install certifi chardet google-api-core google-cloud-bigquery google-cloud-pubsub idna python-dateutil pytz requests six urllib3"
+RUN /bin/bash -c "chmod a+x get_jq.sh && ./get_jq.sh && mv jq-1.5 .."
+
+CMD ["make", "pytests"]


add a new line at the end of this file, as well as other files

chaodaiG · 2021-11-10T14:51:43Z

requirements3.txt

@@ -9,4 +9,4 @@ PyYAML==5.3
 ruamel.yaml==0.16.5
 setuptools==44.0.0
 sh==1.12.14
-singledispatch==3.4.0.3
+singledispatch==3.4.0.3


I don't quite understand why this is considered a change, I'm afraid that there is some encoding thing going on, can you revert changes in this file?

chaodaiG · 2021-11-10T14:58:22Z

/label tide/merge-method-squash

khalludi · 2021-11-11T07:52:15Z

Seems like adding the new line at end of requirement3.txt reverted it back to what's in master branch appropriately. Also added \n to the end of all files.

chaodaiG

Thank you!

k8s-ci-robot · 2021-11-11T14:45:37Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chaodaiG, khalludi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [chaodaiG]
~~config/jobs/image-pushing/OWNERS~~ [chaodaiG]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2021-11-11T15:02:14Z

@khalludi: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

key k8s-staging-test-infra.yaml using file config/jobs/image-pushing/k8s-staging-test-infra.yaml

In response to this:

Separates Docker test image code from PR #24034 . This allows us to test that the Docker test image is being built before it is used in the referenced PR.

This PR also partially takes care of the root issue #23830.

(@chaodaiG, please review at your convenience)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Add Docker image for pytest

3be17c9

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 4, 2021

k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Nov 4, 2021

k8s-ci-robot requested review from chaodaiG and cjwagner November 4, 2021 10:10

k8s-ci-robot added area/images area/metrics sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Nov 4, 2021

chaodaiG reviewed Nov 4, 2021

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 4, 2021

chaodaiG reviewed Nov 4, 2021

View reviewed changes

khalludi added 2 commits November 4, 2021 09:03

Fix typo in yaml

36b33c3

Add exec perms to get_jq.sh

b470bd0

khalludi added 2 commits November 4, 2021 11:35

Move new requirements to Dockerfile

760d92c

Add license agreements

d1d1c5f

chaodaiG reviewed Nov 5, 2021

View reviewed changes

Remove testgrid-alert-email

fee2c1a

khalludi added a commit to khalludi/test-infra that referenced this pull request Nov 10, 2021

Moved image pushing to PR kubernetes#24235

c9d64fd

chaodaiG reviewed Nov 10, 2021

View reviewed changes

k8s-ci-robot added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Nov 10, 2021

khalludi added 2 commits November 11, 2021 01:48

Add newline to end of files

9bea2ee

Add newline to requirements3.txt

339c6df

chaodaiG approved these changes Nov 11, 2021

View reviewed changes

k8s-ci-robot assigned chaodaiG Nov 11, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 11, 2021

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 11, 2021

k8s-ci-robot merged commit 28fa43e into kubernetes:master Nov 11, 2021

k8s-ci-robot added this to the v1.23 milestone Nov 11, 2021

palnabarun mentioned this pull request Nov 15, 2021

Add merge restrictions for Kubernetes 1.23 Code Freeze palnabarun/test-infra#1

Closed

JohnNiang mentioned this pull request Nov 16, 2021

Add additional_labels for label plugin kubesphere/test-infra#33

Merged

khalludi added a commit to khalludi/test-infra that referenced this pull request Nov 21, 2021

Moved image pushing to PR kubernetes#24235

72e190f

khalludi mentioned this pull request Nov 21, 2021

Convert Python Bazel targets to Make targets #24034

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Docker image for Python Testing #24235

Add Docker image for Python Testing #24235

khalludi commented Nov 4, 2021

k8s-ci-robot commented Nov 4, 2021

chaodaiG left a comment

chaodaiG Nov 4, 2021

chaodaiG Nov 5, 2021

khalludi Nov 10, 2021

chaodaiG Nov 4, 2021

chaodaiG Nov 4, 2021

chaodaiG Nov 5, 2021

chaodaiG Nov 4, 2021

khalludi Nov 4, 2021

khalludi commented Nov 4, 2021

chaodaiG commented Nov 4, 2021

khalludi commented Nov 5, 2021

chaodaiG left a comment

chaodaiG Nov 5, 2021

chaodaiG Nov 5, 2021

chaodaiG Nov 10, 2021

chaodaiG Nov 10, 2021

chaodaiG commented Nov 10, 2021

khalludi commented Nov 11, 2021

chaodaiG left a comment

k8s-ci-robot commented Nov 11, 2021

k8s-ci-robot commented Nov 11, 2021

Add Docker image for Python Testing #24235

Add Docker image for Python Testing #24235

Conversation

khalludi commented Nov 4, 2021

k8s-ci-robot commented Nov 4, 2021

chaodaiG left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

khalludi commented Nov 4, 2021

chaodaiG commented Nov 4, 2021

khalludi commented Nov 5, 2021

chaodaiG left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chaodaiG commented Nov 10, 2021

khalludi commented Nov 11, 2021

chaodaiG left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Nov 11, 2021

k8s-ci-robot commented Nov 11, 2021