Remove log4j #1319

adriangonz · 2020-01-07T10:48:21Z

Fixes #1318

Changelog

Use HTTPS for training dataset link on H2O example.
Remove log4j from pom.xml on H2O example.

review-notebook-app · 2020-01-07T10:48:27Z

Check out this pull request on

You'll be able to see Jupyter notebook diff and discuss changes. Powered by ReviewNB.

seldondev · 2020-01-07T10:48:35Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: adriangonz

If they are not already assigned, you can assign the PR to them by writing /assign @adriangonz in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

seldondev · 2020-01-07T10:49:40Z

Tue Jan 7 10:49:39 UTC 2020
The logs for [lint] [2] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1319/2.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1319 --build=2

seldondev · 2020-01-07T10:49:51Z

Tue Jan 7 10:49:50 UTC 2020
The logs for [pr-build] [1] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1319/1.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1319 --build=1

axsaucedo · 2020-01-09T10:54:23Z

/test integration

seldondev · 2020-01-09T10:55:50Z

Thu Jan 9 10:55:46 UTC 2020
The logs for [integration] [3] will show after the pipeline context has finished.
https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1319/3.log

impatient try
jx get build logs SeldonIO/seldon-core/PR-1319 --build=3

seldondev · 2020-01-09T11:19:39Z

@adriangonz: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
integration	`c29865a`	link	`/test integration`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

axsaucedo · 2020-01-09T11:25:27Z

Integration tests seem to fail with the following error:


[gw1] PASSED test_rolling_updates.py::test_rolling_update_deployment[graph7.json-graph8.json]

=================================== FAILURES ===================================
___________ test_rolling_update_deployment[graph1.json-graph8.json] ____________
[gw3] linux -- Python 3.6.0 /usr/local/bin/python

from_deployment = 'graph1.json', to_deployment = 'graph8.json'

    @pytest.mark.parametrize(
        "from_deployment,to_deployment",
        [
            ("graph1.json", "graph8.json"),  # From v1alpha2 to v1
            ("graph7.json", "graph8.json"),  # From v1alpha3 to v1
        ],
    )
    def test_rolling_update_deployment(from_deployment, to_deployment):
        from_name = clean_string(from_deployment)
        to_name = clean_string(to_deployment)
        namespace = f"test-rolling-update-{from_name}-{to_name}"
        retry_run(f"kubectl create namespace {namespace}")

        from_file_path = to_resources_path(from_deployment)
        retry_run(f"kubectl apply -f {from_file_path} -n {namespace}")
        # Note that this is not yet parametrised!
        wait_for_rollout("mymodel-mymodel-e2eb561", namespace)

        logging.warning("Initial request")
        r = initial_rest_request("mymodel", namespace)
        assert r.status_code == 200
        assert r.json()["data"]["tensor"]["values"] == [1.0, 2.0, 3.0, 4.0]

        to_file_path = to_resources_path(to_deployment)
        retry_run(f"kubectl apply -f {to_file_path} -n {namespace}")
        r = initial_rest_request("mymodel", namespace)
        assert r.status_code == 200
        assert r.json()["data"]["tensor"]["values"] == [1.0, 2.0, 3.0, 4.0]

        i = 0
        for i in range(100):
            r = rest_request_ambassador("mymodel", namespace, API_AMBASSADOR)
>           assert r.status_code == 200
E           assert 503 == 200
E             -503
E             +200

test_rolling_updates.py:535: AssertionError
------------------------------ Captured log call -------------------------------

The full logs for the tests are:

============================= test session starts ==============================
platform linux -- Python 3.6.0, pytest-5.3.1, py-1.8.1, pluggy-0.13.1 -- /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /
plugins: xdist-1.30.0, forked-1.1.3, cov-2.8.1
gw0 I / gw1 I / gw2 I / gw3 I
[gw0] linux Python 3.6.0 cwd: /workspace/source/testing/scripts
[gw1] linux Python 3.6.0 cwd: /workspace/source/testing/scripts
[gw2] linux Python 3.6.0 cwd: /workspace/source/testing/scripts
[gw3] linux Python 3.6.0 cwd: /workspace/source/testing/scripts
[gw0] Python 3.6.0 (default, Jan  8 2020, 14:14:10)  -- [GCC 8.3.0]
[gw1] Python 3.6.0 (default, Jan  8 2020, 14:14:10)  -- [GCC 8.3.0]
[gw2] Python 3.6.0 (default, Jan  8 2020, 14:14:10)  -- [GCC 8.3.0]
[gw3] Python 3.6.0 (default, Jan  8 2020, 14:14:10)  -- [GCC 8.3.0]
gw0 [36] / gw1 [36] / gw2 [36] / gw3 [36]

scheduling tests via LoadScheduling

test_api_version.py::test_api_version[machinelearning.seldon.io/v1]
test_api_version.py::test_api_version[machinelearning.seldon.io/v1alpha2]
test_api_version.py::test_api_version[machinelearning.seldon.io/v1alpha3]
test_bad_graphs.py::TestBadGraphs::test_duplicate_predictor_name Error from server (SeldonDeployment.machinelearing.seldon.io "dupname" is invalid: [spec.predictors[1]: Invalid value: "mymodel": Duplicate predictor name, spec: Invalid value: "dupname": Traffic must sum to 100 for multiple predictors]): error when creating "../resources/bad_duplicate_predictor_name.json": admission webhook "vseldondeployment.kb.io" denied the request: SeldonDeployment.machinelearing.seldon.io "dupname" is invalid: [spec.predictors[1]: Invalid value: "mymodel": Duplicate predictor name, spec: Invalid value: "dupname": Traffic must sum to 100 for multiple predictors]

[gw3] PASSED test_bad_graphs.py::TestBadGraphs::test_duplicate_predictor_name
test_helm_charts_clusterwide.py::TestClusterWide::test_mab_model Error from server (NotFound): deployments.apps "mymodel-mymodel-7cd068f" not found
Error from server (NotFound): deployments.apps "mymodel-mymodel-7cd068f" not found
Error from server (NotFound): deployments.apps "mymab-mymab-41de5b8" not found
Error from server (NotFound): deployments.apps "mymodel-mymodel-7cd068f" not found
Error: uninstall: Release not loaded: mymodel: release: not found
Error: uninstall: Release not loaded: mymodel: release: not found
Error: uninstall: Release not loaded: mymab: release: not found
Error: uninstall: Release not loaded: mymodel: release: not found

[gw1] PASSED test_api_version.py::test_api_version[machinelearning.seldon.io/v1alpha3]
test_helm_charts_clusterwide.py::TestClusterWide::test_single_model
[gw0] PASSED test_api_version.py::test_api_version[machinelearning.seldon.io/v1alpha2]
test_bad_graphs.py::TestBadGraphs::test_model_name_mismatch Error from server (SeldonDeployment.machinelearing.seldon.io "namemismatch" is invalid: spec.predictors[0].graph: Invalid value: "complex-model_bad_name": Can't find container for Predictive Unit): error when creating "../resources/bad_name_mismatch.json": admission webhook "vseldondeployment.kb.io" denied the request: SeldonDeployment.machinelearing.seldon.io "namemismatch" is invalid: spec.predictors[0].graph: Invalid value: "complex-model_bad_name": Can't find container for Predictive Unit

[gw0] PASSED test_bad_graphs.py::TestBadGraphs::test_model_name_mismatch
test_local_operators.py::TestLocalOperators::test_namespace_operator
[gw2] PASSED test_api_version.py::test_api_version[machinelearning.seldon.io/v1]
test_helm_charts_clusterwide.py::TestClusterWide::test_abtest_model
[gw3] PASSED test_helm_charts_clusterwide.py::TestClusterWide::test_mab_model
test_local_operators.py::TestLocalOperators::test_labelled_operator Error from server (InternalError): error when creating "../resources/model_controller_id.yaml": Internal error occurred: failed calling webhook "mseldondeployment.kb.io": Post https://seldon-webhook-service.test-labelled-operator.svc:443/mutate-machinelearning-seldon-io-v1alpha2-seldondeployment?timeout=30s: dial tcp 10.109.150.26:443: connect: connection refused
Error from server (NotFound): deployments.apps "test-c1-example-cf749e0" not found
Error: uninstall: Release not loaded: mymodel: release: not found
Error: uninstall: Release not loaded: myabtest: release: not found

[gw1] PASSED test_helm_charts_clusterwide.py::TestClusterWide::test_single_model
test_prepackaged_servers.py::TestPrepack::test_tfserving Error from server (NotFound): deployments.apps "mnist-default-725903e" not found

[gw2] PASSED test_helm_charts_clusterwide.py::TestClusterWide::test_abtest_model
test_rolling_updates.py::TestRollingHttp::test_rolling_update1[localhost:8003]
[gw3] PASSED test_local_operators.py::TestLocalOperators::test_labelled_operator
test_prepackaged_servers.py::TestPrepack::test_sklearn Error from server (NotFound): deployments.apps "iris-default-4903e3c" not found

[gw0] PASSED test_local_operators.py::TestLocalOperators::test_namespace_operator
test_prepackaged_servers.py::TestPrepack::test_mlflow
[gw1] PASSED test_prepackaged_servers.py::TestPrepack::test_tfserving
test_prepackaged_servers.py::TestPrepack::test_xgboost Error from server (NotFound): error when deleting "../resources/graph2.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw3] PASSED test_prepackaged_servers.py::TestPrepack::test_sklearn
test_rolling_updates.py::TestRollingHttp::test_rolling_update2[localhost:8003]
[gw2] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update1[localhost:8003]
test_rolling_updates.py::TestRollingHttp::test_rolling_update1[localhost:8004]
[gw0] PASSED test_prepackaged_servers.py::TestPrepack::test_mlflow
test_rolling_updates.py::TestRollingHttp::test_rolling_update2[localhost:8004]
[gw1] PASSED test_prepackaged_servers.py::TestPrepack::test_xgboost
test_rolling_updates.py::TestRollingHttp::test_rolling_update3[localhost:8003] Error from server (NotFound): error when deleting "../resources/graph3.json": seldondeployments.machinelearning.seldon.io "mymodel" not found
Error from server (NotFound): error when deleting "../resources/graph2.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw3] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update2[localhost:8003]
test_rolling_updates.py::TestRollingHttp::test_rolling_update3[localhost:8004] Error from server (NotFound): error when deleting "../resources/graph3.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw2] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update1[localhost:8004]
test_rolling_updates.py::TestRollingHttp::test_rolling_update4[localhost:8003]
[gw0] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update2[localhost:8004]
test_rolling_updates.py::TestRollingHttp::test_rolling_update4[localhost:8004] Error from server (NotFound): error when deleting "../resources/graph4.json": seldondeployments.machinelearning.seldon.io "mymodel" not found
Error from server (NotFound): error when deleting "../resources/graph5.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw1] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update3[localhost:8003]
test_rolling_updates.py::TestRollingHttp::test_rolling_update5[localhost:8003] Error from server (NotFound): error when deleting "../resources/graph4.json": seldondeployments.machinelearning.seldon.io "mymodel" not found
Error from server (NotFound): error when deleting "../resources/graph5.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw3] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update3[localhost:8004]
test_rolling_updates.py::TestRollingHttp::test_rolling_update5[localhost:8004] Error from server (NotFound): deployments.apps "mymodel-mymodel-e2eb561" not found

[gw2] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update4[localhost:8003]
test_rolling_updates.py::TestRollingHttp::test_rolling_update6[localhost:8003]
[gw0] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update4[localhost:8004]
test_rolling_updates.py::TestRollingHttp::test_rolling_update6[localhost:8004] Error from server (NotFound): error when deleting "../resources/graph6.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw1] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update5[localhost:8003]
test_rolling_updates.py::TestRollingHttp::test_rolling_update7[localhost:8003] Error from server (NotFound): error when deleting "../resources/graph6.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw3] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update5[localhost:8004]
test_rolling_updates.py::TestRollingHttp::test_rolling_update7[localhost:8004] Error from server (NotFound): error when deleting "../resources/graph2svc.json": seldondeployments.machinelearning.seldon.io "mymodel" not found
Error from server (NotFound): error when deleting "../resources/graph2svc.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw2] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update6[localhost:8003]
test_rolling_updates.py::TestRollingHttp::test_rolling_update8[localhost:8003]
[gw0] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update6[localhost:8004]
test_rolling_updates.py::TestRollingHttp::test_rolling_update8[localhost:8004] Error from server (NotFound): error when deleting "../resources/graph3svc.json": seldondeployments.machinelearning.seldon.io "mymodel" not found
Error from server (NotFound): error when deleting "../resources/graph3svc.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw3] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update7[localhost:8004]
test_rolling_updates.py::TestRollingHttp::test_rolling_update9[localhost:8004]
[gw1] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update7[localhost:8003]
test_rolling_updates.py::TestRollingHttp::test_rolling_update9[localhost:8003] Error from server (NotFound): error when deleting "../resources/graph4svc.json": seldondeployments.machinelearning.seldon.io "mymodel" not found
Error from server (NotFound): error when deleting "../resources/graph4svc.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw2] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update8[localhost:8003]
test_rolling_updates.py::TestRollingHttp::test_rolling_update10[localhost:8003]
[gw0] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update8[localhost:8004]
test_rolling_updates.py::TestRollingHttp::test_rolling_update10[localhost:8004] Error from server (NotFound): error when deleting "../resources/graph5svc.json": seldondeployments.machinelearning.seldon.io "mymodel" not found
Error from server (NotFound): error when deleting "../resources/graph5svc.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw3] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update9[localhost:8004]
test_rolling_updates.py::test_rolling_update_deployment[graph1.json-graph8.json] Error from server (NotFound): deployments.apps "mymodel-mymodel-e2eb561" not found

[gw1] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update9[localhost:8003]
test_rolling_updates.py::test_rolling_update_deployment[graph7.json-graph8.json] Error from server (NotFound): error when deleting "../resources/graph6svc.json": seldondeployments.machinelearning.seldon.io "mymodel" not found
Error from server (NotFound): error when deleting "../resources/graph6svc.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw2] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update10[localhost:8003]
[gw0] PASSED test_rolling_updates.py::TestRollingHttp::test_rolling_update10[localhost:8004]
[gw3] FAILED test_rolling_updates.py::test_rolling_update_deployment[graph1.json-graph8.json] Error from server (NotFound): error when deleting "../resources/graph8.json": seldondeployments.machinelearning.seldon.io "mymodel" not found

[gw1] PASSED test_rolling_updates.py::test_rolling_update_deployment[graph7.json-graph8.json]

=================================== FAILURES ===================================
___________ test_rolling_update_deployment[graph1.json-graph8.json] ____________
[gw3] linux -- Python 3.6.0 /usr/local/bin/python

from_deployment = 'graph1.json', to_deployment = 'graph8.json'

    @pytest.mark.parametrize(
        "from_deployment,to_deployment",
        [
            ("graph1.json", "graph8.json"),  # From v1alpha2 to v1
            ("graph7.json", "graph8.json"),  # From v1alpha3 to v1
        ],
    )
    def test_rolling_update_deployment(from_deployment, to_deployment):
        from_name = clean_string(from_deployment)
        to_name = clean_string(to_deployment)
        namespace = f"test-rolling-update-{from_name}-{to_name}"
        retry_run(f"kubectl create namespace {namespace}")

        from_file_path = to_resources_path(from_deployment)
        retry_run(f"kubectl apply -f {from_file_path} -n {namespace}")
        # Note that this is not yet parametrised!
        wait_for_rollout("mymodel-mymodel-e2eb561", namespace)

        logging.warning("Initial request")
        r = initial_rest_request("mymodel", namespace)
        assert r.status_code == 200
        assert r.json()["data"]["tensor"]["values"] == [1.0, 2.0, 3.0, 4.0]

        to_file_path = to_resources_path(to_deployment)
        retry_run(f"kubectl apply -f {to_file_path} -n {namespace}")
        r = initial_rest_request("mymodel", namespace)
        assert r.status_code == 200
        assert r.json()["data"]["tensor"]["values"] == [1.0, 2.0, 3.0, 4.0]

        i = 0
        for i in range(100):
            r = rest_request_ambassador("mymodel", namespace, API_AMBASSADOR)
>           assert r.status_code == 200
E           assert 503 == 200
E             -503
E             +200

test_rolling_updates.py:535: AssertionError
------------------------------ Captured log call -------------------------------
WARNING  root:seldon_e2e_utils.py:62 Successfully ran command: kubectl create namespace test-rolling-update-graph1-json-graph8-json
WARNING  root:seldon_e2e_utils.py:62 Successfully ran command: kubectl apply -f ../resources/graph1.json -n test-rolling-update-graph1-json-graph8-json
WARNING  root:seldon_e2e_utils.py:53 Unsuccessful wait command but retrying for mymodel-mymodel-e2eb561
WARNING  root:seldon_e2e_utils.py:51 Successfully waited for deployment mymodel-mymodel-e2eb561
WARNING  root:test_rolling_updates.py:521 Initial request
WARNING  root:seldon_e2e_utils.py:108 Bad status:404
WARNING  root:seldon_e2e_utils.py:138 Sleeping 1 sec and trying again
WARNING  root:seldon_e2e_utils.py:108 Bad status:404
WARNING  root:seldon_e2e_utils.py:151 Sleeping 5 sec and trying again
WARNING  root:seldon_e2e_utils.py:62 Successfully ran command: kubectl apply -f ../resources/graph8.json -n test-rolling-update-graph1-json-graph8-json
=================== 1 failed, 35 passed in 951.18s (0:15:51) ===================
make: *** [Makefile:76: test] Error 1
kind delete cluster
Deleting cluster "kind" ...
Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die.

RafalSkolasinski · 2020-01-09T11:34:17Z

Interesting. I don't see how changes in PR could affect this test?

adriangonz · 2020-01-09T11:46:43Z

It may be something introduced by a previous PR or it may be an intermittent 503 error on the integration tests due to extra load in the testing cluster.

I'll try to replicate this locally.

adriangonz · 2020-01-13T09:31:39Z

Tests seem to pass locally, so I thinks it's safe to assume that this was an intermittent failure.

Adrian Gonzalez-Martin added 3 commits January 7, 2020 10:09

Use https for training set

2ebbbc6

Remove log4j from pom

23dece6

Update link

c29865a

seldondev requested review from axsaucedo and RafalSkolasinski January 7, 2020 10:48

seldondev added the size/XXL label Jan 7, 2020

adriangonz merged commit 4eeb3c8 into SeldonIO:master Jan 13, 2020

adriangonz deleted the update-log4j branch January 13, 2020 13:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove log4j #1319

Remove log4j #1319

adriangonz commented Jan 7, 2020

review-notebook-app bot commented Jan 7, 2020

seldondev commented Jan 7, 2020

seldondev commented Jan 7, 2020

seldondev commented Jan 7, 2020

axsaucedo commented Jan 9, 2020

seldondev commented Jan 9, 2020

seldondev commented Jan 9, 2020

axsaucedo commented Jan 9, 2020

RafalSkolasinski commented Jan 9, 2020

adriangonz commented Jan 9, 2020

adriangonz commented Jan 13, 2020

Remove log4j #1319

Remove log4j #1319

Conversation

adriangonz commented Jan 7, 2020

Changelog

review-notebook-app bot commented Jan 7, 2020

seldondev commented Jan 7, 2020

seldondev commented Jan 7, 2020

seldondev commented Jan 7, 2020

axsaucedo commented Jan 9, 2020

seldondev commented Jan 9, 2020

seldondev commented Jan 9, 2020

axsaucedo commented Jan 9, 2020

RafalSkolasinski commented Jan 9, 2020

adriangonz commented Jan 9, 2020

adriangonz commented Jan 13, 2020