[CI] Tensorflow 2.3.1 upgrade blocker

[tqchen]

Followup of #6706 Some updates in TF and TFLite are needed to enable TF 2.3.1 support

https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/ci-docker-staging/28/pipeline/320

[ANSHUMAN87]

The failures are resolved in #6774

[leandron]

@tqchen I think this can be closed now

[leandron]

Sorry, too early to claim it is completely fixed :(. I just checked again and there was another side effect, this time on Keras testing.

When using pure Keras (not tf.Keras), all of them fail with the same error: AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'

___________________ TestKeras.test_forward_sequential[keras] ___________________

self = <test_forward.TestKeras object at 0x7f20fb333198>
keras = <module 'keras' from '/usr/local/lib/python3.6/dist-packages/keras/__init__.py'>

    def test_forward_sequential(self, keras):
        keras_model = keras.models.Sequential(
            [
                keras.layers.Dense(16, input_dim=32, activation="relu"),
                keras.layers.Dropout(0.5),
                keras.layers.Dense(8, activation="relu"),
                keras.layers.Dropout(0.5),
>               keras.layers.Dense(1, activation="sigmoid"),
            ]
        )

tests/python/frontend/keras/test_forward.py:215: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.6/dist-packages/keras/engine/sequential.py:94: in __init__
    self.add(layer)
/usr/local/lib/python3.6/dist-packages/keras/engine/sequential.py:166: in add
    layer(x)
/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:75: in symbolic_fn_wrapper
    return func(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/keras/engine/base_layer.py:446: in __call__
    self.assert_input_compatibility(inputs)
/usr/local/lib/python3.6/dist-packages/keras/engine/base_layer.py:310: in assert_input_compatibility
    K.is_keras_tensor(x)
/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:695: in is_keras_tensor
    if not is_tensor(x):
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

x = <tf.Tensor 'dense_1_input:0' shape=(None, 32) dtype=float32>

    def is_tensor(x):
>       return isinstance(x, tf_ops._TensorLike) or tf_ops.is_dense_tensor_like(x)
E       AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'

/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:703: AttributeError

I see this looks like tensorflow/models#6177 and tensorflow/tensorflow#38589. Apparently it is not fixed yet.

@ANSHUMAN87 are you able to reproduce this issues when running frontend tests?

[ANSHUMAN87]

@leandron : Sure i will look into it. Can you please confirm whether keras version in ur env is 2.3.1 ?

[u99127]

@ANSHUMAN87 - we just use the same Docker image as upstream ci_cpu,so it would be the version as in ubuntu_install_tensorflow.sh . So I think it's keras 2.3.1

install/ubuntu_install_tensorflow.sh:pip3 install tensorflow==2.3.1 keras==2.3.1 h5py

[leandron]

Yes, I confirm as per https://github.com/apache/incubator-tvm/blob/main/docker/install/ubuntu_install_tensorflow.sh and also by checking the installation logs.

[ANSHUMAN87]

Okay, i checked the failures, we have to upgrade the keras version too, along with it. Will raise one PR for it now.

[leandron]

I confirm now all the issues are gone.

[ANSHUMAN87]

Thanks @leandron for confirmation!

[tqchen]

Still having problem https://ci.tlcpack.ai/job/temp-ci-docker-staging/job/ci-stage2/25/execution/node/408/log/

[tqchen]

seems was due to #6808 changes to keras 2.3.1

[leandron]

Still having problem https://ci.tlcpack.ai/job/temp-ci-docker-staging/job/ci-stage2/25/execution/node/408/log/

Sorry, I reviewed it and it was something I changed wrong. New PR coming in a few seconds.

[leandron]

So, you need #6810 and #6808 on your ci branch, then according to my tests, all is working.