Skip to content
This repository has been archived by the owner on Dec 1, 2021. It is now read-only.

Apply tf_upgrade_v2 for utils #914

Merged
merged 2 commits into from
Mar 16, 2020
Merged

Apply tf_upgrade_v2 for utils #914

merged 2 commits into from
Mar 16, 2020

Conversation

ytfksw
Copy link
Contributor

@ytfksw ytfksw commented Mar 12, 2020

What this patch does to fix the issue.

This patch makes blueoil/utils compatible with tf2

Here's what I did:

install newest tenosrflow (tensorflow-gpu==1.5.2)
Run the upgrade script tf_upgrade_v2
Check the upgrade report for warnings and errors
Run test (make test)

Link to any relevant issues or pull requests.

@blueoil-butler blueoil-butler bot added the CI: auto-run Run CI automatically label Mar 12, 2020
@ytfksw
Copy link
Contributor Author

ytfksw commented Mar 12, 2020

This is the whole convert log:

TensorFlow 2.0 Upgrade Script
-----------------------------
Converted 13 files
Detected 3 issues that require attention
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
File: utils/predict_output/writer.py
--------------------------------------------------------------------------------
utils/predict_output/writer.py:57:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
utils/predict_output/writer.py:105:8: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
--------------------------------------------------------------------------------
File: utils/predict_output/output.py
--------------------------------------------------------------------------------
utils/predict_output/output.py:130:16: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
================================================================================
Detailed log follows:

================================================================================
================================================================================
Input tree: 'utils'
================================================================================
--------------------------------------------------------------------------------
Processing file 'utils/executor.py'
 outputting to 'utils/executor.py'
--------------------------------------------------------------------------------

91:24: INFO: Renamed 'tf.graph_util.convert_variables_to_constants' to 'tf.compat.v1.graph_util.convert_variables_to_constants'
117:9: INFO: `name` passed to `name_scope`. Because you may be re-entering an existing scope, it is not safe to convert automatically,  the v2 name_scope does not support re-entering scopes by name.

117:9: INFO: Renamed 'tf.name_scope' to 'tf.compat.v1.name_scope'
128:29: INFO: tf.summary.merge requires manual check. The TF 1.x summary API cannot be automatically migrated to TF 2.0, so symbols have been converted to tf.compat.v1.summary.* and must be migrated manually. Typical usage will only require changes to the summary writing logic, not to individual calls like scalar(). For examples of the new summary API, see the Effective TF 2.0 migration document or check the TF 2.0 TensorBoard tutorials.
128:29: INFO: Renamed 'tf.summary.merge' to 'tf.compat.v1.summary.merge'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/box.py'
 outputting to 'utils/box.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/config.py'
 outputting to 'utils/config.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/horovod.py'
 outputting to 'utils/horovod.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/__init__.py'
 outputting to 'utils/__init__.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/image.py'
 outputting to 'utils/image.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/module_loader.py'
 outputting to 'utils/module_loader.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/random.py'
 outputting to 'utils/random.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/predict_output/__init__.py'
 outputting to 'utils/predict_output/__init__.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/predict_output/writer.py'
 outputting to 'utils/predict_output/writer.py'
--------------------------------------------------------------------------------

57:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
105:8: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/predict_output/output.py'
 outputting to 'utils/predict_output/output.py'
--------------------------------------------------------------------------------

130:16: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/tfds_builders/object_detection.py'
 outputting to 'utils/tfds_builders/object_detection.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'utils/tfds_builders/classification.py'
 outputting to 'utils/tfds_builders/classification.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

@ytfksw
Copy link
Contributor Author

ytfksw commented Mar 13, 2020 •
edited
Loading

We got 3 warnings about keras.model.save

WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.

Since we don't use Keras.model now, it is irrelevant.

Actually, we call np.save or PIL.Image.save:
utils/predict_output/writer.py:57:4 (np.save)
utils/predict_output/writer.py:105:8 (PIL.Image.save)
utils/predict_output/output.py:130:16 (PIL.Image.save)

@ytfksw ytfksw marked this pull request as ready for review March 13, 2020 08:17
@bo-code-review-bot
Copy link

This PR needs Approvals as follows.

  • Ownership Approval for / from iizukak, tkng, ruimashita
  • Readability Approval for Python from tkng, tsawada

Please choose reviewers and requet reviews!

Click to see how to approve each reviews

You can approve this PR by triggered comments as follows.

  • Approve all reviews requested to you (readability and ownership) and LGTM review
    Approval, LGTM

  • Approve all ownership reviews
    Ownership Approval or OA

  • Approve all readability reviews
    Readability Approval or RA

  • Approve specified review targets

    • Example of Ownership Reviewer of /: Ownership Approval for / or OA for /
    • Example of Readability Reviewer of Python: Readability Approval for Python or RA for Python

  • Approve LGTM review
    LGTM

See all trigger comments

Please replace [Target] to review target

  • Ownership Approval
    • Ownership Approval for [Target]
    • OA for [Target]
    • Ownership Approval
    • OA
    • Approval
  • Readability Approval
    • Readability Approval for [Target]
    • RA for [Target]
    • [Target] Readability Approval
    • [Target] RA
    • Readability Approval
    • RA
    • Approval
  • LGTM
    • LGTM
    • lgtm

iizukak
iizukak approved these changes Mar 15, 2020
View reviewed changes
Copy link
Member

@iizukak iizukak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OA

@iizukak iizukak requested a review from tsawada March 15, 2020 23:59
tsawada
tsawada approved these changes Mar 16, 2020
View reviewed changes
Copy link
Contributor

@tsawada tsawada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Readability Approval

@iizukak
Copy link
Member

iizukak commented Mar 16, 2020

/ready

@bo-mergebot
Copy link
Contributor

bo-mergebot bot commented Mar 16, 2020

⏳Merge job is queued...

@bo-mergebot
Copy link
Contributor

bo-mergebot bot commented Mar 16, 2020

😥Status check failed. Please fix problems and send /ready again.

@tfujiwar
Copy link
Contributor

/ready

@bo-mergebot
Copy link
Contributor

bo-mergebot bot commented Mar 16, 2020

⏳Merge job is queued...

@bo-mergebot bo-mergebot bot merged commit f353c13 into blue-oil:master Mar 16, 2020
@ytfksw ytfksw mentioned this pull request May 29, 2020
Open
@ytfksw ytfksw self-assigned this Jun 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@iizukak iizukak iizukak approved these changes

@tsawada tsawada tsawada approved these changes

Assignees

@ytfksw ytfksw

Labels
CI: auto-run Run CI automatically
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ytfksw @iizukak @tfujiwar @tsawada