tf_upgrade_v2 for cmd #875

ytfksw · 2020-02-21T12:54:17Z

What this patch does to fix the issue.

I converted only the blueoil/cmd folder with the command line tool tf_upgrade_v2 to run on tensorflow2 first.
It seems difficult to convert at once, so I applied it only to blueoil / cmd.

Here's what I did:

install newest tenosrflow (tensorflow-gpu==1.5.2)
Run the upgrade script tf_upgrade_v2
Check the upgrade report for warnings and errors
Run test (make test)

The only change this time is to replace the operator with compat.v1.

Link to any relevant issues or pull requests.

ytfksw · 2020-02-21T12:55:24Z

This is the whole log file.

TensorFlow 2.0 Upgrade Script
-----------------------------
Converted 14 files
Detected 7 issues that require attention
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
File: cmd/export.py
--------------------------------------------------------------------------------
cmd/export.py:46:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
cmd/export.py:47:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
cmd/export.py:49:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
cmd/export.py:52:8: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
--------------------------------------------------------------------------------
File: cmd/train.py
--------------------------------------------------------------------------------
cmd/train.py:38:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
--------------------------------------------------------------------------------
File: cmd/tune_ray.py
--------------------------------------------------------------------------------
cmd/tune_ray.py:258:15: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
--------------------------------------------------------------------------------
File: cmd/convert_weight_from_darknet.py
--------------------------------------------------------------------------------
cmd/convert_weight_from_darknet.py:135:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
================================================================================
Detailed log follows:

================================================================================
================================================================================
Input tree: 'cmd'
================================================================================
--------------------------------------------------------------------------------
Processing file 'cmd/export.py'
 outputting to 'cmd/export.py'
--------------------------------------------------------------------------------

46:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
47:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
49:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
52:8: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
98:18: INFO: Renamed 'tf.global_variables_initializer' to 'tf.compat.v1.global_variables_initializer'
102:21: INFO: Renamed 'tf.ConfigProto' to 'tf.compat.v1.ConfigProto'
103:11: INFO: Renamed 'tf.Session' to 'tf.compat.v1.Session'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/init.py'
 outputting to 'cmd/init.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/train.py'
 outputting to 'cmd/train.py'
--------------------------------------------------------------------------------

38:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
132:18: INFO: Renamed 'tf.global_variables_initializer' to 'tf.compat.v1.global_variables_initializer'
133:27: INFO: Renamed 'tf.local_variables_initializer' to 'tf.compat.v1.local_variables_initializer'
144:23: INFO: Renamed 'tf.global_variables' to 'tf.compat.v1.global_variables'
155:25: INFO: Renamed 'tf.ConfigProto' to 'tf.compat.v1.ConfigProto'
156:24: INFO: Renamed 'tf.GPUOptions' to 'tf.compat.v1.GPUOptions'
169:25: INFO: Renamed 'tf.ConfigProto' to 'tf.compat.v1.ConfigProto'
173:11: INFO: Renamed 'tf.Session' to 'tf.compat.v1.Session'
177:23: INFO: tf.summary.FileWriter requires manual check. The TF 1.x summary API cannot be automatically migrated to TF 2.0, so symbols have been converted to tf.compat.v1.summary.* and must be migrated manually. Typical usage will only require changes to the summary writing logic, not to individual calls like scalar(). For examples of the new summary API, see the Effective TF 2.0 migration document or check the TF 2.0 TensorBoard tutorials.
177:23: INFO: Renamed 'tf.summary.FileWriter' to 'tf.compat.v1.summary.FileWriter'
179:38: INFO: tf.summary.FileWriter requires manual check. The TF 1.x summary API cannot be automatically migrated to TF 2.0, so symbols have been converted to tf.compat.v1.summary.* and must be migrated manually. Typical usage will only require changes to the summary writing logic, not to individual calls like scalar(). For examples of the new summary API, see the Effective TF 2.0 migration document or check the TF 2.0 TensorBoard tutorials.
179:38: INFO: Renamed 'tf.summary.FileWriter' to 'tf.compat.v1.summary.FileWriter'
180:21: INFO: tf.summary.FileWriter requires manual check. The TF 1.x summary API cannot be automatically migrated to TF 2.0, so symbols have been converted to tf.compat.v1.summary.* and must be migrated manually. Typical usage will only require changes to the summary writing logic, not to individual calls like scalar(). For examples of the new summary API, see the Effective TF 2.0 migration document or check the TF 2.0 TensorBoard tutorials.
180:21: INFO: Renamed 'tf.summary.FileWriter' to 'tf.compat.v1.summary.FileWriter'
185:21: INFO: Renamed 'tf.assign' to 'tf.compat.v1.assign'
304:32: INFO: Renamed 'tf.graph_util.convert_variables_to_constants' to 'tf.compat.v1.graph_util.convert_variables_to_constants'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/output_event.py'
 outputting to 'cmd/output_event.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/profile_model.py'
 outputting to 'cmd/profile_model.py'
--------------------------------------------------------------------------------

57:18: INFO: Renamed 'tf.global_variables_initializer' to 'tf.compat.v1.global_variables_initializer'
60:21: INFO: Renamed 'tf.ConfigProto' to 'tf.compat.v1.ConfigProto'
61:11: INFO: Renamed 'tf.Session' to 'tf.compat.v1.Session'
148:17: INFO: Renamed 'tf.profiler.profile' to 'tf.compat.v1.profiler.profile'
148:52: INFO: Renamed 'tf.profiler.ProfileOptionBuilder' to 'tf.compat.v1.profiler.ProfileOptionBuilder'
175:11: INFO: Renamed 'tf.profiler.profile' to 'tf.compat.v1.profiler.profile'
175:46: INFO: Renamed 'tf.profiler.ProfileOptionBuilder' to 'tf.compat.v1.profiler.ProfileOptionBuilder'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/predict.py'
 outputting to 'cmd/predict.py'
--------------------------------------------------------------------------------

80:18: INFO: Renamed 'tf.global_variables_initializer' to 'tf.compat.v1.global_variables_initializer'
84:21: INFO: Renamed 'tf.ConfigProto' to 'tf.compat.v1.ConfigProto'
85:11: INFO: Renamed 'tf.Session' to 'tf.compat.v1.Session'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/__init__.py'
 outputting to 'cmd/__init__.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/convert.py'
 outputting to 'cmd/convert.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/main.py'
 outputting to 'cmd/main.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/build_tfds.py'
 outputting to 'cmd/build_tfds.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/measure_latency.py'
 outputting to 'cmd/measure_latency.py'
--------------------------------------------------------------------------------

58:18: INFO: Renamed 'tf.global_variables_initializer' to 'tf.compat.v1.global_variables_initializer'
64:11: INFO: Renamed 'tf.Session' to 'tf.compat.v1.Session'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/tune_ray.py'
 outputting to 'cmd/tune_ray.py'
--------------------------------------------------------------------------------

119:47: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
126:49: INFO: tf.train.polynomial_decay requires manual check. To use learning rate decay schedules with TensorFlow 2.0, switch to the schedules in `tf.keras.optimizers.schedules`.

126:49: INFO: Renamed 'tf.train.polynomial_decay' to 'tf.compat.v1.train.polynomial_decay'
213:18: INFO: Renamed 'tf.global_variables_initializer' to 'tf.compat.v1.global_variables_initializer'
214:32: INFO: Renamed 'tf.local_variables_initializer' to 'tf.compat.v1.local_variables_initializer'
216:25: INFO: Renamed 'tf.ConfigProto' to 'tf.compat.v1.ConfigProto'
217:24: INFO: Renamed 'tf.GPUOptions' to 'tf.compat.v1.GPUOptions'
218:20: INFO: Renamed 'tf.Session' to 'tf.compat.v1.Session'
258:15: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/evaluate.py'
 outputting to 'cmd/evaluate.py'
--------------------------------------------------------------------------------

109:18: INFO: Renamed 'tf.global_variables_initializer' to 'tf.compat.v1.global_variables_initializer'
110:27: INFO: Renamed 'tf.local_variables_initializer' to 'tf.compat.v1.local_variables_initializer'
114:11: INFO: Renamed 'tf.Session' to 'tf.compat.v1.Session'
117:24: INFO: tf.summary.FileWriter requires manual check. The TF 1.x summary API cannot be automatically migrated to TF 2.0, so symbols have been converted to tf.compat.v1.summary.* and must be migrated manually. Typical usage will only require changes to the summary writing logic, not to individual calls like scalar(). For examples of the new summary API, see the Effective TF 2.0 migration document or check the TF 2.0 TensorBoard tutorials.
117:24: INFO: Renamed 'tf.summary.FileWriter' to 'tf.compat.v1.summary.FileWriter'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'cmd/convert_weight_from_darknet.py'
 outputting to 'cmd/convert_weight_from_darknet.py'
--------------------------------------------------------------------------------

56:18: INFO: Renamed 'tf.global_variables_initializer' to 'tf.compat.v1.global_variables_initializer'
60:20: INFO: Renamed 'tf.global_variables' to 'tf.compat.v1.global_variables'
63:11: INFO: Renamed 'tf.Session' to 'tf.compat.v1.Session'
135:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
--------------------------------------------------------------------------------

iizukak · 2020-02-21T12:56:22Z

@ytfksw Thanks. nice PR!!!!

ytfksw · 2020-02-26T10:39:45Z

There are seven same warnings:
"WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function."

As the warning message says, if Keras model.save is used, the code should be checked manualy. But it is irrelevant since blueoil does not use Keras model.

bo-code-review-bot · 2020-02-26T10:39:56Z

This PR needs Approvals as follows.

Ownership Approval for / from iizukak, tkng, ruimashita
Readability Approval for Python from tkng, tsawada

Please choose reviewers and requet reviews!

Click to see how to approve each reviews

You can approve this PR by triggered comments as follows.

Approve all reviews requested to you (readability and ownership) and LGTM review
Approval, LGTM
Approve all ownership reviews
Ownership Approval or OA
Approve all readability reviews
Readability Approval or RA
Approve specified review targets
- Example of Ownership Reviewer of /: Ownership Approval for / or OA for /
- Example of Readability Reviewer of Python: Readability Approval for Python or RA for Python
Approve LGTM review
LGTM

See all trigger comments

Please replace [Target] to review target

Ownership Approval
- Ownership Approval for [Target]
- OA for [Target]
- Ownership Approval
- OA
- Approval
Readability Approval
- Readability Approval for [Target]
- RA for [Target]
- [Target] Readability Approval
- [Target] RA
- Readability Approval
- RA
- Approval
LGTM
- LGTM
- lgtm

ytfksw · 2020-03-02T02:57:27Z

@iizukak Could you please assign a reviewer?

iizukak · 2020-03-02T03:02:36Z

@ytfksw
OK.
First, Can you list up future PR like #741 ?
I want to know the amount of work. Someone may want to try other part of this work.

ytfksw · 2020-03-02T06:55:04Z

@iizukak
For future PR, I identified the targets to change and estimated the difficulty.
#479 (comment)

iizukak · 2020-03-02T06:59:47Z

@ytfksw Great. Thanks.

iizukak

Ownership Approval

tsawada

Readability Approval

iizukak · 2020-03-02T23:56:06Z

@ytfksw Thanks. After this PR, You can continue this work?

iizukak · 2020-03-02T23:56:12Z

/ready

bo-mergebot · 2020-03-02T23:56:14Z

⏳Merge job is queued...

bo-mergebot · 2020-03-03T00:20:33Z

😥Status check failed. Please fix problems and send /ready again.

ytfksw · 2020-03-03T07:12:15Z

@iizukak No, conversion tf2 for cmd is done. But we need LGTM from someone to merge...
Could you assign someone or give LGTM.

tf_upgrade_v2 for cmd

0991d2d

blueoil-butler bot added the CI: auto-run Run CI automatically label Feb 21, 2020

Avoid 120 character limit

bf4f8bf

ytfksw marked this pull request as ready for review February 26, 2020 10:39

iizukak approved these changes Mar 2, 2020

View reviewed changes

iizukak requested a review from tsawada March 2, 2020 07:01

tsawada approved these changes Mar 2, 2020

View reviewed changes

Merge branch 'master' into convert_cmd

b12b54d

iizukak added the CI: test-all Run all tests once label Mar 3, 2020

blueoil-butler bot removed the CI: test-all Run all tests once label Mar 3, 2020

iizukak merged commit 0907fb7 into blue-oil:master Mar 3, 2020

ytfksw mentioned this pull request May 29, 2020

Consider upgrading to TensorFlow 2.0 #479

Open

ytfksw self-assigned this Jun 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tf_upgrade_v2 for cmd #875

tf_upgrade_v2 for cmd #875

ytfksw commented Feb 21, 2020 •

edited

Loading

ytfksw commented Feb 21, 2020

iizukak commented Feb 21, 2020

ytfksw commented Feb 26, 2020 •

edited

Loading

bo-code-review-bot bot commented Feb 26, 2020

ytfksw commented Mar 2, 2020

iizukak commented Mar 2, 2020 •

edited

Loading

ytfksw commented Mar 2, 2020

iizukak commented Mar 2, 2020

iizukak left a comment

tsawada left a comment

iizukak commented Mar 2, 2020

iizukak commented Mar 2, 2020

bo-mergebot bot commented Mar 2, 2020

bo-mergebot bot commented Mar 3, 2020

ytfksw commented Mar 3, 2020

tf_upgrade_v2 for cmd #875

tf_upgrade_v2 for cmd #875

Conversation

ytfksw commented Feb 21, 2020 • edited Loading

What this patch does to fix the issue.

Link to any relevant issues or pull requests.

ytfksw commented Feb 21, 2020

iizukak commented Feb 21, 2020

ytfksw commented Feb 26, 2020 • edited Loading

bo-code-review-bot bot commented Feb 26, 2020

ytfksw commented Mar 2, 2020

iizukak commented Mar 2, 2020 • edited Loading

ytfksw commented Mar 2, 2020

iizukak commented Mar 2, 2020

iizukak left a comment

Choose a reason for hiding this comment

tsawada left a comment

Choose a reason for hiding this comment

iizukak commented Mar 2, 2020

iizukak commented Mar 2, 2020

bo-mergebot bot commented Mar 2, 2020

bo-mergebot bot commented Mar 3, 2020

ytfksw commented Mar 3, 2020

ytfksw commented Feb 21, 2020 •

edited

Loading

ytfksw commented Feb 26, 2020 •

edited

Loading

iizukak commented Mar 2, 2020 •

edited

Loading