tf_upgrade_v2 for configs #899

ytfksw · 2020-03-06T03:31:20Z

What this patch does to fix the issue.

I converted only the blueoil/configs with the command line tool tf_upgrade_v2 to run on tensorflow2.

Since the template of the config in blueoil/templates cannot be changed by the script,
first I convert the sample configuration file in the configs folder.

Here's what I did:

install newest tenosrflow (tensorflow-gpu==1.5.2)
Run the upgrade script tf_upgrade_v2
Check the upgrade report for warnings and errors
Run test (make test)

Link to any relevant issues or pull requests.

ytfksw · 2020-03-06T03:32:50Z

This is the whole convert log:

TensorFlow 2.0 Upgrade Script
-----------------------------
Converted 30 files
Detected 0 issues that require attention
--------------------------------------------------------------------------------
================================================================================
Detailed log follows:

================================================================================
================================================================================
Input tree: 'configs'
================================================================================
--------------------------------------------------------------------------------
Processing file 'configs/core/classification/lm_resnet_quantize_cifar10.py'
 outputting to 'configs/core/classification/lm_resnet_quantize_cifar10.py'
--------------------------------------------------------------------------------

79:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
81:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/lmnet_v1_quantize_cifar10_tune.py'
 outputting to 'configs/core/classification/lmnet_v1_quantize_cifar10_tune.py'
--------------------------------------------------------------------------------

89:29: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
98:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/lmnet_quantize_cifar100.py'
 outputting to 'configs/core/classification/lmnet_quantize_cifar100.py'
--------------------------------------------------------------------------------

75:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
77:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/lmnet_v1_quantize_cifar10.py'
 outputting to 'configs/core/classification/lmnet_v1_quantize_cifar10.py'
--------------------------------------------------------------------------------

75:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
77:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/lmnet_cifar10.py'
 outputting to 'configs/core/classification/lmnet_cifar10.py'
--------------------------------------------------------------------------------

72:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
74:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/lmnet_cifar100.py'
 outputting to 'configs/core/classification/lmnet_cifar100.py'
--------------------------------------------------------------------------------

72:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
74:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/resnet_cifar10.py'
 outputting to 'configs/core/classification/resnet_cifar10.py'
--------------------------------------------------------------------------------

73:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
75:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/lmnet_quantize_cifar10.py'
 outputting to 'configs/core/classification/lmnet_quantize_cifar10.py'
--------------------------------------------------------------------------------

76:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
78:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/darknet_quantize_ilsvrc_2012.py'
 outputting to 'configs/core/classification/darknet_quantize_ilsvrc_2012.py'
--------------------------------------------------------------------------------

79:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
81:29: INFO: tf.train.polynomial_decay requires manual check. To use learning rate decay schedules with TensorFlow 2.0, switch to the schedules in `tf.keras.optimizers.schedules`.

81:29: INFO: Renamed 'tf.train.polynomial_decay' to 'tf.compat.v1.train.polynomial_decay'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/lmnet_openimagesv4.py'
 outputting to 'configs/core/classification/lmnet_openimagesv4.py'
--------------------------------------------------------------------------------

71:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
73:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/lmnet_v1_cifar10.py'
 outputting to 'configs/core/classification/lmnet_v1_cifar10.py'
--------------------------------------------------------------------------------

71:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
73:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/darknet_cifar10.py'
 outputting to 'configs/core/classification/darknet_cifar10.py'
--------------------------------------------------------------------------------

72:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
74:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/mobilenet_v2_cifar10.py'
 outputting to 'configs/core/classification/mobilenet_v2_cifar10.py'
--------------------------------------------------------------------------------

72:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
74:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/classification/darknet_quantize_cifar10.py'
 outputting to 'configs/core/classification/darknet_quantize_cifar10.py'
--------------------------------------------------------------------------------

75:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
77:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/segmentation/segnet_quantize_cityscapes.py'
 outputting to 'configs/core/segmentation/segnet_quantize_cityscapes.py'
--------------------------------------------------------------------------------

75:26: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/segmentation/segnet_quantize_camvid.py'
 outputting to 'configs/core/segmentation/segnet_quantize_camvid.py'
--------------------------------------------------------------------------------

73:26: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/segmentation/lm_segnet_v0_quantize_camvid.py'
 outputting to 'configs/core/segmentation/lm_segnet_v0_quantize_camvid.py'
--------------------------------------------------------------------------------

71:26: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/segmentation/segnet_quantize_camvid_tune.py'
 outputting to 'configs/core/segmentation/segnet_quantize_camvid_tune.py'
--------------------------------------------------------------------------------

88:29: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
96:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/segmentation/lm_bisenet_quantize_camvid.py'
 outputting to 'configs/core/segmentation/lm_bisenet_quantize_camvid.py'
--------------------------------------------------------------------------------

83:26: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/segmentation/lm_segnet_v1_quantize_camvid.py'
 outputting to 'configs/core/segmentation/lm_segnet_v1_quantize_camvid.py'
--------------------------------------------------------------------------------

75:26: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/segmentation/segnet_camvid.py'
 outputting to 'configs/core/segmentation/segnet_camvid.py'
--------------------------------------------------------------------------------

69:26: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/segmentation/segnet_quantize_bdd100k.py'
 outputting to 'configs/core/segmentation/segnet_quantize_bdd100k.py'
--------------------------------------------------------------------------------

69:26: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/object_detection/lm_fyolo_bdd100k.py'
 outputting to 'configs/core/object_detection/lm_fyolo_bdd100k.py'
--------------------------------------------------------------------------------

83:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
85:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/object_detection/yolo_v2_quantize_pascalvoc_2007_2012.py'
 outputting to 'configs/core/object_detection/yolo_v2_quantize_pascalvoc_2007_2012.py'
--------------------------------------------------------------------------------

190:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
192:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/object_detection/lm_fyolo_quantize_pascalvoc_2007_2012.py'
 outputting to 'configs/core/object_detection/lm_fyolo_quantize_pascalvoc_2007_2012.py'
--------------------------------------------------------------------------------

102:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
104:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/core/keypoint_detection/lm_single_pose_v1_quantize_mscoco.py'
 outputting to 'configs/core/keypoint_detection/lm_single_pose_v1_quantize_mscoco.py'
--------------------------------------------------------------------------------

88:26: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
90:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/example/object_detection.py'
 outputting to 'configs/example/object_detection.py'
--------------------------------------------------------------------------------

91:26: INFO: Renamed 'tf.train.MomentumOptimizer' to 'tf.compat.v1.train.MomentumOptimizer'
93:29: INFO: Renamed 'tf.train.piecewise_constant' to 'tf.compat.v1.train.piecewise_constant'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/example/classification.py'
 outputting to 'configs/example/classification.py'
--------------------------------------------------------------------------------

83:26: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/convert_weight_from_darknet/yolo_v2.py'
 outputting to 'configs/convert_weight_from_darknet/yolo_v2.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'configs/convert_weight_from_darknet/darknet19.py'
 outputting to 'configs/convert_weight_from_darknet/darknet19.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

ytfksw · 2020-03-06T03:44:24Z

From this PR, we know that the value of NETWORK.OPTIMIZER_CLASS、NETWORK.LEARNING_RATE_FUNC should be changed.

Also I have realized also that the blueoil/template files itself has no tensorflow function.
We have to change generate_lmnet_config.py instead of the blueoil/template.

https://github.com/blue-oil/blueoil/blob/master/blueoil/generate_lmnet_config.py#L159-L162

https://github.com/blue-oil/blueoil/blob/master/blueoil/generate_lmnet_config.py#L176-178

bo-code-review-bot · 2020-03-06T10:51:54Z

This PR needs Approvals as follows.

Ownership Approval for / from iizukak, tkng, ruimashita
Readability Approval for Python from tkng, tsawada

Please choose reviewers and requet reviews!

Click to see how to approve each reviews

You can approve this PR by triggered comments as follows.

Approve all reviews requested to you (readability and ownership) and LGTM review
Approval, LGTM
Approve all ownership reviews
Ownership Approval or OA
Approve all readability reviews
Readability Approval or RA
Approve specified review targets
- Example of Ownership Reviewer of /: Ownership Approval for / or OA for /
- Example of Readability Reviewer of Python: Readability Approval for Python or RA for Python
Approve LGTM review
LGTM

See all trigger comments

Please replace [Target] to review target

Ownership Approval
- Ownership Approval for [Target]
- OA for [Target]
- Ownership Approval
- OA
- Approval
Readability Approval
- Readability Approval for [Target]
- RA for [Target]
- [Target] Readability Approval
- [Target] RA
- Readability Approval
- RA
- Approval
LGTM
- LGTM
- lgtm

iizukak

Ownership Approval

tsawada

Readability Approval

iizukak · 2020-03-10T23:42:56Z

/ready

bo-mergebot · 2020-03-10T23:42:58Z

⏳Merge job is queued...

blueoil-butler bot added the CI: auto-run Run CI automatically label Mar 6, 2020

tf_upgrade_v2 for configs

90e7300

ytfksw force-pushed the convert_tf2_configs branch from 5e0351a to 90e7300 Compare March 6, 2020 05:10

ytfksw mentioned this pull request Mar 6, 2020

convert tf2 for blueoil/generate_lmnet_config.py #902

Merged

ytfksw marked this pull request as ready for review March 6, 2020 10:51

iizukak approved these changes Mar 9, 2020

View reviewed changes

iizukak requested a review from tsawada March 10, 2020 13:20

tsawada approved these changes Mar 10, 2020

View reviewed changes

Merge branch 'master' into convert_tf2_configs

a9eb490

bo-mergebot bot merged commit a0078e1 into blue-oil:master Mar 11, 2020

ytfksw mentioned this pull request May 29, 2020

Consider upgrading to TensorFlow 2.0 #479

Open

ytfksw self-assigned this Jun 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tf_upgrade_v2 for configs #899

tf_upgrade_v2 for configs #899

ytfksw commented Mar 6, 2020 •

edited

Loading

ytfksw commented Mar 6, 2020

ytfksw commented Mar 6, 2020

bo-code-review-bot bot commented Mar 6, 2020

iizukak left a comment

tsawada left a comment

iizukak commented Mar 10, 2020

bo-mergebot bot commented Mar 10, 2020

tf_upgrade_v2 for configs #899

tf_upgrade_v2 for configs #899

Conversation

ytfksw commented Mar 6, 2020 • edited Loading

What this patch does to fix the issue.

Link to any relevant issues or pull requests.

ytfksw commented Mar 6, 2020

ytfksw commented Mar 6, 2020

bo-code-review-bot bot commented Mar 6, 2020

iizukak left a comment

Choose a reason for hiding this comment

tsawada left a comment

Choose a reason for hiding this comment

iizukak commented Mar 10, 2020

bo-mergebot bot commented Mar 10, 2020

ytfksw commented Mar 6, 2020 •

edited

Loading