msg: remove gencpp and genmsg submodules; remove gencpp imports #13572

TSC21 · 2019-11-22T16:27:42Z

Describe problem solved by this pull request
After some verification, I noticed:

We are not using gencpp for anything (!) - there was probably a reason it was added some years ago, but now it's just there, occupying space as an unused dependency and submodule;
If we were using genmsg and gencpp from source because of lack of compatibility with other OS's, after some search I found that we do have genmsg available by pip: https://pypi.org/project/pyros-genmsg/. It just needs to be tested in other platforms, like Windows and MacOS.

Describe your solution
Completely remove the genmsg and gencpp submodules and all the places where gencpp was being imported (since it wasn't really being used).

dagar · 2019-11-22T16:29:21Z

Sounds good to me. Goodbye submodules!

TSC21 · 2019-11-22T16:30:26Z

For this to build correctly, we are going to need to update the base bionic and xenial containers to install python-genmsg by apt. python3-genmsg is being installed on the ROS2 containers, since is being used by px4_ros_com to generate the micro-RTPS agent code, but I guess to make this compatible with Python 3 as well, and following the developments in #12809, the install of python3-genmsg should also be moved to the base containers.

julianoes · 2019-11-22T16:45:08Z

Tested on macOS, built fine after pip3 install --user empy pyros-genmsg.

TSC21 · 2019-11-22T16:49:03Z

Tested on macOS, built fine after pip3 install --user empy pyros-genmsg.

Great stuff! Let's make sure we update the containers accordingly! @MaEtUgR can you do something about AppVeyor?

TSC21 · 2019-11-22T16:51:08Z

@dagar are you able to update the MacOS side on CI? For now, pip2 install --user empy pyros-genmsg should be enough (as we are still moving to Python3). But you can always install it for both and it gets already solved.

TSC21 · 2019-11-22T17:02:50Z

Containers to be updated in PX4/PX4-containers#213.

TSC21 · 2019-11-23T18:15:04Z

I keep bumping into the same error as in #12809:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3356: ordinal not in range(128)

@julianoes do you know what might be causing this? I can't reproduce it locally (and I have the same Python version).

Quickly looking for the cause, this might be an encoding issue.

TSC21 · 2019-11-23T18:33:50Z

We might have to setup these on the base containers. As soon as I get a chance I will test it locally on the containers.

TSC21 · 2019-11-24T00:14:57Z

@MaEtUgR is it possible for you to update the Windows CI here for the python genmsg dependency?

bkueng · 2019-11-25T10:12:30Z

Can you improve the error outputs? I'm getting

python import error:  No module named 'genmsg'

Required python packages not installed.

On a Debian/Ubuntu system please run:

  sudo apt-get install python3-empy
  sudo pip3 install catkin_pkg

On MacOS please run:
  sudo pip3 install empy catkin_pkg

On Windows please run:
  easy_install empy catkin_pkg

Whereas pyros-genmsg is needed.

TSC21 · 2019-11-25T10:15:32Z

Can you improve the error outputs? I'm getting

python import error:  No module named 'genmsg'

Required python packages not installed.

On a Debian/Ubuntu system please run:

  sudo apt-get install python3-empy
  sudo pip3 install catkin_pkg

On MacOS please run:
  sudo pip3 install empy catkin_pkg

On Windows please run:
  easy_install empy catkin_pkg

Whereas pyros-genmsg is needed.

Yes absolutely.

TSC21 · 2019-11-25T16:26:58Z

PX4/PX4-containers#214 brings the fix for the decoding issues. Tested locally and on Jenkins. Just needs a container update now.

This should ignore compiler warnings from the includes.

TSC21 · 2020-01-13T21:09:00Z

@julianoes success!! Finally :D

LorenzMeier · 2020-01-13T21:16:41Z

msg/tools/px_generate_uorb_topic_files.py


 On MacOS please run:
-  sudo pip install empy catkin_pkg


Have the docs been updated?

Good point. We'll check them.

TSC21 · 2020-01-13T21:48:28Z

Let's bring this in and update the docs where needed.

TSC21 added Admin: Cleanup (housekeeping) 🧹 Communication (MAVLink) 📡 labels Nov 22, 2019

TSC21 requested review from dagar and julianoes November 22, 2019 16:27

TSC21 self-assigned this Nov 22, 2019

TSC21 changed the title ~~msg: remove gencpp and genmsh submodules; remove gencpp imports~~ msg: remove gencpp and genmsg submodules; remove gencpp imports Nov 22, 2019

TSC21 mentioned this pull request Nov 22, 2019

install pyros-genmsg with pip3 (equivalent to python3-genmsg) on base containers PX4/PX4-containers#213

Merged

TSC21 requested a review from bkueng November 23, 2019 09:45

TSC21 force-pushed the cleanup/remove_genmsg_gencpp_submodules branch 2 times, most recently from 9a367f3 to b281c9b Compare November 23, 2019 18:11

TSC21 force-pushed the cleanup/remove_genmsg_gencpp_submodules branch from b51a0fb to 29db151 Compare November 23, 2019 18:27

TSC21 force-pushed the cleanup/remove_genmsg_gencpp_submodules branch from eea8756 to 0786bc5 Compare November 25, 2019 14:50

TSC21 mentioned this pull request Nov 25, 2019

Add UTF-8 language encoding PX4/PX4-containers#214

Merged

TSC21 force-pushed the cleanup/remove_genmsg_gencpp_submodules branch from 0786bc5 to ccfc4b7 Compare November 29, 2019 09:56

TSC21 mentioned this pull request Nov 29, 2019

Multiple unaligned type cast #12319

Closed

TSC21 force-pushed the cleanup/remove_genmsg_gencpp_submodules branch 3 times, most recently from aefe830 to d4b3425 Compare November 29, 2019 10:45

julianoes and others added 10 commits January 13, 2020 16:35

Tools: use Python 3 in validate_yaml.py

22ce22a

Tools: use Python3 for upload_log.py

5c79a94

if building with catkin, don't specifically look for Python 3

a9235a9

CI: update all docker tags

245961b

Update Snapdragon docker tag

ac9fde1

beaglebone: treat robotcontrol as system include

db928b5

This should ignore compiler warnings from the includes.

rostests: add PYTHONPATH export path so to find Python 2 packages

aadc744

Jenkins: update snapdragon container tag

74a8d95

run mission_test over python2

8cde9f3

don't specifically export PYTHONPATH

8608c48

TSC21 force-pushed the cleanup/remove_genmsg_gencpp_submodules branch 3 times, most recently from 9e23424 to 662edeb Compare January 13, 2020 16:51

Jenkins: export PYTHONPATH before Catkin build

e1b239b

TSC21 force-pushed the cleanup/remove_genmsg_gencpp_submodules branch from 662edeb to e1b239b Compare January 13, 2020 16:56

update submodule sitl_gazebo

bcf557d

TSC21 force-pushed the cleanup/remove_genmsg_gencpp_submodules branch from ebc206e to ed253b9 Compare January 13, 2020 18:51

px4modulesdoc: move to Python3

413d3b9

TSC21 force-pushed the cleanup/remove_genmsg_gencpp_submodules branch from ed253b9 to 413d3b9 Compare January 13, 2020 19:08

TSC21 added 2 commits January 13, 2020 19:41

mission_test: fix px4tools func import

877b158

Jenkins: reactivate ROS2-related deployment

59b0c75

TSC21 requested a review from dagar January 13, 2020 21:09

Merge branch 'master' into cleanup/remove_genmsg_gencpp_submodules

2478e47

TSC21 requested a review from LorenzMeier January 13, 2020 21:12

LorenzMeier approved these changes Jan 13, 2020

View reviewed changes

TSC21 merged commit 4f362f5 into master Jan 13, 2020

TSC21 deleted the cleanup/remove_genmsg_gencpp_submodules branch January 13, 2020 21:48

julianoes mentioned this pull request Jan 15, 2020

Use Python 3 everywhere #12809

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

msg: remove gencpp and genmsg submodules; remove gencpp imports #13572

msg: remove gencpp and genmsg submodules; remove gencpp imports #13572

TSC21 commented Nov 22, 2019

dagar commented Nov 22, 2019

TSC21 commented Nov 22, 2019

julianoes commented Nov 22, 2019

TSC21 commented Nov 22, 2019

TSC21 commented Nov 22, 2019 •

edited

Loading

TSC21 commented Nov 22, 2019

TSC21 commented Nov 23, 2019 •

edited

Loading

TSC21 commented Nov 23, 2019

TSC21 commented Nov 24, 2019

bkueng commented Nov 25, 2019

TSC21 commented Nov 25, 2019

TSC21 commented Nov 25, 2019

TSC21 commented Jan 13, 2020

LorenzMeier Jan 13, 2020

julianoes Jan 13, 2020

TSC21 commented Jan 13, 2020

msg: remove gencpp and genmsg submodules; remove gencpp imports #13572

msg: remove gencpp and genmsg submodules; remove gencpp imports #13572

Conversation

TSC21 commented Nov 22, 2019

dagar commented Nov 22, 2019

TSC21 commented Nov 22, 2019

julianoes commented Nov 22, 2019

TSC21 commented Nov 22, 2019

TSC21 commented Nov 22, 2019 • edited Loading

TSC21 commented Nov 22, 2019

TSC21 commented Nov 23, 2019 • edited Loading

TSC21 commented Nov 23, 2019

TSC21 commented Nov 24, 2019

bkueng commented Nov 25, 2019

TSC21 commented Nov 25, 2019

TSC21 commented Nov 25, 2019

TSC21 commented Jan 13, 2020

LorenzMeier Jan 13, 2020

Choose a reason for hiding this comment

julianoes Jan 13, 2020

Choose a reason for hiding this comment

TSC21 commented Jan 13, 2020

TSC21 commented Nov 22, 2019 •

edited

Loading

TSC21 commented Nov 23, 2019 •

edited

Loading