Usefulness of cloudsim logs

[peci1]

It seems to me that most of the logs we get from cloudsim are practically useless. Why should I be interested in an atop log from the bridge container? I'd be much more interested into an atop log from the solution container, but these aren't recorded (and unless we publish the atop log as a ROS message, it can't be saved anywhere). The same holds for all the ROS log files from ~/.ros/log except rosout.log and master.log.

Is there any way to get better data from cloudsim, e.g. ~/.ros/log from the solution container, an atop log etc.?

Also, the robot_data bagfiles could be compressed (ideally with bz2) to fit more data in them. It won't help with images, but it will help a lot with pointclouds and odometry messages.

[zbynekwinkler]

Why should I be interested in an atop log from the bridge container?

Because the bridge container runs on the same aws instance as your solution. Before you joint us on the cloudsim effort, there were some problems with messages getting mysteriously lost. If your solution puts too much CPU pressure on the machine, the bridge might be starved of CPU and could start dropping messages to the floor (the default ROS behavior in this case). The logs on bridge had been added to be able to debug such a problem, should it happen again (as it seems to be the case in #655).

Is there any way to get better data from cloudsim, e.g. ~/.ros/log from the solution container, an atop log etc.?

Frankly, I don't know why this directory is not volume mounted on the aws instance so these files would be left over there to pick up when the containers exit (that is what we do when we run the dockers locally). Instead we have the bagfiles... we are happy we have at least those.

Also, the robot_data bagfiles could be compressed (ideally with bz2) to fit more data in them.

We compress the data before sending it to the bag to deal with the size issue.

[peci1]

But how would I notice the CPU is under stress if atop only shows nodes run by the bridge container?

[peci1]

Otherwise, I partly understand why .ros/log from solution containers isn't transferred - cloudsim has no control over what do people save in this folder, so they couldn't guarantee it fits into some size limit.

But as there's no size limit on rosout.log, you could theoretically use it for storing data (not saying we're gonna hack it like this, but it definitely is an option).

I'd consider more fair to specify a data limit for solution containers' .ros/log and anybody could save anything there as long as it fits (finding a policy choosing what to delete from larger archives would of course be difficult).

[zbynekwinkler]

But how would I notice the CPU is under stress if atop only shows nodes run by the bridge container?

It will just tell you that the bridge is not going crazy and not stressing the CPUs you need to be using in your solution.

But as there's no size limit on rosout.log, you could theoretically use it for storing data (not saying we're gonna hack it like this, but it definitely is an option).

That is something what we had in place before the advent of the bag files. We were sending base64 encoded binary data to rosout. Let me just say that rosout is not built for that 😉

@peci1 I also see the ways the current system could be improved but I am happy with the rosbag we have since I was here when we were using base64 rosout 😉. As for priorities - I am happier when the simulation does not crash #631 and when breadcrumbs are being deployed #655.