rep-0100.rst

REP: 100 Title: ROS Stack Separation Author: Tully Foote <[email protected]>, Ken Conley <[email protected]>, Brian Gerkey <[email protected]> Status: Final Type: Standards Track Content-Type: text/x-rst Created: 18-Sep-2010 Post-History: 18-Sep-2010, 20-Oct-2010, 20-Nov-2010

Abstract

The monolithic ROS stack, as released in ROS 1.0 and ROS 1.2, is being separated into several, smaller stacks: ros, ros_comm, rx, and documentation. These smaller stacks will enable installation on more platforms, better decoupling of design, and easier release. It is also hoped that the base ros stack, which will contain the packaging and build system, will be useful in other software projects that don't make use of the ROS communication system.

Specification

New packages are marked with an asterisk *.

ros stack

• mk
• rosbash
• rosbuild
• rosboost_cfg
• rosclean
• roscreate
• rosdep
• rosemacs
• roslib
• rospack
• rosmake
• rosunit*

ros_comm stack

ROS client libraries

• roslang
• rospy
• roslisp
• roscpp
• roscpp_serialization*
• roscpp_traits*
• rostime*

ROS graph tools

• rosbag
• rosbagmigration
• rosconsole
• rosgraph
• roslaunch
• rosmsg
• rosmaster
• rosnode
• rosout
• rosparam
• rosrecord
• rosservice
• rostest
• rostopic
• topic_tools

ROS core ontology

• roscore_migration_rules
• rosgraph_msgs*
• std_msgs
• std_srvs

Libraries and Utilities

• message_filters
• roswtf
• xmlrpcpp

ros_release stack

ros_release is an existing ROS stack used to for libraries and scripts related to releasing and installing ROS stacks. Two Python modules from roslib will be moved into it as new packages.

• rosdistro*
• vcstools*

rx stack

The rx stack contains GUI-related tools for ROS, which are generally wxWindows-based. The 'rx' prefix is commonly used to denote ROS tools that provide a graphical user interface.

• rxbag
• rxdeps
• rxgraph
• rxtools
• wxPython_swig_interface
• wxswig
• xdot

documentation stack

• rosdoc

The documentation stack will only contain the rosdoc codebase in the initial release. Work is currently being done on rosdoc to separate it into a plugin-based model so that specific documentation capabilities can be added or removed as needed. For example, documentation needs for ROS.org are being moved into the rosdoc_rosorg package and will migrate to another stack. This work is independent of this REP.

Deleted Packages

The following packages will be removed as part of this separation. The rosrecord package was superceded by rosbag in ROS 1.2. The genmsg_cpp stack is being removed in favor of Python-based message generators. This is discussed further in the backward compatibility section.

• genmsg_cpp
• gtest

Header, Clock, and Log

Header will be moved from roslib into the std_msgs package.

Clock, and Log will be moved from the roslib package to the rosgraph_msgs package.

For backwards compatibility, the generated source code files for these messages will be released with the roslib package in ROS 1.4 but marked as deprecated.

Moving roslib C++ libraries

roscpp makes use of several header and cpp files in roslib. These will be moved into new packages within the ros_comm stack.

• roscpp_traits
• roscpp_serialization
• rostime

These packages use the generic 'ros' namespace, so the expected impact on existing packages is expected to be minor.

Moving roslib python modules

Several roslib python modules will be moved to rosgraph and elsewhere.

The following modules will be moved to rosgraph:

• roslib.genpy
• roslib.gentools
• roslib.message
• roslib.msgs
• roslib.network
• roslib.rostime
• roslib.srvs
• roslib.xmlrpc

The following modules will be moved to rosdistro (new package):

• roslib.distro

The following modules will be moved to vcstools (new package):

• roslib.vcs

The following modules will be heavily refactored:

• roslib.scriptutil

The following modules will be removed (already deprecated):

• roslib.masterapi

rosunit

The main role of rosunit is to replace the bare test functionality of rostest. The bare test function wraps unit tests when they are executed and adds a configurable timeout. It also produces an Ant JUnit XML file from the results. This functionality is generally useful in the ROS build system and is also necessary to run unit tests on the ros stack.

The rosunit package will get the following resources from rostest:

• rostest/bin/cleanunit
• rostest/bin/test-results-dirs
• rostest/bin/coverage-html
• rostest/bin/rostest-check-results

These scripts are used internally by build tools and do not have high visibility. They will be given newer, more descriptive names after the move.

The following script will be refactored, with some of its required routines moved to rosunit:

• rostest/src/rostest/rostestutil.py

rosbuild

rosbuild will be converted to use scripts moved to the rosunit package.

For scripts that currently cannot be moved to the rosunit package (e.g. rostest itself), the behaviors are defined below.

rosbuild_add_roslaunch_check(): this macro requires use of the roslaunch-check.py. This script will be moved to the roslaunch package, which remains with the ros_comm stack. To minimize impact on existing packages, rosbuild will continue to provide the rosbuild_add_roslaunch_check() macro, but it will throw an error if it cannot find roslaunch.

There is more discussion of these issues, see rosbuild testing features.

$ROS_ROOT/bin executables

All executables, unless otherwise marked for removal, currently in $ROS_ROOT/bin will remain there. Binary executables will be converted to bash scripts using rosrun.

Motivation

Several main motivations have been identified for this separation

• Making the "core" ROS lighter weight, which enables - Easier porting to other platforms - Smaller footprint
• Enabling projects to use the ROS packaging and build system separate from the communication infrastructure
• Easier maintainability
• Better decoupling of design interests between the packaging and communication architectures

Rationale

rosbuild testing features

rosbuild explicitly uses the following packages:

• rosbuild
• rospack
• gtest
• roslib
• rostest

The use of rostest is an issue, because rostest will be in the ros_comm stack, while rosbuild will be in the ros stack. rosbuild uses the following tools from rostest:

• (*) bin/rostest-check-results
• (*) bin/test-results-dir
• (*) bin/rostest-results
• (*) bin/coverage-html
• (+) bin/roslaunch-check.py
• (X) bin/rostest

The tools marked (*) will be moved to the new rosunit package, but the rest will remain in ros_comm packages.

Options for handling this separation include:

1. Follow the rosbuild_add_roslaunch_check() model: leave everything where it is, and if bin/rostest isn't available, throw an error if the user calls any of rosbuild_add_gtest*() or rosbuild_add_pyunit*().
2. Separate out the bin/rostest --bare behavior and put it somewhere lower, such as rosunit.

We have chosen to implement Option 2, as it is the better long term solution. Conceptually, rosunit implements unit testing functionality, and rostest implements integration testing functionality. rosunit has no graph dependencies, though it does make use of the roslaunch process manager. This will be resolved in the short term by copying the process manager out of roslaunch. In the future, we can envision an implementation of roslaunch that does not depend on the graph and could be used as a library for rosunit.

Header, Clock, and Log

Three msg IDL files are currently stored in the roslib package as they are common to all ROS client libraries. The roslib package must go into the base ros stack as it contains many of the libraries used by the package and build tools, but we wish to keep the ros stack decoupled from client libraries issues.

The Clock and Log msg files are not considered to be user facing. They are also specific to the ROS middleware implementation and would pollute a more generic ontology like std_msgs. The designation of a new rosgraph_msgs package clearly associates these msg files as being specific to representing state of the ROS graph. We originally intended to create a roslang_msgs package, but decided that a rosgraph_msgs was a better categorization. In the future, we might add messages for the ROS Master state and other graph data.

The Header msg file is user facing and will go into std_msgs. It would be preferable to be able to move the Header msg out of the ROS stacks entirely, but that is out of scope for this proposal.

The impact of changing the location of these messages requires further study. The Header message is generally not referred to as "roslib/Header", which will hopefully minimize the impact in the ROS code base, despite it being commonly used. The Clock and Log messages generally have limited use outside of ROS client libraries (e.g. simulators, GUI tools), and will hopefully be easy to migrate.

Moving of roslib python modules

The roslib libraries selected for moving are generally to support ROS communication. Leaving them in the ros stack would create a coupling between ros_comm and the ros stack that would make future development difficult. Moving of roslib python modules is expected to have low external impact as these libraries are generally for internal use. It is expected to have high impact internally as many tool make use of these representations.

genmsg_cpp deletion

The genmsg_cpp package currently lives at the root of the ROS dependency tree so that it can be compiled before any messages are generated. This design causes many problems, though the main one of concern is that it introduces client-library-specific information in the base ROS stack. In order to meet our goals of decoupling the ROS packaging system from the ROS communication system, this implementation artifact needs to be removed. Despite its name, genmsg_cpp is no longer used for roscpp and only affects roslisp, rosoct, and rosjava. The effect on these packages is described in the Backwards Compatibility section.

Client libraries are instead expected to develop Python-based message generators (e.g. rospy and roscpp) or dynamically convert the msg IDL to code (e.g. roslua).

documentation stack

The placement of the rosdoc package was considered for both the ros and documentation stacks. The rosdoc tool is a generally useful tool as it provides a mechanism for auto-generating documentation across a ROS-package-based code base. Comparison to similar tools, like javadoc, shows that these documentation tools are generally distributed with the main system.

Thus, moving the rosdoc package to its own, separate "documentation" stack does generate problems from a user perspective. In particular, they may be confused that it is missing from the ros stack. This concern is balanced by multiple motivations:

• rosdoc has heavy-weight dependencies (rxdeps, graphviz, doxygen, sphinx, latex).
• Future implementations of rosdoc will likely break it into multiple packages in a plugin model (e.g. rosdoc_sphinx, rosdoc_rosorg, etc...).
• The ROS Distribution system and packaging system can provide the documentation stack in common configurations.

Keeping $ROS_ROOT/bin executables

The inclusion of ros_comm and rx executables with the ros stack is unfortunate. It does not significantly increase the size of the ros stack as binary executables will be replaced with bash scripts, but it does add non-functioning executables to a bare ros installation.

Our debian package system does not allow another ROS stack to build into $ROS_ROOT/bin. Work on a ROS install target (REP 102 [1]) will hopefully provide a clean solution in the future.

Backwards Compatibility

This separation is expected to break the existing, experimental rosoct and rosjava libraries due to the removal of the genmsg_cpp removal.

The roslisp message generator is currently implemented in genmsg_cpp, but is in the process of being converted to the Python-based approach.

For additional backwards compatibility, libraries for msg handling as well as core msg types (Header, Log, and Clock) will be left in the roslib package for the ROS 1.4 release. These will be marked as deprecated and removed in the ROS 1.5 release.

Scheduling

Early planning and cleanup for this separation went into the ROS 1.2 release, including rewriting the roscpp message generator in Python. For the ROS 1.4 release, the rx and documentation stacks are expected to be cleanly separated. For reasons discussed above, clean separation of the ros and ros_comm stacks are not expected for the ROS 1.4 release. In particular, the changes related to 'moving of roslib Python modules' is not expected to be fully completed by the ROS 1.4 release due to the difficulty in moving and renaming Python modules.

All preference will be given to maintaining a stable system over the separation. Although it is desirable to have a perfectly clean separation, the majority of our design goals even if this is not achieved in the ROS 1.4 release.

References

[1]	REP 102 (http://ros.org/reps/rep-0102.html)

Copyright

This document has been placed in the public domain.