REP: 122 Title: Filesystem Hierarchy Standard layout changes for ROS Author: Ken Conley, Troy Straszheim, Morten Kjaergaard, Ethan Rublee, Tully Foote, Brian Gerkey, Jack O'Quin, Dirk Thomas Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 8-Feb-2012 ROS-Version: Groovy Post-History: 8-Feb-2012, 17-Feb-2012, 21-Sep-2012
This REP describes a new ROS directory hierarchy for installed files based on the Filesystem Hierarchy Standard (FHS) [1]. It discusses the implications of those changes, and their effects on ROS code being converted to that standard.
To allow installation of multiple ROS distributions on a single
machine, this layout is normally created within a
distribution-specific install prefix. But, this specification supports
installs within /usr, as well.
This REP has evolved since the original Fuerte draft. The "Fuerte Differences" section explains differences between that preliminary implementation and the current specification.
The majority of this specification is due to the hard work of Troy Straszheim and Morten Kjaergaard on the catkin build system, which is the reference implementation on which this REP is based. This REP does not document catkin itself and is intended to be implementation-neutral.
This specification defines changes to the filesystem layout, associated environment variable updates, and necessary modifications to ROS itself.
This document mostly avoids details of the catkin build system [10].
For clarity, the install prefix is referred to as
/opt/ros/<distro>. The install prefix does not have to follow this
exact pattern, but it is the convention for ROS binary distributions.
This layout is also intended to enable installations within the
standard /usr hierarchy.
The layout within the install prefix is:
bin/ etc/ include/ lib/ share/ stacks/
This generally follows FHS conventions, with the addition of the
stacks directory for backwards compatibility with rosmake
packages. The install also contains setup files to easily configure
environment variables for this prefix.
These subdirectories are described in greater detail below.
bin is used to store top-level executables, i.e. executables that
should be included in every ROS user's PATH. This replaces the
old ROS_ROOT/bin location that stored core ROS binaries, like
rostopic and rospack.
ROS packages that are converted to use this new layout may install
into bin, but should do so sparingly as it is included in the
user's PATH. Developers should also beware of name collisions in
thise executables. In general, try to name executables so that their
origin is obvious, perhaps using the ROS package name.
In the case of a special install prefix, PATH must include this
location, e.g.:
PATH=/opt/ros/<distro>/bin:$PATH
This location is used to install global configuration files for a
particular ROS distribution. Within the context of ROS, we introduce
etc/ros, where ROS-specific system configuration files are stored.
Additionally, there is an etc/catkin and etc/langs that are
used by the catkin build system. Although etc/langs configures
the active ROS message generators, its implementation is specific to
message generation using catkin, not specified here.
ROS packages using the new install layout must install all header
files to the include directory. Assuming a ROS package uses the
namespace <ros-package-name>, its include files are stored in:
include/<ros-package-name>/
The specific include directories for a package are located using
pkg-config or CMake find_package().
ROS packages using the new install layout must install library files
to the lib directory.
Python modules are installed to lib/pythonX.Y, where X.Y is
the Python version. This directory follows the standard Python
specification for the target platform, which may be
lib/pythonX.Y/dist-packages, or lib/pythonX.Y/site-packages,
or somewhere else.
In the case of a special install prefix, PKG_CONFIG_PATH must
enable locating libraries and include directories via pkg-config.
LD_LIBRARY_PATH must be also set so that executables can load
shared libraries. We use the settings:
PKG_CONFIG_PATH=/opt/ros/<distro>/lib/pkgconfig:$PKG_CONFIG_PATH LD_LIBRARY_PATH=/opt/ros/<distro>/lib:$LD_LIBRARY_PATH
PYTHONPATH must also be set in a special install prefix and is
platform dependent. For example, on Ubuntu Precise with Python 2.7, the
setting is:
PYTHONPATH=/opt/ros/<distro>/python2.7/dist-packages:$PYTHONPATH
All package-relative executables and scripts are installed in
lib/<ros-package-name>/ subdirectory.
All architecture-independent package-relative assets are explicitly
installed to share/<ros-package-name>. These include:
- package definition files (
package.xml) - launch files
- other explicitly installed package-relative data such as bag files, or parameter definitions
This is the installation location for rosmake-based stacks. It is
included in the ROS_PACKAGE_PATH, but has lower precedence than
packages following the new REP 122 layout. Converted packages do not
use this folder.
For backwards compatibility with rosmake build files, ROS_ROOT
must exist within the new layout. However, its scope has been narrowed
to finding legacy rosbuild and mk resources. Other uses, such
as finding ROS binaries, are no longer supported. Thus, ROS_ROOT
is deprecated. Although there are no plans to remove it, avoid using
it for any new features.
For a full specification and rationale, see REP 123 [9], which also proposes new ROS filesystem environment variables based on the REP 122 layout.
Python modules now install into a dist-packages/site-packages
in the lib directory in the install prefix. This installation
uses the standard Python distutil/setuptools toolchain.
The Python install directory is added to the PYTHONPATH. Thus,
these modules are immediately accessible via Python import and do
not require a roslib.load_manifest().
In order to prevent unnecessary accumulation to the Python
sys.path, roslib.load_manifest() does not recurse through
packages built with catkin.
Python modules and scripts that are part of the FHS install layout
must not use load_manifest as it does not work properly within
an out-of-source build.
The ROS Groovy release migrates many low-level libraries to use a
standard Filesystem Hiearchy Standard-like layout [2] contained in a
ROS-distribution-specific install prefix
(e.g. /opt/ros/groovy/). The goals of this conversion are many:
- Make ROS libraries more "normal"
- Enable the ROS toolchain to use existing libraries for packaging
and release (e.g.
git-buildpackage) - Enable ROS messages, libraries and tools to be easily integrated
with other build systems via
pkg-configand CMake'sfind_package(). - Enable ROS binary releases to be accepted into Debian, Ubuntu, and other Linux distributions.
- Preserve the ability to install multiple ROS distributions on the same machine.
- Eventually eliminate the need for ROS-specific filesystem
environment variables if the installation is to a standard prefix,
such as
/usr.
The final goal should be possible with converted stacks, but has not yet been verified. The main focus has been on preserving the ability to install multiple ROS distributions on the same machine, as robots frequently are shared, expensive resources.
Compatibility with the Filesystem Hiearchy Standard has been a long-term goal for ROS. ROS began as an in-source, package-relative system for finding, developing, building, and distributing code. Multiple attempts have been made to transition towards an FHS-compatible layout for ROS, including the withdrawn REP 102 [5] and a prototype "rosbuild2" system that was deployed in ROS Electric. Experiences with those previous efforts have enabled us to accomplish this goal with minimal compatibility issues.
One major goal these changes have enabled is much better integration with external libraries, such as PCL. The standalone version of PCL 1.x can now build against the required ROS messages using CMake, although with significant modifications. The intent is for ROS to use the standalone PCL 2.0 libraries without modification. Hopefully, many more libraries will be able to transition to standalone implementations.
The rationale for using the Filesystem Hierarchy Standard is simple: it makes ROS releases conform to generally-accepted open source practice. Most of this REP describes straightforward translations to that standard.
There are no known regressions related to the new layout for building
rosmake stacks that are specific to this specification. We have been
able to build legacy stacks on top of the REP 122-compliant stacks by
using the ROS manifest "export" mechanism with the output of
pkg-config. No changes were necessary to the rosmake toolchain
based on this REP. Legacy stacks still use the same installation
procedure as before.
We have put together a migration guide [6] that generally notes issues with increased link/include strictness that are separate from this REP.
rosmake will recurse into packages that are installed in share.
As this location does not contain Makefile files, it will treat
this as not buildable.
The changes in this specification break compatibility with roslaunch remote launches as the environment configuration is very different. The non-backwards-compatible changes required in roslaunch are specified separately in REP 124 [7].
As described above, ROS_ROOT preserves backwards-compatibility for
mk and rosbuild files. For example,:
include($ENV{ROS_ROOT}/core/rosbuild/rosbuild.cmake)
resolves to the same path.
The Python environment bootstrapper, roslib.load_manifest(), is
safe to use in rosmake-based packages. However, it is not necessary
for importing Python modules that use the FHS install layout.
As noted above, roslib.load_manifest() should not be used in
converted packages.
Changes to both ROS_ROOT and environment setup have created
incompatibilities with the rosinstall tool. There are also separate
issues with rosinstall and catkin compatibility when building
completely from source that are separate from the issues of install
layout changes. These are resolvable and are handled by the affected
scripts.
The share/ does not preserve the full contents of ROS packages
within the installed layout. This affects package-relative workflows,
like using``rosed`` to quickly view header file information. There
are two categories of contents no longer locatable by package-relative
tools:
- Resources that are installed in a different location (e.g. C++ header files in
include)- Sources resources (e.g. C++
.cpp files) that are no longer installed due to explicit installation targets.
With Groovy, stacks and packages no longer have the same container
relationship as before. It is no longer possible to determine the
packages that a stack contains in the install layout. So, the
share/ layout does not preserve any stack-package relationship.
The tool most directly affected is roscreate-stack. Although
convenient, it has little utility moving forward, so we made the
difficult decision to delete it.
A reference implementation was initially deployed in the ROS Fuerte release. There have been some changes since then, which will be incorporated in the ROS Groovy release.
This section notes differences between the Fuerte implementation and the current specification.
In Fuerte the FHS layout was only applied during the installation
step. The build folder was not a supported environment to run the
compiled binaries and generated code. While some simple applications
worked features like i.e. discovering plugins were not supported in
the build folder. Therefore the make install step was always
necessary before developers could run their modified code. This made
it less convenient and increased the round-trip time during
development substantially.
In Groovy the build folder contains a subdirectory buildspace
which resembles the same structure than an installation. The content
of that folder also follows FHS conventions. It enables to run any
application from buildspace as well as if it would be installed.
Fuerte package manifests [3] had a <catkin/> tag to indicate that
they are built using the catkin build system. That tag was a
performance optimization for tools like rosmake and the Python
roslib.load_manifest() API. It was not required, but was strongly
recommended.
Catkin packages in Groovy replace the traditional manifest.xml
with a newly-defined package.xml, defined in REP 127 [11].
ROS Fuerte used share to install all ROS manifest and
package-relative resources. That included package-relative binaries,
which is contrary to normal FHS standards as share/ is intended
for "read-only architecture independent data files" [8].
As documented above, some archtecture-independent files (like launch
and parameter files) are still installed to
share/<ros-package-name>/. But, some that landed there in Fuerte
are now omitted or stored somewhere else:
manifest.xmlis no longer usedstack.xmlis no longer used- package-relative-executables (both binary and script) moved to
lib/<ros-package-name>/instead.
| [1] | Wikipedia: Filesystem Hiearchy Standard (http://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard) |
| [2] | ROS filesystem environment variables (http://ros.org/doc/api/rospkg/html/environment.html) |
| [3] | ROS Manifest XML Tags Reference (http://www.ros.org/doc/api/rospkg/html/manifest_xml.html) |
| [4] | REP 109: Unary Stacks (http://ros.org/reps/rep-0109.html) |
| [5] | REP 102: ROS Install Target (http://ros.org/reps/rep-0102.html) |
| [6] | ROS Fuerte Migration Guide (http://ros.org/wiki/fuerte/Migration) |
| [7] | REP 124: Changes to roslaunch and rosrun for REP 122 and catkin build system (http://ros.org/reps/rep-0124.html) |
| [8] | Filesystem Hierarchy Standard 2.3 (http://www.pathname.com/fhs/pub/fhs-2.3.pdf) |
| [9] | REP 123: ROS_ETC_DIR, ROS_DISTRO environment variables and ROS_ROOT changes (http://ros.org/reps/rep-0123.html) |
| [10] | Catkin build system documentation (http://ros.org/wiki/catkin) |
| [11] | REP 127: Specification of package manifest format (http://ros.org/reps/rep-0127.html) |
This document has been placed in the public domain.