API Design of ISIS Dependency Libraries - PSMRTS

[KrisBecker]

• Topic: ISIS External Library Dependency API Design - PSMRTS
• Date: October 12, 2023
• Author: Kris J. Becker

Introduction

The University of Arizona (UA) has been funded to deliver ISIS software developed during the OSIRIS-REx encounter at asteroid 101955 Bennu to the USGS ISIS repository for distribution to the science community. This effort has been funded as part of the OREX extended mission now called OSIRIS-APophis EXplorer (APEX), which will orbit the asteroid 99942 Apophis shortly after a close Earth flyby in 2029.

We intend to provide much of this work in the form of a shared library, called the Planetary Shape Model and Ray Tracing System, (PSMRTS), that will serve as an external ISIS build and runtime dependency. Our goal is to design the library such that it has utility and value outside of ISIS as a standalone package that can be used by the scientific community to aid in other research and development activities. This includes the USGS CSM environment among others. As such, we would like to solicit feedback and recommendations on best practices for design and implementation strategies that will help guide the development of this library.

1. Enhanced Support for Small Body Cartography in the ISIS System
   1. Implications for ISIS System
2. The Planetary Shape Model and Ray Tracing System Library
   1. PSMRTS Objectives and Design Specifications
   2. PSMRTS Features and Capabilities
   3. Examples of Existing Popular APIs
      1. NAIF SPICE Toolkit
      2. GEOS Computational Geometry
      3. TensorFlow Machine Learning Models
      4. FFI Concepts
   4. PSMRTS Example
      1. Details/Description of Figure 1.
3. Objectives of this Discussion

Section 1 will describe the features provided by PSMRTS . Section 2 describes the library API design detail and provides examples of potential architectures. Section 3 lists a number of questions and discussion items where we request feedback from USGS and the community at large.

1. Enhanced Support for Small Body Cartography in the ISIS System

The features and capabilities in this library were essential modifications and additions to the ISIS system that were required to meet the cartographic and mapping objectives of OSIRIS-REx. While there were some basic capabilities in ISIS for support of small, irregular body cartography, further evaluation and testing identified additional requirements. New features were added to the ISIS 3.6 version and maintained and tested by OREX during three years of proximity operations at Bennu. These features and capabilities were also used in additional projects that involved mapping of 433 Eros (NEAR), 67P Churyumov-Gerasimenko (Rosetta), 19P Borrelly (Deep Space 1), 81P Wild 2 (Stardust), 9P Tempel 1 (Stardust-NExT, Deep Impact) and 103P Hartley 2 (EPOXI (Deep Impact)). The following is a summary of these features and capabilities enabled by this contribution:

• Implemented shape model instance sharing of Bullet shape models to enable use in qview, qmos, findfeatures and jigsaw applcations to name a few.
  • Instance sharing allows for faster instantiation and less memory usage.
• Developed techniques to specify use of a ray tracing engine, such as Bullet, NAIF DSK and Embree in spiceinit that persists in each image thereafter.
  • Simplified image processing workflow.
• Added support for the OBJ format to the Bullet shape model.
  • All Bennu shape models were released as OBJ files.
• Implemented methods to specify multiple shape models that are shared for every image.
  • Significantly increases efficiency and eliminates out-of-memory errors due to redundant reloads of the same shape model.
• Implemented ray tracing prioritization of multiple shape models per image in the Bullet shape model system
  • Allows for tiling of multiple high resolution shape models to cover a region of interest.
• Implemented threaded loading of multiple shape models.
  • Significantly decreases startup times. Load times are nearly equal to a single OBJ load if there are sufficient threads available.
• Fixed Bullet facet partitioning that eliminates the need for USGS to maintain a special version of the Bullet library.
  • This enables direct use of any Bullet library release.
• Fixed support for regional, non-global shape models.
  • Necessary for site specific (e.g., sample site, landing site) cartographic work
• Fixes computations of emission and incidence angles due to stateful object errors in the NAIF DSK, Bullet and Embree shape models.
  • Current public ISIS implementation needs this fix. See #5036.
• Fixed issues in noproj where keywords were dropped that caused loss of ray trace engine history and resulted in errors in resulting image.
• Add explicit control of thread usage and ray tracing performance metrics.

These additions enabled the following capabilities and improvements in ISIS processing and mapping techniques.

• High precision foreground topography occlusion detection during orthorectified cartographic mapping.
• High precision sun illumination and shadowing detection. This is also a new backplane option added to phocube that can be used as a mask. It is also (optionally) considered in orthorectified mapping.
• Precise computations of local photometric angles and oblique pixel resolution.
• Runtime switching of ray trace engine and shape models without rerunning of spiceinit. This significantly improves support for footprintinit, jigsaw and orthorectified mapping as well as enhanced analysis and research activities (e.g., testing/comparisons of shape models in near real time).
• Development of simulated images from shape models for unsupervised feature matching and establishment of ground truth control networks.
• High precision control and orthorectified projections of cartographic global maps at 5 cm/pixel and 4 mm/pixel regional sample return site maps of Bennu.
• Full disk (i.e., flyby observations) control/bundle adjustment and geometric backplanes of various comets.
• Specialized application, shape2map, to convert tessellated shape models into ISIS 2.5D shape models.

1.i Implications for ISIS System

Upon completion and inclusion in the current ISIS version, this library will be used in an implementation of a new shape model class derived from the base class ShapeModel. This new shape model will fully replace and incorporate the functionality of the NaifDskShape, BulletShapeModel and EmbreeShapeModel classes into a single ShapeModel class implementation. This will significantly reduce external build/runtime dependencies, code volume, and reduce maintenance costs of enhanced cartographic and geometric support for small irregularly shaped celestial bodies in the ISIS system.

In general, there will be several categories of contributions that the APEX team will deliver to the USGS/ISIS repository:

1. Bug fixes, application/class improvements/enhancements and new applications and classes
2. A shared library that provides enhanced, specialized cartographic support for small irregular bodies
3. Documentation and tutorials

Item 1 will consist of normal Issues/PR cycles as they are completed and prepared for inclusion into the ISIS system. In addition to software documentation, item 3 will provide tutorials focused on the UA/ISIS enhancements that were used to process image data of Bennu from the OREX OCAMS and NAVCAM instruments. Item 2 is the main topic of this document. This is a standalone shared library that incorporates all of ISIS' current small, irregular body shape model cartography features and capabilities. This includes support for the NAIF digital shape kernels (DSK), Bullet Physics SDK, Intel Embree ray tracing libraries. These libraries are specifically designed for specialized ray tracing of high resolution object (or shape) models that are essentially tessellated plate models.

Tessellated plate models consist of a set of floating point 3D vectors that represent surface points from the object body origin and a set of 3D integer indexes that refer to these vectors in the order in which they are stored (in memory). This configuration formulates interconnected triangles (commonly referred to as facets) which describe the topography of an object's surface. These models can be stored in files in a variety of forms including NAIF DSK, Wavefront's OBJ, PLY and various other 3D shape model formats. This system will not support ISIS 2.5D digital elevation models (DEM) directly and will not have an ISIS dependency. However, ISIS DEM ShapeModel classes exist to support these shape models.

2. The Planetary Shape Model and Ray Tracing System Library

The new PSMRTS library will provide a C-based application programming interface (API) that is also application binary interface (ABI) compatible. Through abstraction and foreign function interfaces (FFI), this shared library will provide the high precision cartographic capabilities in the UA/ISIS system that were developed for the OREX encounter at Bennu. UA/ISIS mainly used the Bullet ray tracing system for cartographic processing of OCAMS images. However the NAIF and Embree systems were also useful for a variety of activities during the mission. Hence, support for these systems will also be included in the PSMRTS library.

PSMRTS will focus primarily on support for small irregularly shaped celestial bodies. PSMRTS will be provided as an external dependency similar to ALE, SpiceQL and other third party libraries ISIS uses for build and runtime requirements. Internally in ISIS, we will provide a ShapeModel class that encapsulates the PSMRTS usage which will minimize the scope of required changes and ease maintenance. As per the build and installation requirements for ISIS, the PSMRTS shared library will be available on Anaconda as a conda-forge install package.

2.i PSMRTS Objectives and Design Specifications

The objective of this document regarding PSMRTS is to solicit feedback on design and implementation strategies that will help guide the development of this library. The goal is to provide a versatile and feature rich system that can maximize utility in many computing environments. PSMRTS will be a pure C++ system that provides a C-based API/ABI. Our main focus will be to determine and implement the best FFI design that offers the most utility, versatility and ease of adaptability of C-like wrappers in programming languages such as Python, Java, TypeScript, Rust, and others. We do not intend to provide or maintain any other wrappers other than the C API wrapper and will therefore rely on other contributors to provide these interfaces. We aspire to make development and maintenance of these FFIs as easy and simple as possible.

2.ii PSMRTS Features and Capabilities

The PSMRTS system is intended to replace the ISIS small body cartography features and capabilities that currently exist in the system. During the OREX encounter at Bennu, the UA/IPWG significantly modified and enhanced support in an early version of the ISIS system. New capabilities were added in ISIS version 3.5.2 but for various reasons, were not at that time ported into any subsequent ISIS release.

Below are some of the capabilities and features of UA/ISIS that will be included in PSMRTS. The FFI will require APIs/ABIs that expose and/or make use of these utilities.

1. Provide a globally accessible, shareable pool of tessellated plate models comprised of 3D floating point vectors and indexes that create facet-based shape models and objects.
2. Initially provide support for and specification of NAIF DSK toolkit, Bullet and Embree ray tracing models for cartographic shape model configurations.
3. Ability to reuse existing shape models without creating or reloading copies of existing vector/index datasets.
4. Support sharing of existing ray tracing model instances to minimize initialization overhead.
5. Create prioritized ordering of ray tracing shape models by analysis and determination of most-to-least common ground coverage of high resolution shape models and camera FOV footprints.
6. Fundamental form of ray tracing is observer and look direction in body-fixed coordinates.
7. Avoid stateful shape models and ray tracing objects.
8. Prefer stack allocation versus heap allocations where appropriate to minimize overhead and memory fragmentation and maximize efficiency.
9. Provide thread-safe and threading where appropriate/indicated.

2.iii Examples of Existing Popular APIs

We have examples of various varieties of existing APIs in popular and widely used software systems. Many of these systems contain elements of FFI component design, and do not strictly adhere to or utilize a single feature of many possible FFI options. Most common FFI design/implementations use handles or opaque pointers. Each type has strengths and weaknesses, but we would like to hear from those who have experience with FFIs and help us make informed decisions that maximize usefulness and scope of PSMRTS.

Below are some examples of software systems that demonstrate uses of different forms and combinations of FFIs. We provide these as an example of existing libraries/APIs some that are incorporated into the ISIS ecosystem. These examples are intended to start the conversation as to how the PSMRTS API should be designed in a way that is most accessible and useful to ISIS and other ecosystems. It doesn’t have to be exact, the point is to communicate that we are giving these not as direct design guides or even because they matter individually, but because they are examples of how people currently do it and we want to hear if people like how they do it, or if they’d prefer something else.

2.iii.a NAIF SPICE Toolkit

The NAIF SPICE APIs uses handles to refer to certain file references and objects utilized in a global parameter, variable and object pool. There are several language wrapper implementations provided with this toolkit. The Python spiceypy wrapper is a widely used, independent wrapper of the NAIF C API toolkit.

The fundamental mechanism of the NAIF SPICE toolkit C API uses file handles to opened DSK shape models. All operations for ray tracing use the DSK file handle in its interface. Below is a small code example that issues a ray trace of a DSK.
The function dasopr_c opens a DSK shape model and establishes a reference using a handle. The handle is an integer value that serves as a reference to a structure internal to the NAIF SPICE toolkit library that is used in all subsequent operations related to the ray tracing using this DSK file. The ISIS class NaifDskPlateModel shows the current implementation that will be replaced by the PSMRTS library.

// Open a DSK and obtain the handle
SpiceChar  dskfile  [ 1024 ];
SpiceInt   dskhandle;
dasopr_c( dskFile, &dskhandle );

// Get file descriptor
SpiceBoolean  found;
SpiceDLADescr dladsc;
dlabfs_ ( dskhandle, &dladsc, &found );
if ( !found ) {
     std::cerr << "No segments found in DSK file " + dskfile ;
     exit (1 );
}

//  Find first segment...
SpiceDSKDescr dladsc;
dskgd_c ( dskhandle, &dladsc, &dskdsc );

// ... set these body fixed observer position and look direction vector
SpiceDouble  observer[3]; // == get_bf_observer_location();
SpiceDouble  lookdir[3];  // == get_bf_camera_look_direction();

// Ray trace the look direction for dksfile shape model...
SpiceInt     plateid;
SpiceDouble  surfpt[3];
dskx02_c ( dskhandle, &dladsc, observer, lookdir,
           &plateid,  surfpt, &found);
if ( !found ) {
     std::cerr << "Body fixed intercept not found!";
     exit ( 2 );
}

2.iii.b GEOS Computational Geometry

The GEOS computational geometry C-API/ABI uses structs and objects allocated on the heap and referenced using opaque pointers to these objects. This FFI is used to create the Python library shapely wrap the GEOS library.

The following small code example demonstrates use of the GEOS C-API that is based upon opaque pointers. This C-API is used in the ISIS system by the GisTopology and GisGeometry classes. Use of these classes is demonstrated in the GisIntersectStrategy class.

/* geos_hello_world.c */

#include <stdio.h>  /* for printf */
#include <stdarg.h> /* for va_list */

/* Only the CAPI header is required */
#include <geos_c.h>

/*
* GEOS requires two message handlers to return
* error and notice message to the calling program.
*
*   typedef void(* GEOSMessageHandler) (const char *fmt,...)
*
* Here we stub out an example that just prints the
* messages to stdout.
*/
static void
geos_msg_handler(const char* fmt, ...)
{
    va_list ap;
    va_start(ap, fmt);
    vprintf (fmt, ap);
    va_end(ap);
}

int main()
{
    /* Send notice and error messages to the terminal */
    initGEOS(geos_msg_handler, geos_msg_handler);

    /* Read WKT into geometry object */
    GEOSWKTReader* reader = GEOSWKTReader_create();
    GEOSGeometry* geom_a = GEOSWKTReader_read(reader, "POINT(1 1)");

    /* Convert result to WKT */
    GEOSWKTWriter* writer = GEOSWKTWriter_create();
    char* wkt = GEOSWKTWriter_write(writer, geom_a);
    printf("Geometry: %s\n", wkt);

    /* Clean up allocated objects */
    GEOSWKTReader_destroy(reader);
    GEOSWKTWriter_destroy(writer);
    GEOSGeom_destroy(geom_a);
    GEOSFree(wkt);

    /* Clean up the global context */
    finishGEOS();
    return 0;
}

This approach is being considered for maintaining results of ray traces where the returned constructs contain details such as the observer, look direction, status and surface intersections and normals from spacecraft instruments and light sources. Further computations can be made with these interfaces such as photometric data (e.g., emission, incidence and phase) as well as topographic occlusions.

2.iii.c TensorFlow Machine Learning Models

The TensorFlow machine learning system is developed mainly in C++. It is an open source platform for machine learning that provides stable Python and C++ APIs. The C API is designed toward simplicity and uniformity instead of convenience.

2.iii.d FFI Concepts

There are many other types of FFIs that can be used to produce C-APIs that are also application binary compatible (ABI). Design guidelines offer some ideas for FFI interfaces using mixed language programming techniques. We also intend to support the Windows platform. String handling is a fundamental need for most implementations and present challenges in mixed language environments.

2.iv PSMRTS Example

The following short code segment is an example of what the PSMRTS C-API might look like. The observer is assumed to be provided in body fixed coordinates for a framing camera via the spacecraft_body_fixed_position() method. The look direction is provided in the body_fixed_look_direction_vector() method in the CameraModel object.

CameraModel camera( "OREX", "OCAMS", observer_position_epoch );

PSMRTS_Object body = psmrts_init_object_tracer( "Bennu", PSMRTS_BULLET_SYSTEM );

psmrts_load_model( "l_00050mm_alt_ptm_2545n20105_v020.obj", body);
psmrts_load_model( "l_00050mm_alt_ptm_3241n19631_v020.obj", body);

double observer[3] = camera.spacecraft_body_fixed_position();
double lookdir[3];

for ( size_t line = 0 ; lines < camera.lines() ; line++ ) {
     for ( size_t sample = 0 ; camera.samples() ; sample++ ) {
          camera.body_fixed_look_direction_vector( line, sample, lookdir );
          PSMRTS_Raytrace ray = psmrts_intersect( body, observer, lookdir );
          if ( psmrts_raytrace_isvalid( ray ) ) {
               double radius = psmrts_radius( ray );
          }
     }
}

The figure below demonstrates the capabilities of the PSMRTS system.

Figure 1. Eight 5cm tile were used in orthorectified projection of the OCAMS image.

2.iv.a Details/Description of Figure 1.

Image A, 20190404T175836S168_pol_iofL2pan, was acquired of Bennu on January 4, 2019 at a distance of ~5km. It is radiometrically calibrated I/F with an average of ~7cm/pixel. Prior to projecting the image, it was controlled to ground using the shape model.

Image B is an orthorectified projected version of Image A into a equirectangular map using a 80cm global DTM. It has the outline of eight 5cm tessellated/faceted tiles superimposed over the map to provide context for Images C & D. The eight 5cm tiles, minimally and collectively, provide coverage of Image A ground FOV with nearly 35 million facets that are managed in the UA/ISIS Bullet Ray Tracing system for all geometric operations. In comparison, the 80cm global DTM has ~3.36 million facets for all of Bennu’s surface.

Image C is the same projection as Image B, but uses a list of 8 separate 5cm tiles without ray tracing prioritization. In this projection mode, all 8 5cm tiles are checked for surface intersection at each pixel. The tile containing the closest surface intercept is chosen and the body-fixed X, Y, Z coordinate of the DTM is converted to a corresponding latitude/longitude coordinate.

Image D is using the same list of 8 5cm tiles as Image C, but the ray trace from observer to surface is systematically applied to each 5cm tile, in the order provided in the DTM list, until a valid surface intercept is found. When the first valid surface intercept occurs, the other DTMs are not ray traced.

In Image C, you will notice significant anomalies that occurred during orthorectified mapping via ray tracing only in areas of common tile coverage. This indicates very slight (millimeter to centimeter) differences in common overlapping areas of the tiles. These differences are likely due to Poisson reconstruction being applied independently to each tile, resulting in minute differences in common areas, particularly on the edges of boulders and in large areas of differences in topography of the terrain. Image D does not show this same effect because of ray tracing prioritization of the DTM tile list and early termination upon the first surface intercept. It should also be noted that unprioritized processing takes significantly more time. The number of tiles that can be used for any single image is limited to the amount of memory available on the computer system.

3. Objectives of this Discussion

Below are questions and topics of which we are requesting feedback and discussion from the ISIS and scientific community. This identifies the type of information we seek to aid and support the development, utility and adoption of this library.

1. We are hoping for constructive comments/suggestions regarding best practices to develop a diverse, yet simple, implementation of the C-API. The objective is to provide an easy API implementation to code for other languages such as Python, TypeScript, Rust, etc…
2. How should strings be handled?
3. We expect significant returns (numbers and volume) of data from ray traces. Some criticisms of the Bullet system is the use of significant allocated memory, which is costly to manage when utilized in an environment such as this and results in memory fragmentation and decreased performance. What are the best ways to prevent this that does not lead to development/data management problems and performance degradation (specifically, we expect an extremely large number of ray tracing operations will occur – at least two per pixel)?
4. What is the most effective and easiest use of threading? Boost.Thread has a feature that provides an interrupt() mechanism that may be used to cleanly (i.e., no corruption, memory leaks, reentrant state, etc…) early-terminate a thread. But this must be considered in the C++ implementation and the effort may not provide the expected benefits.
5. How many shape model formats should we support?
6. For OBJ (and potentially other formats), do we also retain and provide the metadata to decorate the object? This will increase the overhead and complexity (and will be targeted for a future revision).
7. For clarity, all the Qt dependencies will be removed in favor of C++ standards to minimize dependencies and target standardized constructs. What other standards/approaches should be used?
8. What is the best way to specify the list of (prioritized) shape models and which ray tracing engine is preferred by the (ISIS, specifically, but not exclusive) user?
9. What type/level of debugging and threading control do users want/need?

And some issues specific to ISIS:

10. How does an ISIS user specify which ray trace engine to use? UA/ISIS uses an IsisPreferences file, that contains a ShapeModel group, specifed in spiceinit only and records in the Kernels group the RayTraceEngine keyword. This allows users to run editlab to change as needed/desired. This preserves the ray tracing engine until the kernel group disappears.
11. How should multiple shape model files be provided in spiceinit? In UA/ISIS, we specify a PVL file with a .conf file extension that contains a ShapeModel keyword containing the prioritized list of shape model files. This allows users to change the contents at will without any additional consideration. editlab can be run to set it to Ellipsoid if needed. Whatever is used, ShapeModelFactory must recognize this file and process appropriately.
12. The IsisPreferences file can also be used to further parameterize the ray trace engine. For example, you can specify the size (i.e., number of facets) of each partition in the UA/ISIS Bullet shape model. You can also add a one-time use of debugging and control use of threading upon loads for shape model files. You can also specify a Tolerance in meters of surface intercept precision. And how to behave should an error occur (e.g., continue to the next ray tracing engine or fail). How else might this be accomplished?

[KrisBecker]

I gave a talk this week at the ADASS 2023 Conference here at the University of Arizona titled Cartographic Mapping using High Resolution Shape Models. It highlights the features and capabilities of the PSMRTS system. The slides are available here. And there will be a paper available soon.

Also, for those interested, we have recently published a paper about the OSIRIS-APEX mission titled OSIRIS-APEX: An OSIRIS-REx Extended Mission to Asteroid Apophis.

[twilson271828]

@KrisBecker Kris, this library sounds amazing! I look forward to reading the paper when it is available. There are a number of technical problems this library appears to address that are also being faced by the research group I am working with. It sounds like you are working with a very talented group of people at UofA.

[KrisBecker]

Hey @twilson271828, thanks for the shoutout!

We are working on getting the library out in an open source setting. We wish to encourage community involvement and participation. Our goal is to make it useful in diverse settings and prepare for the OSIRIS-APEX encounter of Apophis. We are anticipating an initial release mid-2024.

Thanks...

[Kelvinrr]

@KrisBecker So I've been looking through the code on the repo. From looking it seems you have some of the proposed PSMRTS functionality working already using the bullet shape model? That is, within the existing ISIS shape model factory and plugin paradigm.

I think there is an important question here of: does this absolutely need to be a separate library with a different API? In other words, shape models already have a relatively well-defined API, is there a way to re-purpose that API and make a component that can also be installed on its own? I am not saying this is necessarily the best route but maybe something worth considering as it could potentially require less work.

To get at some of the questions proposed, I'm going to leave the more in the-weeds questions without obvious answers for later.

How should strings be handled?

AFAIK, sticking with C++ strings is the way to go. SWIG handles string conversions pretty well when wrapping C++ in other languages. I understand the number of string conversions being a performance concern, but I don't know a convenient way to solve this problem, personally.

For clarity, all the Qt dependencies will be removed in favor of C++ standards to minimize dependencies and target standardized constructs. What other standards/approaches should be used?

I'm generally in favor of sticking with std C++ whenever possible. It's not the most performant but makes things simpler to manage. The newer C++ standards cover a lot of what boost and QT used to do for us before, and what hasn't been implemented yet may have third party implementations that are designed to be replaced with the standard implementation (e.g. https://github.com/gulrak/filesystem).

What is the best way to specify the list of (prioritized) shape models and which ray tracing engine is preferred by the (ISIS, specifically, but not exclusive) user?

Configuration has always been a pain, with our newer libraries we have tried a priority list like: parameter, env var, user config file, default config. Where the setting passed in one is prioritized over the next. I think this is more or less a duplicate of question 10 other than ISIS might have to forward these preferences.

What type/level of debugging and threading control do users want/need?

I think this is really a question for users (obviously), but in our newer libraries we have moved to having obnoxiously dense logging with fine controls over what level the user wants. I would suggest this route.

How does an ISIS user specify which ray trace engine to use?

I think the implementation as I you describe and I see it in the code, using the kernels group, is reasonable. I imagine, at least within the ISIS environment, the best user-facing way to switch shape models outside of spiceinit would have to use a separate app (shapeinit?)

How should multiple shape model files be provided in spiceinit?

I would avoid extra configs if possible. I don't see why it couldn't be passed in? Maybe a system to pick the optimum shape models similar to kernels. If the models are shipped with ISISDATA, having portable paths should be possible.

The IsisPreferences file can also be used to further parameterize... How else can we accomplish this?

I think within ISIS, re-using the ISIS preferences file is a good idea. We should be able to overwrite with passed parameters, etc, similar to how I mention in the other question about params.

[Kelvinrr]

We considered just porting the current implementation in UA/ISIS 3.5+ into the current ISIS version. It seems that this approach puts a significant burden on the USGS ISIS developer staff in several areas. Just getting the revisions into the system would require a significant effort to complete the PR process. These changes are also quite invasive within the current API with high probability of API incompatibility changes that would delay adoption and distribution. Another burden would be the USGS staff taking on maintenance of this code base. Complete separation of PSMRTS limits these burdens significantly and greatly reduces its presence/footprint in the ISIS system. One anticipated outcome is to reduce the number of ISIS dependencies with PSMRTS.

It would be less effort for USGS if PSMRTS has a good host somewhere outside of the DOI-USGS namespace with clear and consistent governance. Even then, I imagine the effort to integrate the library would have similar challenges. I think the bigger question about re-using the existing ISIS plugin paradigm is more about:

1. Can these classes be shipped without significant portions of ISIS? That is, with more than what is in ISIS core. If not, how much work would be needed compared to re-engineering.
2. Does the ISIS C++ API lend well to wrapping in other languages? As we know, most of ISIS's API is far from the stable C-style API we would want. Therefore, how much effort is required to change that?
3. Is the ISIS C++ API for shape models useful on its own?

If the effort for 1+2 > new PSMRTS library (including all the complications of dealing with governance + update cadences from a "friend" dependency + integrating into ISIS), then it's probably worth going that route. idk if it is yet, maybe I can look at the code and get an idea. If 3 isn't true, then re-using ISIS's API is a non-starter.

SWIG provides API interfaces that parse code to generate a compatible callable API for other languages. This approach does not adequately separate the C++ API from other parsed/autogenerated language APIs to achieve stability. Any PSMRTS C++ API change would be incorporated into the other language APIs causing breakage. We aspire to produce a stable C API similar to GEOS and other systems to minimize the impact of PSMRTS evolution in other languages. Interestingly, the GEOS community has had several discussions about the C++ and C API usability and stability.

I guess I don't understand how C++ strings would cause the API to be unstable (maybe potential ABI changes?). SpiceQL heavily uses C++17 types for everything and most of the swig code is barebones. Most of the complications (from my experience) is more from relying heavily on patterns that vary widely from language to language. For example, python wrappers become a problem if you use overloading. Python is duck typed so therefore does not support overloaded functions. If a C++ lib predominantly uses types that are SWIG ships conversion code for already, then you wouldn't need to change much in the SWIG code if the API changes. Only if names of files changes. IMHO using c-style paradigms is more important than whether you only use c-types.

Very high resolution shape model tiles presents significant challenges for this feature. This is mainly restricted to spiceint. Some lists can be quite large and specification of the ray tracing engine is not available as parameters to spiceinit. This currently also needs the IsisParameter file. Simplifying this with, say, a config file, would be great. Using a more comprehensive JSON config file might be an acceptable alternative such as the ISD structures.

This feature has complications in that when using regional shape model tiles, ISIS requires, in many situations, surface intercepts of sub-spacecraft lat/lon, sub-solar lat/lon, and north/south poles. Many times the tiles won't provide coverage for all of these intercepts. So PSMRTS will provide the specification of a list of shape models, where one could be a global model. It seems that in many of these cases, an ellipsoid model would be adequate rather than adding the (likely significant!) resource overhead of loading and managing a plate model. This implies a different ray tracing model for this purpose.

If the list is so complicated, could we not implement a similar mechanism for choosing shape models like we do for kernels already? Does there exist a similar pattern of shape model quality as there is for kernel quality? If so, then the user can specify, for example, level of detail (LOD), similar to how one would specify kernel quality, and a spatial query can determine coverage. Therefore, we avoid the user having to manage a list of files.

[KrisBecker]

It would be less effort for USGS if PSMRTS has a good host somewhere outside of the DOI-USGS namespace with clear and consistent governance.

We will be hosting PSMRTS on a publicly accessible repo outside of DOI-USGS. Our intent is to adopt and follow open source governance practices that are conformant to NASA SPD-41a polices.

1. Can these classes be shipped without significant portions of ISIS? That is, with more than what is in ISIS core. If not, how much work would be needed compared to re-engineering.

These classes will be simplified by stripping out all ISIS dependencies with the goal to minimize dependencies within PSMRTS. We expect a modest amount of re-engineering will be required but the core functionality code will remain intact.

2. Does the ISIS C++ API lend well to wrapping in other languages? As we know, most of ISIS's API is far from the stable C-style API we would want. Therefore, how much effort is required to change that?

It does not seem feasible to provide an ISIS C API at this point. One main reason, as you point out, is ISIS is not stable and the C API would endure significant breaking changes. This makes the effort difficult to support and maintain until stability can be achieved. Abstracting many of the ISIS software infrastructure subsystems to standalone libraries and minimizing/eliminating non-standard and/or third party components would provide a better opportunity for this effort. That is a significant effort with unspecified timelines. PSMRTS is small component of this approach with expected availability and releases within a year.

3. Is the ISIS C++ API for shape models useful on its own?

The ISIS C++ API in its current form it is rather complex mainly due to other interactions (e.g., Target and PVL labels) within the ISIS s/w infrastructure. It would be rather difficult to isolate and provide a suitable interface to just the shape model subsystem.

I guess I don't understand how C++ strings would cause the API to be unstable (maybe potential ABI changes?)...

The instability issue is not restricted to C++ strings. SWIG would provide external interfaces to all the class parameters in some form that can be translated to the other language compatibilities. The end result would be exposure of significant content of ISIS s/w components which would ultimately contribute to instability of the C API. ABI compatibility would likely not be assured under these conditions. Continuous changes to the SWIG interface due to incompatibilities seems to be a zero sum game that would also occur, to some extent, in a custom C API. SWIG also adds another third party dependency that we wish to minimize for sustainability. Many other software libraries opt for a custom C API to address many of these concerns.

If the list is so complicated, could we not implement a similar mechanism for choosing shape models like we do for kernels already?

Surely this could be done. I described the approach we implemented in the mission operations environment. Formal applications and integration into ISIS could make this easier on users.

[Kelvinrr]

The instability issue is not restricted to C++ strings. SWIG would provide external interfaces to all the class parameters in some form that can be translated to the other language compatibilities. The end result would be exposure of significant content of ISIS s/w components which would ultimately contribute to instability of the C API. ABI compatibility would likely not be assured under these conditions. Continuous changes to the SWIG interface due to incompatibilities seems to be a zero sum game that would also occur, to some extent, in a custom C API. SWIG also adds another third party dependency that we wish to minimize for sustainability. Many other software libraries opt for a custom C API to address many of these concerns.

Maybe there is some confusion here? I was originally talking about the idea of this new API (not the current ISIS one), not being negatively impacted by using C++ strings. As Swig handles them natively, and whether you're using c-strings or C++ strings, you'd still have to change the wrapper code if the API changes. That is, assuming you are manually overriding or extending SWIG default behavior.

Surely this could be done. I described the approach we implemented in the mission operations environment. Formal applications and integration into ISIS could make this easier on users

IMHO, I think such a system would be necessary. Maybe use something akin to spatial-lite to index and query shape models, preferably in a way that avoids having to store and/or ship these indexes with the shape models like ISIS's .db files do.

[Kelvinrr]

The ISIS C++ API in its current form it is rather complex mainly due to other interactions (e.g., Target and PVL labels) within the ISIS s/w infrastructure. It would be rather difficult to isolate and provide a suitable interface to just the shape model subsystem.

What do you expect the process for integrating PSMRTS back into ISIS would look like? My first instinct would be to wrap it as a Shape Model plugin.

[KrisBecker]

Maybe there is some confusion here? I was originally talking about the idea of this new API (not the current ISIS one), not being negatively impacted by using C++ strings. As Swig handles them natively, and whether you're using c-strings or C++ strings, you'd still have to change the wrapper code if the API changes. That is, assuming you are manually overriding or extending SWIG default behavior.

We will take a look at this. It does look vey easy and there are a lot of positive things said about SWIG. To get something up and running this may be the way to go initially.

What do you expect the process for integrating PSMRTS back into ISIS would look like? My first instinct would be to wrap it as a Shape Model plugin.

Certainly a ShapeModel implementation will need to be provided.

[jlaura]

I have a number of questions that relate to the governance, lifecycle, and funding of this library. As this discussion is on the ISIS repository, I assume that a primary consumer of the proposed library is intended to be ISIS. As we are all well aware, ISIS is an infrastructural piece of software for the community and any dependencies to ISIS need to be equally well supported.

To get the discussion started, here are some high-level questions:

• What is the governance model that this project is going to adopt?
• What is the license for this library?
• Is this project adopting semantic versioning? If not, what versioning mechanism is planned?
• Is the project releasing an alpha / beta? What then is the proposed adoption plan for external, consuming libraries?
• How many persons are maintaining or supporting this library and what level of effort are you anticipating having them contribute annually? Asking to understand what the anticipated release schedule for this library is and at what cadence you expect the project to be able to incorporate bug fixes and enhancements (both from within the core dev team and from external contributors).
• What projects are you anticipating using this library outside of ISIS? This will help understand the level of community support the project is prepared to provide disparate users.

[KrisBecker]

All very good questions. I may not have all the answers to these questions now, but here is what I know.

This work is funded by NASA as part of the OSIRIS-APEX extended mission as an overguide funding request. Part of this proposal is to provide PSMRTS to the science community for maximum utility. PSMRTS functionality is currently entirely implemented in a old release of ISIS that was used for OSIRIS-REx mission mapping operations. It has also been used in several of our PDART proposals but otherwise has no other external exposure.

Our preference is to collaborate with the USGS to make this work available to the public/science community in ISIS and subsequent releases. This is preferred as OSIRIS-APEX intends to use ISIS for cartographic processing of asteroid Apophis (encounter in 2029) and this functionality is necessary for planned mapping objectives. We will also be contributing to the ISIS public release other improvements and bug fixes to ISIS applications and classes that were made during the OSIRIS-REx mission.

Clearly there is a certain amount of risk to this approach, given limited funding and resources. And USGS ISIS maintainers are under no obligation to accept/adopt PSMRTS in any form, direct or indirect as an ISIS dependency. Hence, to satisfy our agreement and mitigate this risk, we will focus on releasing PSMRTS as a standalone library under an open source model conformant to NASA SPD-41a guidelines. The intent is to provide this as an ISIS library dependency much like ALE and USGSCSM, which are USGS internally maintained libraries.

What is the governance model that this project is going to adopt?

We will adopt an open source software model that follows best practices in relevant open source and research communities as recommended in the SPD-41a.

No discussions or decisions have occurred regarding the governance model as of yet.

What is the license for this library?

As with ISIS, we intend to use the Creative Commons Zero license.

Is this project adopting semantic versioning? If not, what versioning mechanism is planned?

Semantic versioning is under consideration. We are also considering something similar to the GEOS version policy.

Is the project releasing an alpha / beta? What then is the proposed adoption plan for external, consuming libraries?

We will release an alpha version when we have a functioning version of the library that supports NAIF DSK toolkit and Bullet. Embree is less of a concern at this time as it does not support double precision. When Embree is added, there will be a beta release.

The PSMRTS system will provide a framework which is intended to ease support of adding of other ray tracing systems/libraries and shape model formats.

We will stand up a separate repo that will contain our initial integration of PSMRTS into ISIS as a dependency to establish a CI model to continually monitor ISIS compatibility. Once this model is functional, we will use this to validate results with our UA/ISIS version.

Note that one of our deliverables is documentation on how you use ISIS with this functionality and provide tutorials that regenerate the Recon A Nightingale mosaic (at 4 mm/pixel resolution) and other mapping products created during the mission. When validation is successful and functional in the integrated ISIS system, we will release a major version of the PSMRTS library. This system will serve as the initial implementation that we will issue a PR to the ISIS repo.

We will also work on making builds of PSMRTS available as an Anaconda install as well as making the library available in the Microsoft vcpkg repo. Consumers of the system can install from Anaconda or choose to install using vcpkg that provides additional build options. We will also intend to support Windows.

How many persons are maintaining or supporting this library and what level of effort are you anticipating having them contribute annually? Asking to understand what the anticipated release schedule for this library is and at what cadence you expect the project to be able to incorporate bug fixes and enhancements (both from within the core dev team and from external contributors).

We currently have two 0.5 FTEs working on the PSMRTS and ISIS project. We are adding an additional 0.20 FTE and a student to assist at 0.25 FTE. This allocation will continue for the next year.

Testing and development will continue at a minimum of 0.25 FTE developer/student support time during OSIRIS-APEX cruise to Apophis. We may pursue additional projects/funds as need arises during this time to support use and enhance the PSMRTS system until OSIRIS-APEX arrival at Apophis in 2029. Nominal mission operations are scheduled to end November 2030 with the potential for mission extensions. It is unclear what level of support will be provided beyond that but the project will maintain OSS status throughout and accept contributions from outside contributors.

What projects are you anticipating using this library outside of ISIS? This will help understand the level of community support the project is prepared to provide disparate users.

At this time, the project is not well known. We are addressing this by participating in conferences, such as ADASS where a talk was given and a paper will be published. We did have one inquiry at ADASS as to when the library would be released. So we do hope that others find the system useful, and will provide those users support consistent with OSS processes.

[Kelvinrr]

Testing and development will continue at a minimum of 0.25 FTE developer/student support time during OSIRIS-APEX cruise to Apophis. We may pursue additional projects/funds as need arises during this time to support use and enhance the PSMRTS system until OSIRIS-APEX arrival at Apophis in 2029. Nominal mission operations are scheduled to end November 2030 with the potential for mission extensions. It is unclear what level of support will be provided beyond that but the project will maintain OSS status throughout and accept contributions from outside contributors.

Thanks for the transparency on how much funding this currently has. Just to be explicit about it, does this mean that until 2030 there will be 0.25 FTE of effort put into this? Does this include work to perform releases, reviews, bug fixes and contribute internal code back into the repo?

[Kelvinrr]

To wrap up the rest of the questions for completion.

What is the most effective and easiest use of threading? Boost.Thread has a feature that provides an interrupt() mechanism that may be used to cleanly (i.e., no corruption, memory leaks, reentrant state, etc…) early-terminate a thread. But this must be considered in the C++ implementation and the effort may not provide the expected benefits.

I am more biased towards:

1. Not relying on boost, I feel it's a headache of a dependency to manage most of the time.
2. multi-processing over multi-threading. multi-processing is often much easier to deal with and can scale better in HPC environments.

Would it be possible to enable spawning single threaded tasks that spread large jobs over multiple processes instead?

How many shape model formats should we support?

I imagine the minimum would be everything that ISIS supports now, except for what is exclusive to ISIS like cube-based 2.5D models.

We expect significant returns (numbers and volume) of data from ray traces... What are the best ways to prevent this that does not lead to development/data management problems and performance degradation?

The best I can imagine accelerating an "extremely large" number of ray-tracing operations is to consider GPU acceleration (I think this is supported experimentally by bullet?). But, from previous experience, a "lazier" approach of enabling multi-processing, following some commonsense practices to prevent redundant operations (idk enough about the intricacies of bullet to know exactly what this would require, maybe you can give some more insight here), and throwing it on a cluster seems to work better and is less of a headache to implement.

[dgolish]

The UA ISIS (including the PSMRTS library) archival effort is funded by an overguide within the OSIRIS-APEX proposal at 1.0 FTE through ~FY25. That overguide funds only the archival effort itself (preparation of the code for submission, iteration with USGS during submission, and documentation of the code). All anticipated releases and reviews of the library will occur during this period.

However, OSIRIS-APEX will naturally continue to use ISIS and the PSMRTS library during cruise to prepare for asteroid operations and throughout the Apophis encounter. The University of Arizona will continue to fund an employee at >=0.5 FTE to support operations and data product processing. That FTE will support the PSMRTS library, including bug fixes and feature expansions if necessary, for APEX mission goals. If/when the community identifies bugs or other required changes to the library during this period, it will be in APEX's best interest to address those issues, particularly during cruise. Once operations begins, critical bugs will be addressed to achieve missions requirements, but other support will be of lower priority.