astrobee
astrobee copied to clipboard
nasa
Localization should function better in environments darker than the nominal JEM configuration
Astrobee ISS activity AB104 ISAAC-4 (2022/02/08) had crew manually move Bumble around the ISS Node 2 module to collect data for building a sparse map. This was our first time collecting Astrobee imagery in Node 2.
During ISAAC-4, we were surprised to see the NavCam imagery was severely underexposed for the most part, despite the fact that the crew had set the lighting to "nominal" levels for Node 2, and Astrobee's fixed NavCam exposure settings were set normally. We realized that, compared to the JEM module we normally operate in, there are fewer light fixtures in Node 2, they are pointed differently, and because Node 2 typically contains more stowage items attached to the walls, there may be an increased impact from shadows cast by bags placed near the lights.
Toward the end of the activity, we were able to manually increase the NavCam gain parameter in cameras.config from 50 to 125 and saw images that qualitatively appeared much brighter and closer to properly exposed.
The recommended response is to consider a variety of strategies for Astrobee to better accommodate variable lighting. It may help to adopt a dynamic exposure configuration strategy for the NavCam (which could be traditional auto-exposure/auto-gain, or perhaps a more conservative custom approach like manually adjusting the exposure when moving into a new module). Alternatively, the sparse mapper and onboard localization software may be able to maintain acceptable performance with darker images, possibly helped by tricks like pre-processing the imagery or tuning parameters used in feature extraction.
With any of these approaches, it's important to understand our requirements for lighting/exposure consistency between prior sparse mapping time and localization time. Dynamic exposure settings make achieving consistency more complicated. If a high degree of consistency is in fact required, is that a necessary property of our localization approach, or an implementation issue that can be mitigated with further study?
The biggest sensitivity in localization is to individual lights going on and off, rather than to a uniformly darker image. (But all things being equal, a brighter but not saturated image is better than a dark image).
The histogram equalization is quite good at taking care of such global brightness changes. And SURF can handle things either way but a lower threshold should be set in map-building.
In, short, I think this is not a show-stopper. I guess the simplest way of figuring things out is to build a little map, say just 10 images, without registration, and examine the number of features as it does the work and later by looking at things with nvm_visualize. Then to rebuild it with BRISK features and to localize against it.
One could use lower thresholds, if necessary. For example, build_map can be invoked with
--min_surf_features 400 --max_surf_features 600 --min_surf_threshold 5 -default_surf_threshold 10 --max_surf_threshold 1000 --early_break_landmarks 400
(The above thresholds could be a little too low, and then give too many features, in that case they can be increased somewhat.)
When rebuilding the map with BRISK, one should not forget to use the --histogram_equalization flag, and one should consider lowering --default_brisk_threshold and and --max_brisk_threshold to 70 from the current 90 or 100 or so.
When localizing, if no luck with default params, one may consider editing localization.config and also decreasing those thresholds to values as above.
