Consider GeoAPI for referencing and metadata aspects of new UCAR API

[desruisseaux]

Following on the NetCDF-Java - Looking Ahead blog, I propose to use GeoAPI for referencing by coordinates aspects (map projections) and possibly some metadata. Note that I'm not proposing to adopt any particular implementation — GeoAPI is only a set of Java interfaces, implementation can stay the UCAR one. This is an approach similar to JDBC for databases. GeoAPI can be at the hearth of UCAR new API or can be kept isolated as wrappers in a separated module, at your choice.

What is GeoAPI

GeoAPI 3.0.1 is an adopted Open Geospatial Consortium (OGC) standard. It takes the UML models of the following standards and translate them into Java interfaces:

• ISO 19103 — Conceptual schema language
• ISO 19115 — Metadata
• ISO 19111 — Spatial Referencing by Coordinates

For example map projection can be represented by an instance of ProjectedCRS. This interface provides methods for querying the datum, the axis units, their directions, etc. All those properties are defined in ISO 19111, which has been carefully designed by geodesic experts.

Using GeoAPI would add the following weight to the library:

• GeoAPI 3.0.1 JAR file: 212 kb.
• JSR-363 (unit of measurement) JAR file: 28 kb.

Benefits

A first benefit of using GeoAPI is to inherit knowledge from the ISO 19111 expert group. Projects designing their own API are at risk of missing important aspects if an accuracy better than a few meters is desired. Referencing by coordinates is both crucial to geospatial libraries and very often an underestimated challenge. Even the popular PROJ.4 library had significant design flaws (fixed in PROJ.6 by adopting the ISO 19111 conceptual model completed by inspiration from GeoAPI).

A second benefit is to give the possibility to plugin alternative implementations if desired. Existing implementations include JNI wrappers around the PROJ.6 library and Apache Spatial Information System (pure Java). If the netCDF library uses CoordinateReferenceSystem interface and CoordinateOperation for transforming coordinates, then the library can perform map projections using its own internal implementation, or using Apache SIS, PROJ.6 or other implementations at user choice and transparently.

A third benefit is that it can enable new services. For example it is possible to use Apache SIS Well Known Text (WKT) parser for reading a coordinate reference system definition and instantiating UCAR map projection objects (not necessarily SIS objects) from that WKT.

Another benefit is that if netCDF, PROJ.6 and Apache SIS API are derived from the same conceptual model (ISO 19111), then the similarities make easier for a developer familiar with one library to master the "referencing by coordinates" aspect of the other library.

Inconvenient

Less control from UCAR on that API (but GeoAPI development is open and any interested OGC members have equal vote). However it is always possible to define custom extensions in implementation classes.

Implementation strategy

Two approaches are possible. The first one is to use GeoAPI interfaces everywhere netCDF API needs to represent a coordinate reference system or a coordinate operation.

• Advantage: allow to plugin directly an alternative implementation when desired. More than one implementation can coexist in the same JVM.
• Inconvenient: GeoAPI dependency become mandatory (the JAR file is 212 kb).

A second approach is to keep the core of netCDF library independent of GeoAPI and implement GeoAPI interfaces as wrappers around UCAR netCDF in a separated module.

• Advantage: more flexibility about whether or not using GeoAPI and which version.
• Inconvenient: the capability to plugin alternative implementations into netCDF library (for example for using some PROJ.6 map projections not yet provided by netCDF library) is reduced or more difficult.

Existing material

Wrappers already exist as a demonstration project. I would be happy to donate that code if desired; it could be used as a starting point. I can also volunteer for contributing to the work of introducing GeoAPI in the netCDF library if desired, and proposing maintenance when GeoAPI is updated to a new version.

[JohnLCaron]

Hi Martin:

Im looking over your org.opengis.wrapper.netcdf code to see what it looks like, and what changes would be needed to use the new netcdf-java APIs. Our API is extensively rewritten and not backwards compatible. We are using a 6 month major release schedule, and look to release version 6 by the end of the year, and start version 7 after that.

We are considering the possibility of adding a separate geoapi module that wraps netcdf-java to provide an implementation of GeoAPI.

If I understand correctly your package needs org.opengis:geoapi:4.0-SNAPSHOT? Is this available in any maven repository?

Any comments on how complete your package is? Is there a lot of work still to be done to implement GeoAPI? Im looking hopefully at just porting your work to the new netcdf-java library.

Regards, John

[desruisseaux]

Hello John

This is good news! A separated module wrapping new netCDF API looks good to me. I propose to implement GeoAPI 3.0.1, which is current OGC official release and is deployed on Maven Central. In Apache SIS and in PROJ-JNI wrappers I have a separated branch named "geoapi-4.0" which implements GeoAPI 4.0-SNAPSHOT, but I do that only for anticipation. I suggest to omit that branch if too cumbersome. I can help to downgrade the dependency from 4.0-SNAPSHOT to 3.0.1; the changes for netCDF wrappers should be small.

Before to implement GeoAPI interfaces, a first step would be to get an implementation of javax.measure.Unit interface defined by JSR-363 (Units of Measurement). This is the only GeoAPI dependency. I think that UCAR has its own unit system, so you could choose to create javax.measure.Unit wrappers around UCAR implementation. Alternatively you can choose an existing JSR-363 implementation. This is what the current org.opengis.wrapper.netcdf package does for simplicity, but not necessarily what I would recommend. Instead if UCAR invested in implementing a unit of measurement framework it makes sense to use it.

In the meantime I can prepare in a separated repository an org.opengis.wrapper.netcdf package downgraded to GeoAPI 3.0.1 and without the Units class (or with an empty skeleton) if we replace that class with wrappers around UCAR unit system. You could then use that material for adaptations to new UCAR API.

Third step would be to migrate the tests that do not depend on GeoAPI 4.0-SNAPSHOT. After completion, I would suggest to delete the org.opengis.wrapper.netcdf package from GeoAPI repository and redirect to UCAR repository instead.

Does above plan would be okay?

Regards, Martin

[JohnLCaron]

Hi Martin:

1. Why do you recommend against using "an existing JSR-363 implementation" ?
2. Downgrading to GeoAPI 3.0.1 seems like the right thing to do.
3. How complete is the existing org.opengis.wrapper.netcdf towards implementing all of geoAPI?

Reagrds, John

[desruisseaux]

Hello John

It is not exactly a recommendation against using an existing JSR-363 implementation. It seems to me that UCAR library has its own unit system, in which case I assume that using that system may give more consistency with UCAR internal working. If a user gets a unit from UCAR (e.g. from some read operation) and later gave that unit to another UCAR method (e.g. to some write operation), we could avoid the risk of information lost that a "UCAR unit → JSR-363 implementation → UCAR unit" cycle may cause. Instead it would be just wrap/unwrap.

It is also possible to not commit to any JSR-363 implementation, let user choose the JSR-363 implementation of his/her choice and see later if doing wrappers around UCAR unit system is desired. Actually the org.opengis.wrapper.netcdf package is already designed that way.

For question 3, the existing org.opengis.wrapper.netcdf wrappers cover 2 or 3 parts:

Metadata (ISO 19115)

That standard contains about 400 properties. The current wrappers map about 15~20 of them to global attributes. E.g. the identificationInfo/extent/geographicElement/eastBoundLongitude property maps to the ACDD "geospatial_lon_max" global attribute. I think current wrappers provide a good starting point; more mapping can be added progressively in the future.

I think that variable attributes are not yet mapped, but I can help for completing that part.

Referencing (ISO 19111)

Current wrappers are complete enough for instantiating geographic, projected, vertical and temporal Coordinate Reference Systems (CRS) backed by UCAR map projection engine, and for performing coordinate transformations between them. It can be demonstrated by executing a subset of GeoAPI conformance test suite, which verifies the map projection correctness.

Future work

The wrappers contains some work for the support of grid coverages (rasters), but the corresponding GeoAPI interfaces have not been officially released yet. That part would be omitted for an initial release based on GeoAPI 3.0.1 (users can use directly UCAR API in the meantime), but can be kept on a branch for future work.

[desruisseaux]

The Apache Software Foundation (ASF), the Open Geospatial Consortium (OGC) and the OSGeo foundation will do a joint code sprint on February 17 to 19. I proposed an OGC GeoAPI track during that code sprint, with tasks listed in a https://github.com/opengeospatial/joint-ogc-osgeo-asf-sprint-2021/issues/2 comment. Would it be an opportunity for starting this "GeoAPI wrappers for UCAR library" task?

[desruisseaux]

Hello John

I would like to take the opportunity of OGC/ASF/OSGeo code sprint next week for getting the ball rolling. I would like to create a GitHub source code repository and start migrating the current GeoAPI wrappers for UCAR library. What should be the actions?

• Do we create a repository in UCAR group, or do I create a temporary repository in some personal space for giving you more time to evaluate?
• Which package name?
• Which license? (current code is in public domain, so we can apply whatever license you wish).

Maybe a good way to progress would be to have someone from UCAR to register to the code sprint (this is free and open to everyone) and connect to the GeoAPI virtual room sometime during those 3 days?

[lesserwhirls]

Greetings @desruisseaux! I think the code sprint would be a great opportunity to start this work. Let me ask a few others at Unidata if they have plan to attend already, and if not, I'll get signed up. I'm happy to setup a repository within the Unidata organization for this effort. In terms of a package name, not quite sure. Do you have any suggestions?

[desruisseaux]

Hello Lesser, thanks for the reply! For the package name, I suggest something starting with ucar for consistency with existing UCAR API. Maybe ucar.geoapi?

[lesserwhirls]

Greetings @desruisseaux - it looks like I missed the deadline to register (sorry about that). Would I still be able to drop into the GeoAPI virtual room?

[desruisseaux]

I think so. Let me send an email to the organizer.

[desruisseaux]

Hello all. The GeoAPI channel is open from today to Friday inclusive: https://meet.jit.si/GeoAPI

If anyone is curious about what are the other channels, the complete list is here.

[lesserwhirls]

Thank you @desruisseaux! I'm going to drop in the GeoAPI virtual room today at 1:30 EST.

[lesserwhirls]

For those following along, the initial work on this effort will be done over at https://github.com/Unidata/geoapi-netcdf-java.

Here is a summary of the next steps (much more detail above):

• Move the netCDF-java based wrappers from their current location into the new, temporary repository.
• Upgrade the wrappers to use version 5 of netCDF-java and the new public API (currently using netCDF-Java version 4).
• Investigate a netCDF-Java udunits package based implementation of javax.measure.Unit interface, defined by JSR-363 (Units of Measurement).
  • This could be done as a direct implementation in the ucar.nc2.units package (provided by edu.ucar:cdm-core) and making it part of the public API, or by providing an optional wrapper based on the ucar.units package (provided by edu.ucar:udunits). I'm personally leaning towards the first option.

[desruisseaux]

Hello @lesserwhirls . If you have a chance, could you connect to the Jitsi room at about 16:00 EST (30 minutes before the brief in the main code sprint room) for a quick update on current state?

[lesserwhirls]

Sure thing, but I will only be able to talk via chat as I will be in another meeting from 15:00 - 17:00 EST. Will take be ok?

[desruisseaux]

Yes no problem. We can also postdone to tomorrow if more convenient.

[desruisseaux]

Hello @lesserwhirls . If you have 10 minutes available between 15:00 and 15:30 EST (the end of the code sprint), I could share my screen for a small demo and for showing some key points in the code. If you do not have time, no problem.

[lesserwhirls]

I'm so sorry Martin, I'm just now seeing your message. I had planned on trying to join the room today but I could not make my schedule work. I'd love to meet up sometime next week if you have the time / availability.

[desruisseaux]

Sure just let me know when you will be available (no rush). If the https://meet.jit.si/GeoAPI is not open any more, we will create another Jitsi room on another server.

[desruisseaux]

Hello @lesserwhirls . Do you have some availability this week or later for a short presentation (~10 minutes) on what the wrappers can do? During the Joint ASF-OGC-OSGeo code sprint, I presented a demo opening a netCDF file using UCAR library and formatting its Coordinate Reference System (CRS) in Well Known Text (WKT) format using Apache SIS.

[lesserwhirls]

Hello @desruisseaux! I do have time this week and next to meet you for a demo. What would be your preferred method for coordinating a time?

[desruisseaux]

Thanks for the merge! I will update GeoAPI web site tomorrow for redirecting to UCAR. For a demo, any day of your choice except Thursday is fine for me, if possible sometime between 11:00 and 16:00 Pacific time (I presume this is your time zone?).

[lesserwhirls]

I am in Mountain time (currently UTC-6. In which time zone are you located? How does 12:00 MDT (18:00 UTC) on Friday, March 19th look for you? I'm happy to create a Google Meet for us to use, or we can use another platform if you'd like.

[desruisseaux]

Friday 18:00 UTC is perfect for me. This is 19:00 in my time zone. Google meet is fine for me.

[lesserwhirls]

Excellent - I've just sent out an invite to your geomatys email address for this Friday. Is it ok with you if I invite a few others from Unidata (and possibly @JohnLCaron if he is available?)?

[desruisseaux]

Sure!