spring-boot-data-geode icon indicating copy to clipboard operation
spring-boot-data-geode copied to clipboard

Tie-up tasks for Cache Data Import/Export using JSON

Open jxblum opened this issue 5 years ago • 3 comments

The following things need to be investigated further:

  • DONE - Test with a ClientCache and client LOCAL Region(s) with no connections (i.e. no Pool defined) to a cluster/server using PDX

Currently Throws:

...
Caused by: org.apache.geode.pdx.JSONFormatterException: Could not parse JSON document: [Source: (String)"{"@type":"example.app.model.Customer","id":1,"name":"Jon Doe"}"; line: 1, column: 63]
...
Caused by: org.apache.geode.cache.CacheClosedException: Client pools have been closed so the PDX type registry is not available.

:(

  • DONE - Test using @EnableClusterAware annotation (this is currently not working because the "Cluster Configuration Push" happens in the SmartLifecycle phase of container startup and the "Import" happens in the BeanPostProcessor.postProcessAfterInitialization() callback on the Region, which occurs before the push... #sigh)

  • DONE - Test using @EnableCachingDefinedRegions annotation (i.e. Regions created from Spring's Cache Abstraction annotations or JSR-107, JCache API annotations)

  • Test using @EnableClusterDefinedRegions annotation (i.e. Regions created on the client from the server Region definitions)

  • Test export/import with entity having natural id (e.g. Book.isbn) that would need to include a @identifier JSON object metadata field. Possible for POJO stored in the Region since we can inspect the object class type, not possible when the Region value is a PdxInstance that did not originate from JSON. In that case, the user must manually make sure the non-JSON generated PdxInstance includes a @identifier PDX field before it is exported to JSON.

  • Add more documentation around features & limitations (gotchas): ** Java 8 Types, JSR-310 (Time) Types and Jackson 2 vs. 3, etc. ** Use of multiple Cache Data Importers and ordering using the @Order annotation or the Ordered interface on Cache Data Importer/Exporter bean definitions. ** Explain the (hidden) use of Predicate's to filter Regions on import and export. ** Explain the API/SPI for writing extensions, such as Cache Data Importers/Exporters for XML, XLS, SQL DML or other data formats. ** Explain the types of numeric values: e.g. application domain model class (e.g. Customer) might define an id field of type Long but the value for the corresponding JSON field (i.e. "id") will be a byte as processed by JSONFormatter, therefore Region.get(key) lookups must be customersRegion.get((byte) 1) even if the "Customers" Region is defined as Region<Long, Customer>. Also the Region entry value will be a PdxInstance and not Customer. ** ???

  • ???

jxblum avatar May 13 '20 18:05 jxblum

Completed handling for client LOCAL Regions. See Issue #91, commit and test class.

jxblum avatar Jun 16 '20 23:06 jxblum

Completed handling for Cache Data Imports/Exports in conjunction with the @EnableClusterAware annotation. See Issue #90, commit and test class, and specifically the configuration.

jxblum avatar Jun 16 '20 23:06 jxblum

Filed the following Apache Geode JIRA tickets related to these problems:

  • GEODE-8235 - "Server should not be required to have an available PDX type registry for ClientCache applications."

  • GEODE-8254 - "JSONFormatter cannot parse JSON Arrays."

  • GEODE-8255 - "JSONFormatter does not properly generate the @type metadata field."

  • GEODE-8256 - "The Jackson ObjectMapper used by a PdxInstance does not properly handle Java 8 Types."

  • GEODE-8257 - "_ The Jackson ObjectMapper used by a PdxInstance cannot handle typed JSON Objects._"

  • GEODE-8258 - "_ Cannot implement PdxInstance and store in a Region._"

jxblum avatar Jun 17 '20 13:06 jxblum