JAXB/Moxy Unmarshalling assigns all field values to Map<String,Object> rather than the specific field provided for it

[Aravinda93]

I am using the JAXB/MOXY within my application and it reduces my code drastically. However, I am facing a small issue due to the @XmlPath. I tried a lot of things but none was able to help so posting the same here.

I have posted the complete issue with the code sample on StackOverflow: https://stackoverflow.com/questions/67648941/jaxb-moxy-unmarshalling-assigns-all-field-values-to-mapstring-object-rather-th

Basically, the problem is that: During unmarshalling, the dedicated fields are not taken into consideration at all. All the values are unmarshalled to Map<String.Object>. As per my understanding it's happening because of the annotation @XmlPath(".") but If I remove this annotation then it won't work as expected. Can someone please help me with this issue?

Following is my XML that I would like to convert to JSON: (Note: Name and Age are dedicated fields and others are the user-defined field.)

<Customer>
  <name>BATMAN</name>
  <age>2008</age>
  <google:main xmlns:google="https://google.com">
    <google:sub>bye</google:sub>
  </google:main>
</Customer>

Following is my Customer class used for marshaling, unmarshalling by Moxy and Jackson: (Note: Name and Age are dedicated fields and others is the user-defined field. I want Map<String,Object> others field to store only the values that cannot be mapped directly to POJO such as google:main and its children from above XML)

@XmlRootElement(name = "Customer")
@XmlType(name = "Customer", propOrder = {"name", "age", "others"})
@XmlAccessorType(XmlAccessType.FIELD)
public class Customer {
  private String name;
  private String age;

  @XmlPath(".")
  @XmlJavaTypeAdapter(TestAdapter.class)
  private Map<String, Object> others;
  //Getter, Setter and other constructors
}

Following is my Main class which will be used for marshaling and unmarshalling. Also, to convert to JSON and XML.

class Main {

  public static void main(String[] args) throws JAXBException, XMLStreamException, JsonProcessingException {

    //XML to JSON
    JAXBContext jaxbContext = JAXBContext.newInstance(Customer.class);
    Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
    InputStream inputStream = Main.class.getClassLoader().getResourceAsStream("Customer.xml");
    final XMLInputFactory xmlInputFactory = XMLInputFactory.newInstance();
    final XMLStreamReader streamReader = xmlInputFactory.createXMLStreamReader(inputStream);
    final Customer customer = unmarshaller.unmarshal(streamReader, Customer.class).getValue();
    final ObjectMapper objectMapper = new ObjectMapper();
    final String jsonEvent = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(customer);
    System.out.println(jsonEvent);

    //JSON to XML
    Marshaller marshaller = jaxbContext.createMarshaller();
    marshaller.setProperty(Marshaller.JAXB_FRAGMENT, Boolean.TRUE);
    marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
    marshaller.marshal(customer, System.out);
  }
}

When I convert the XML->JSON then I get the following output: (If you observe the fields name and age are not taken as the dedicated fields from Customer class rather its taken as random fields and written within the others)

{
  "name" : "",
  "age" : "",
  "others" : {
    "google:main" : [ {
      "google:sub" : "bye"
    } ],
    "name" : "BATMAN",
    "age" : "2008"
  }
}

I want my output to be something like this: (I want my dedicated fields (name, age) to be mapped first then if there are any unknown fields then map them later within others MAP). Please note that I do not want to get others to tag within my JSON. I want to get the names of the fields only for the dedicated fields.

{
  "name": "BATMAN",
  "age": 2008,
  "google:main": {
    "google:sub": "bye"
  }
}

Following is the XML that I would like to get during the marshaling. Also, please note I am using @XmlPath(".") so that I do not get the others tag within my XML during marshaling.

<Customer>
    <name>BATMAN</name>
    <age>2008</age>
    <google:main>>
        <google:sub>bye</google:sub>
    </google:main>
</Customer>

Following is my TestAdapter class which will be used for the Userdefined fields:

class TestAdapter extends XmlAdapter<Wrapper, Map<String, Object>> {

  @Override
  public Map<String, Object> unmarshal(Wrapper value) throws Exception {
    System.out.println("INSIDE UNMARSHALLING METHOD TEST");
    final Map<String, Object> others = new HashMap<>();

    for (Object obj : value.getElements()) {
      final Element element = (Element) obj;
      final NodeList children = element.getChildNodes();

      //Check if its direct String value field or complex
      if (children.getLength() == 1) {
        others.put(element.getNodeName(), element.getTextContent());
      } else {
        List<Object> child = new ArrayList<>();
        for (int i = 0; i < children.getLength(); i++) {
          final Node n = children.item(i);
          if (n.getNodeType() == Node.ELEMENT_NODE) {
            Wrapper wrapper = new Wrapper();
            List childElements = new ArrayList();
            childElements.add(n);
            wrapper.elements = childElements;
            child.add(unmarshal(wrapper));
          }
        }
        others.put(element.getNodeName(), child);
      }
    }

    return others;
  }

  @Override
  public Wrapper marshal(Map<String, Object> v) throws Exception {
    Wrapper wrapper = new Wrapper();
    List elements = new ArrayList();
    for (Map.Entry<String, Object> property : v.entrySet()) {
      if (property.getValue() instanceof Map) {
        elements.add(new JAXBElement<Wrapper>(new QName(property.getKey()), Wrapper.class, marshal((Map) property.getValue())));
      } else {
        elements.add(new JAXBElement<String>(new QName(property.getKey()), String.class, property.getValue().toString()));
      }
    }
    wrapper.elements = elements;
    return wrapper;
  }
}

@Getter
class Wrapper {

  @XmlAnyElement
  List elements;
}

Complete code can be found here:

JAXB Moxy XMLPath issue.java.zip

[rfelcman]

Hello,

I see two issues in Your input XML and application design.

• map any XML content from e.g.

<google:main>
     <google:sub>bye</google:sub>
</google:main>

to private Map<String, Object> others; it looks problematic to me, because there must be logic to decide which element holding text value and which is container for the other elements (tree vs. flat list) Or is there just simple structure like?

<google:main>
     <google:sub1>bye01</google:sub1>
     <google:sub2>bye02</google:sub2>
     <google:sub3>bye03</google:sub3>
     <google:sub4>bye04</google:sub4>
</google:main>

• How to isolate content other than <name>BATMAN</name> <age>2008</age>. Maybe some white/black list in adapter? Is it possible to slightly redesign input document to wrap "XMLAny" content into "expected" element? I mean something like

<other>
  <google:main>
     <google:sub>bye</google:sub>
  </google:main>
</other>

As a possible workaround should be used https://www.eclipse.org/eclipselink/api/2.7/org/eclipse/persistence/oxm/annotations/XmlReadTransformer.html https://www.eclipse.org/eclipselink/api/2.7/org/eclipse/persistence/oxm/annotations/XmlWriteTransformer.html maybe with https://stackoverflow.com/questions/1068636/exclude-certain-elements-from-selection-in-xpath

[Aravinda93]

@rfelcman Thanks a lot for your response. I am trying to debug this issue for the last 1 week and the Moxy code for the last 2-3 days but finding no solution.

1. With regards to the problem with XML content: I have 2 dedicated fields (name and age) rest are provided by users dynamically with namespaces so I have no way to predict them beforehand that's the reason I was using the Map<String, Object> so first Moxy can read all the known elements such name and age then whatever left will be considered as user-defined values and store it within Map<String, Object> others.

I have a brief understanding of the Jackson where they use @JsonAnySetter and @JsonAnyGetter to map all unknown key-value pair to Map<String,Object> so I was trying out such a method.

If I need to build logic to populate Mao<String, Object> others with non-matching entries from XML then how can I do it? Do I need to add another XMLAdapter or something? Can you please share some light on that?

2. With regards to changing the XML format a bit I wish I had the option to tweak the XML then I would have wrapped all unknown elements into a field like you mentioned other so I can read easily. But the real XML is coming from a standard application and follows some standard format due to which changing the structure of the XML is out of my control. As they are user-defined they can be anything and follow any order. I cannot exclude them either as they are an important part of XML and would like to have them in the output JSON.

I am using the @XmlPath(".") for the same reason so that it does not write the others node in the XML but the child contents are written to XML using my Custom XMLAdapter.

Workarounds that I have tried: I have a few workarounds that I have tried/thought but they are also not workings but wanted to bring to your notice so you can provide some suggestion:

Workaround-1 Is there any other way to avoid the node others during marshaling? As of now I am using @XmlPath(".") specifically to avoid writing the node others in my XML but retain its content. If there is any other way to achieve this then we can remove the @XmlPath(".") from POJO. This can fix my problem during the unmarshalling because removing this annotation fixes everything but I need it during the marshaling.

I searched a lot and finally, I ended with the @XmlPath("."). Is there a way to achieve the same with some form of custom XMLAdapter?

Workaround-2 I tried to create one more field private Object any in my POJO with the annotation @XmlAnyElement(lax=true) so it can take all the unknown nodes and values from XML. Also, I retained the Map<String, Object> others as I need it during marshaling. Then used the setter method of the private Object any to populate the Map<String, Object> others.

Also, I made sure that I use the name, age and others during the marshaling and name, age and any fields during the unmarshalling using the @XmlNamedObjectGraphs. I used the following during unmarshalling unmarshaller.setProperty(UnmarshallerProperties.OBJECT_GRAPH, "full"); but it is also not working and throwing me the error during the unmarshalling:

Internal Exception: java.lang.NullPointerException: Cannot invoke "org.eclipse.persistence.internal.oxm.mappings.UnmarshalKeepAsElementPolicy.isKeepUnknownAsElement()" because "keepAsElementPolicy" is null]

However, the marshaling works fine. If I can make this somehow work then I guess I have a workaround for this issue. Can you please suggest what's causing this issue and is there a fix for this? Following is my Customer.class after modifications:

@XmlNamedObjectGraphs({
    @XmlNamedObjectGraph(
        name = "full",
        attributeNodes = {
            @XmlNamedAttributeNode("name"),
            @XmlNamedAttributeNode("age"),
        }
    )
})
@XmlRootElement(name = "Customer")
@XmlType(name = "Customer", propOrder = {"name", "age", "others","any"})
@XmlAccessorType(XmlAccessType.FIELD)
@Getter
@Setter
@NoArgsConstructor
public class Customer {
  private String name;
  private String age;

  @XmlPath(".")
  @XmlJavaTypeAdapter(TestAdapter.class)
  @JsonSerialize(using = CustomExtensionsSerializer.class)
  private Map<String, Object> others = new HashMap<>();

  @XmlTransient
  private Object any;

  @XmlAnyElement(lax = true)
  public void setAny(Object obj) {
    Element element = (Element) obj;
    others.put(element.getNodeName(), element.getTextContent());
  }

  @JsonAnySetter
  public void setOthers(String key, Object value) {
    others.put(key, value);
  }
}

Workaround-3 Is there a way to force the ordering during the unmarshalling. I have already added the proporder to my Customer.class but I am not sure if that's being used during the unmarshalling. If I can force the order to JAXB/Moxy then I can make sure that name and age are unmarshalled first so that they will have the respective values from XML then whatever is unable to match will be populated within the Map<String,Object>. I am not sure if there is a way but just a thought.

My XML would look something like this (I am really sorry I missed out on the namespace part. The google thing is coming from the namespace prefix)

<Customer xmlns:google="https://google.com/test">
    <name>BATMAN</name>
    <age>2008</age>
    <google:main>
        <google:sub>bye</google:sub>
    </google:main>
</Customer>

Thanks a lot for the links I am looking into them and trying to build a workaround approach for this particular issue. I will update you here if something works out. Meanwhile, I wanted to answer your questions so you have some background of the issue and suggest if there is something better or a quick fix.

I referred to this blog but they are using only one field Map<String,Object>. All I need to do is add an @XmlElement along with this Map<String,Object>: http://blog.bdoughan.com/2013/06/moxys-xmlvariablenode-using-maps-key-as.html

After a lot of trying and searching I gathered few similar issues on the Stackoverflow which are quite old but have no responses on them:

1. https://stackoverflow.com/questions/27110927/unmarshalling-complex-xml-using-jaxb-moxy/67775861
2. https://stackoverflow.com/questions/33694824/jaxb-moxy-can-i-use-xmlpaths-and-xmlelements-to-model-a-choice-if-i-have-an

I hope I have answered all your questions and provided the things that I have been trying. It would be of great help if you could provide me some suggestions/workaround for this issue. If you need any clarification then please let me know I would be more than happy to provide the same.

[Aravinda93]

@rfelcman Thanks for your response. I was finally able to find a workaround for this issue using the Jackson approach during the Unmarshalling.

However, I am still curious if this is a bug from JAXB/Moxy during the Unmarshalling or this is expected behavior when @XmlPath(".") is combined with the @XmlElement

Basically following is the problem: If there is a field Map<String, Object> which has been annotated with @XmlPath(".") and a normal @XmlElement then during the unmarshalling all the values are assigned to the Map<String, Object> as it has @XmlPath(".") and it would ignore the other @XmlElement completely.

As per my understanding during the Unmarshalling it should first look for all @XmlElement and then finally the @XmlPath("."). Please confirm if this is an issue or expected behavior as it can be helpful to somebody in the future.

Following is the example class which can be used:

@XmlRootElement(name = "Customer")
@XmlType(name = "Customer", propOrder = {"name", "age", "others"})
@XmlAccessorType(XmlAccessType.FIELD)
public class Customer {
  private String name;
  private String age;

  @XmlPath(".")
  @XmlJavaTypeAdapter(TestAdapter.class)
  private Map<String, Object> others;
  //Getter, Setter and other constructors
}

In this class during the unmarshalling even if the name and age are provided then they are ignored and set to empty string. All values are populated within the Map others.

@rfelcman Looking forward to your response and some insights into this. Thanks in advance for your response.

References:

1. https://stackoverflow.com/q/67648941/7584240
2. https://stackoverflow.com/q/27110927/7584240
3. https://stackoverflow.com/q/33694824/7584240
4. http://blog.bdoughan.com/2013/06/moxys-xmlvariablenode-using-maps-key-as.html