Generating Java POJO's from the schema

[Celludriel]

Hey,

I have been trying to generate Java POJO's from the 1.2 schema since entur's solution a) doesn't work on windows, b) isn't maintained for version 1.2 nor does the old version work anymore either.

I've progressed pretty far but now I'm at a roadblock. I'm not sure this is because of how I'm generating the POJO's with XJC or that this is actual a schema error. There are several elements twice defined in the xsd's that are giving conflicts.

	Line 12: org.xml.sax.SAXParseException: 'topographicPlacesInFrame_RelStructure' is already defined
	Line 151: org.xml.sax.SAXParseException: 'topographicPlaces_RelStructure' is already defined
	Line 290: org.xml.sax.SAXParseException: 'groupsOfPlacesInFrame_RelStructure' is already defined
	Line 429: org.xml.sax.SAXParseException: 'TopographicPlace' is already defined
	Line 572: org.xml.sax.SAXParseException: 'TopographicPlace_VersionStructure' is already defined
	Line 711: org.xml.sax.SAXParseException: 'TopographicDescriptiveGroup' is already defined
	Line 850: org.xml.sax.SAXParseException: 'TopographicRelationGroup' is already defined
	Line 989: org.xml.sax.SAXParseException: 'topographicPlaceDescriptors_RelStructure' is already defined
	Line 1128: org.xml.sax.SAXParseException: 'TopographicPlaceDescriptor_VersionedChildStructure' is already defined
	Line 1267: org.xml.sax.SAXParseException: 'GroupOfPlaces' is already defined
	Line 1410: org.xml.sax.SAXParseException: 'GroupOfPlaces_VersionStructure' is already defined
	Line 1549: org.xml.sax.SAXParseException: 'GroupOfPlacesGroup' is already defined
	Line 1688: org.xml.sax.SAXParseException: 'TopographicPlaceView' is already defined
	Line 1831: org.xml.sax.SAXParseException: 'TopographicPlace_DerivedViewStructure' is already defined

Can it be these need to be moved to one seperate XSD and defined only once instead the same name several times over several xsd's in the same namespace ?

[skinkie]

Never mind. I see what you have found.

[skinkie]

My tentative conclusion is that country_version is a complete duplicate of netex_topographicPlace_version, but country_support is in fact different.

[Aurige]

The country_version file is a dead file (leftover from some old intermediate work) and is not used anywhere in the XSD ... it has to be deleted ! (it's not in the XML-Spy project file neither, explaining why it was not detected earlier). The country_support one is used and needs to be kept As a generic good practice, people should start from the root file (i.e. NeTEx_publication.xsd) and only load included files and not take all the files ... which of course does not prevent us from cleaning our repository

[Celludriel]

Thanks for verifying this. The XJC process of jaxb2 in java indeed takes all files it scans in the src/main/xsd folder and processes them. It doesn't just read the includes and follow a tree. Now manually following the entire import tree and deleting files that are not needed is a very challenging job if you are new to the NeTEx format. So it's really really appreciated if the repository becomes cleaner.

as an addendum to that: there is also a file SupplementProduct.emf, that probably doesn't need to be there ?

[skinkie]

Correct, there was also a test.xsd that should be removed.

[Celludriel]

not sure this is dead code either but I found

<xsd:include schemaLocation="netex_filter_network.xsd"/>

in

netex_all_filters.xsd

this results in

[ERROR] null [-1,-1] com.sun.istack.SAXParseException2: IOException thrown when processing "file:/C:/workspaces/dienstregelingen-api/src/main/xsd/netex_service/netex_filter_network.xsd". Exception: java.io.FileNotFoundException: C:\workspaces\dienstregelingen-api\src\main\xsd\netex_service\netex_filter_network.xsd (The system cannot find the file specified).

And indeed that file does not exist in the repository

[Aurige]

netex_all_filters.xsd is also a leftover file that should be removed... good catch

[skinkie]

@Celludriel if you will be generating NeTEx I would not use the XJC classes, the Entur code is much better. Since it provides some very nice decorators. For parsing you could use it though.

[Celludriel]

@Celludriel if you will be generating NeTEx I would not use the XJC classes, the Entur code is much better. Since it provides some very nice decorators. For parsing you could use it though.

Well I would but our use case in the company atm and this is open for change:

Internally we want to expose the NeTEx data as json, externally to the outer world we want to expose it as XML.

So I can use the entur dependency for the XML no problem, but then I can't decorate it as JSON unless maybe I subclass the generated jaxb2 classes , so yeah I have a bit of work ahead of me. However any help I can give to clean up the NeTEx I gladly provide.

[skinkie]

Is there a reason you want to do this with Java. Python Dataclasses would basically give you the different serialisations for free.

[Celludriel]

Resources, we are a Java development department , for the public bus transport of Belgium. This is our first venture into NeTEx and one of the first steps was to create a java data model from the XSD's , hence why I ended up posting here. We don't have any available python developers available to do what you suggest.

[skinkie]

@Celludriel I notice you work at De Lijn. De Lijn already has a NeTEx export. What is the reason you reimplement it (and not directly pull it from Hastus)?

[Celludriel]

Long story short since we are getting a little of topic from the issue here :)

We use two modules of hastus Hastus and Hastus Daily, Hastus has the long term planning of our vehicles and manpower, Hastus Daily is the short term more or less next 10 to 20 days. We are designing a Timetable API that will get the data out of Hastus by means of OIG exports and we then work those exports into a seperate database. Then we use that database to drive the Timetable API to provide those Timetables in NeTEx format to our users. Those users are internal in the company other departments (as JSON) but also to the outside world (as XML).

I do not know a lot about the why because that is more a management decision I'm just a developer executing what is asked of me :)

I didn't even knew we had that NeTEx export you talk about, I was told this would be a first implementation ....

[skinkie]

On De Lijn's SFTP server GTFS and NeTEx exports are present, that makes me curious why you would need to create a new one. In addition there are several OIG's available that directly pull NeTEx from Hastus, this is mine targeted to the Dutch Profile. For the Belgium Profile the Nordic export would be a great starting point. https://github.com/skinkie/hastus

[skinkie]

@Celludriel Can you check if xjc behaves now?

[Celludriel]

@skinkie Not yet working out of the box

[ERROR] file:/C:/workspaces/dienstregelingen-api/src/main/xsd/netex_framework/netex_genericFramework/netex_grouping_support.xsd [101,70]
org.xml.sax.SAXParseException: A class/interface with the same name "be.delijn.base.application.resources.dienstregelingen.representation.GroupOfEntitiesRefStructure" is already in use. Use a class customization to resolve this conflict.
    at com.sun.tools.xjc.util.CodeModelClassFactory.createClass (CodeModelClassFactory.java:92)
    at com.sun.tools.xjc.util.CodeModelClassFactory.createClass (CodeModelClassFactory.java:52)
    at com.sun.tools.xjc.generator.bean.ImplStructureStrategy$1.createClasses (ImplStructureStrategy.java:52)
    at com.sun.tools.xjc.generator.bean.BeanGenerator.generateClassDef (BeanGenerator.java:416)
    at com.sun.tools.xjc.generator.bean.BeanGenerator.getClazz (BeanGenerator.java:448)
    at com.sun.tools.xjc.generator.bean.BeanGenerator.<init> (BeanGenerator.java:169)
    at com.sun.tools.xjc.generator.bean.BeanGenerator.generate (BeanGenerator.java:141)
    at com.sun.tools.xjc.model.Model.generateCode (Model.java:258)
    at com.sun.tools.xjc.Driver.run (Driver.java:354)
    at org.codehaus.mojo.jaxb2.javageneration.AbstractJavaGeneratorMojo.performExecution (AbstractJavaGeneratorMojo.java:475)
    at org.codehaus.mojo.jaxb2.AbstractJaxbMojo.execute (AbstractJaxbMojo.java:337)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR] file:/C:/workspaces/dienstregelingen-api/src/main/xsd/netex_framework/netex_genericFramework/netex_grouping_support.xsd [87,71]
org.xml.sax.SAXParseException: (Relevant to above error) another "GroupOfEntitiesRefStructure" is generated from here.

This could be fixed with xjb bindings if I'm not mistaken. However there are quite a few of these around.

full output: output.txt

[ue71603]

If anything can be generalized to the wiki, it would be cool: https://github.com/NeTEx-CEN/NeTEx/wiki/Building-code-from-NeTEx-XSD

[Celludriel]

I haven't been able to finish it succesfully yet. They put me on other tasks at work and shelving NeTEx for now until it becomes relevant again in our project. Wish I had better news, I'll try during my downtimes at work if I can get it running correctly but no promises

[ue71603]

@Celludriel thanks for the update. We will start on the wiki page.

[laurentg]

FYI we (ie Mecatran) used xjc successfully to generate java binding classes, although using some hacks and an xjc (xjb) binding configuration.

[skinkie]

@laurentg the problem with xjc is that it does not know what is doing, it just generates everything.