jsbsim
jsbsim copied to clipboard
JSBSim-Team
XML validation
JSBSim provides schemas for XML validation (JSBSim.xsd for flight models, JSBSimScript.xsd for script files and JSBSimSystem.xsd for system files) but they have not been updated for a while so they might reject perfectly valid XML files.
XML files can be tested with xmllint
> xmllint --noout --schema JSBSim.xsd some_path/my_file.xml
It might be interesting to:
- [ ] Update the XML schemas to be in synchronization with the last features (in progress)
- [x] Add a XML validation step to our CI (Travis CI and AppVeyor)
I am interested in contributing to the JSBSim project and figured I would start here. I just wanted to check in first and make sure someone wasn't already working on this issue.
@Fangeye thank you for your interest in JSBSim and welcome ! As far as I know no one has undertaken this topic so your help is very welcome. Would you have any questions, please post them on this issue and we will be very glad to answer them.
As I understand the issue the XML validation files have not been updated as JSBSim has evolved, and so may produce incorrect results when testing an XML file.
So my question is what can be used as an authoritative source for how the XML files should be structured? I am guessing there isn't documentation about the proper structure of the XML files, but if so that makes resolving this straight forward. I am wondering if the XML files in the jsbsim/scripts folder would be sufficient to test the validation files or if I will need to go through the commit history to identify the relevant changes that the validation files need to reflect.
So my question is what can be used as an authoritative source for how the XML files should be structured?
Some XML constructions are documented in our documentation (see the documentation of <switch> from FGSwitch for instance).
I am guessing there isn't documentation about the proper structure of the XML files
Actually the current validation files albeit old contain some kind of documentation of what valid XML files are. Eventually, once updated, the validation files will be the documentation of the proper structure of JSBSim XML files.
I am wondering if the XML files in the jsbsim/scripts folder would be sufficient to test the validation files
Right I suggest that you start using the current XML validation files on the scripts (scripts folder), the aircrafts definition (aircraftfolder), the engines definition (engine folder) and the systems definition (systems folder) files. For these files, I would expect 2 kinds of errors :
- New keywords/options that are not yet recognized by the validation files. In that case, the validation files need to be updated.
- Genuine errors that are reported by the validation files. In that case the script/aircraft/engine/system definition file need to be updated.
Don't forget: 3. The validation file probably assumes the XML sections in some particular order (first the engines, then all systems or similar), which is not what the binary did when I used it. It is, or was, relatively easy to construct configuration files where it was important that the systems and other parts where created in the order they were given in the XML files.
For your information, @Portree-Kid has reported to the FlightGear mailing list that the site https://xmlgrid.net/xml2xsd.html can speed up the process of creating .xsd files from examples of .xml files. The generated file might need some manual clean up but it is reported to be helpful.
Oh thank you @bcoconni and @Portree-Kid! I generated a file of all the validation errors and was about to comb through and see where the discrepancies lie. I will check out the xml2xsd tool and see what it generates. When going through the various XML files I did notice that some of them do not specify an XSD file they should be validated against. (An example of this is the ah1s.xml file in the aircrafts folder.) Should all XML files link to the appropriate XSD file or is this more of a 'nice to have'?
Should all XML files link to the appropriate XSD file or is this more of a 'nice to have'?
My opinion is that we should lead by the example, meaning that our final objective should be that the XML files from the official project should all link to the appropriate XSD file.
From: Bertrand Coconnier [email protected]
Should all XML files link to the appropriate XSD file or is this more of a 'nice to have'?
My opinion is that we should lead by the example, meaning that our final objective should be that the XML files from the official project should all link to the appropriate XSD file.
FWIW, I agree with Bertrand. Last I checked there were XSD files for most types at the jsbsim.org site.
Jon
Yes, but how well do they, the existing XSD files, agree with the actual usage of the XML files for the JSBSim binary?
At the time (but that is 5+ years ago now) I tried to validate my XML files many of my files failed because the order sections wasn't in the order set by the validation (which at that time specified a very specific order) - but the order I had chosen was, at least in some cases, necessary to avoid problems when using the configuration with the JSBSim binary.
From: Anders Gidenstam [email protected]
Yes, but how well do they, the existing XSD files, agree with the actual usage of the XML files for the JSBSim binary?
That’s a good point. The XSD files probably need to be updated.
A bigger question is, after 20 years, is there a better way to present input files? It would be a huge change, but converting the input files to something like Python or Julia be worthwhile discussing? That’s not a near-term thing, but in the future.
Jon
@andgi Unfortunately XSD doesn't support an unordered collection of elements where each element has differing restrictions on number of occurrences. The closest you can get is an unordered collection that has a restriction on the ultimate size of the collection shared by all possible elements.
This is why the XSD file may reject an XML file for having elements out of order when the XML file is perfectly usable. The XSD file uses a sequence tag so it can specify the number of occurrences of each inner element but has the consequence of also enforcing an order on the inner elements.
I found a good discussion about this issue on StackExchange if you want to check it out: https://stackoverflow.com/questions/5089730/is-it-possible-to-restrict-elements-to-a-max-occur-in-a-choice-block
Just to keep everyone up to date, @Zaretto has recently submitted the PR #261 that has updated the XSD files. This partially fulfills our goals however since we still need to make sure that all the XML files are validated by the XSD files.
I have just pushed a set of 3 commits (08e462965cf02407f5a41f107ad628f3dc5810b0, 5ec267199c7a3597d6098bf43e3d59a66f1c89c4, 5e216011aaa980690aba6387620f41e009938d2f) to:
- Add an XML validation stage prior to running anything.
- Update the XML schema for the scripts
JSBSimScripts.xsdto bring it up to date with the current features. - Updated a handful of scripts to pass the new XML validation step (the fixes consisted in removing extra attributes that JSBSim does not use).
The problem that @Fangeye brought up 2 years ago remain valid but I guess we should relax the validation in order to have a validation that is useful (and usable) albeit not perfect. Said in other words, some faulty scripts and aircraft files might pass our XML validation because of the XSD format limitations, but I'd better have a less than perfect validation rather than no validation at all.
AS per https://github.com/JSBSim-Team/jsbsim/issues/337#issuecomment-974842686, the aircraft XML template file aircraft_template.xml is obsolete. There are some tools that generate automatically an XML template file from the XSD schema. I guess we need to consider this option for keeping the template up to date.