Extracting elements from schema files to insert into the API docs

[gfurbush]

The documentation tools that extract elements from the schema files and inserts them into our API docs requires that the element be of type xs:complexType. However, I frequently encounter situations in which the to-be-documented element is a simple xs:element type. Is there anything we can do to capture these as well?

Now it may be that we don’t want to capture all simple elements, so could we add an attribute, such as ‘doc=”true”’?

[kcoleman-marklogic]

I've confirmed that the zip you can download from pubs matches the one produced nightly.

I believe the files in question all live under pubs/raw/apidoc. Some of the .xml files under there are actually schema files and feed the extraction in question.

In broad strokes, whenever there is an entry like the following in the apidoc XML, then the element names and descriptive annotations (xs:documentation schema elements) are extracted and inserted into the final XML/HTML.

<apidoc:usage schema="manage-clusters.xsd" element-name="cluster-query">

There's a lot of properly functioning examples of this in pubs/raw/apidoc/management.xml. You'll find, for example, a manage-clusters.xml file under pubs/raw/apidoc that is manage-clusters.xsd (with a filename prefix change for unknown reasons), and that file defines an xs:complexType named cluster-query. The extraction produces the content you see under the Usage Notes here:

http://docs.marklogic.com/REST/GET/manage/v2

I'll leave it to Gordon to point the way to examples of where it does not work.

[gfurbush]

The problem is occurring with the following endpoints:

http://gordon-2:8002/manage/v2/databases?view=metrics http://gordon-2:8002/manage/v2/hosts?view=metrics http://gordon-2:8002/manage/v2/forests?view=metrics http://gordon-2:8002/manage/v2/servers?view=metrics http://gordon-2:8002/manage/v2/clusters?view=metrics

The documentation attempts to extract from the *-metrics-list elements in the meters.xsd file. For example: xs:complexType name="cluster-metrics-list" .

Only the top-level properties are extracted:

meta metrics-properties metrics-relations related-views

(See http://pubs.marklogic.com:8011/9.0/REST/GET/manage/v2/forests@view=metrics )

And not the child properties. At first I concluded that we need to also extract from simple elements, as stated above. However, now, I'm not so sure that's the problem.

Please investigate and advise if there's something I missed.

[dmcassel]

thanks, this is good information

[dmcassel]

@gfurbush @kcoleman-marklogic , I've found something that tells me why the items at /REST/GET/manage/v2/databases?view=metrics have no notes next to them (still working on the recursing part).

Looking at pubs/raw/apidoc/meters.xml, I see this:

<xs:element name="metrics-properties" type="metrics-properties"/>

It should look like this:

<xs:element name="metrics-properties" type="metrics-properties">
  <xs:annotation>
    <xs:documentation>document metrics-properties here</xs:documentation>
  </xs:annotation>
</xs:element>

That's based on meters.xml as it exists in that zip file, so some of the annotations are just missing. $ grep '<xs:element ' meters.xml | grep '"/>' | wc -l found 773 such cases of elements with no annotations, though I don't know whether they all should have them. (222 xs:elements do not close on the same line, so maybe those do have the annotations.)

More to come.

[dmcassel]

@gfurbush @kcoleman-marklogic and I think I see the problem for nested stuff as well. On /REST/GET/manage/v2?view=query, one of the usage notes items is meta. The relevant entry in manage-clusters.xml looks like this:

<xs:complexType name="meta" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://marklogic.com/manage/clusters" xmlns:cts="http://marklogic.com/cts">
  <xs:annotation>
    <xs:documentation>Properties of the view rather than the resource.</xs:documentation>
  </xs:annotation>
  <xs:sequence>
    <xs:element ref="uri" minOccurs="1" maxOccurs="1"/>
    <xs:element ref="current-time" minOccurs="1" maxOccurs="1"/>
    <xs:element ref="elapsed-time" minOccurs="1" maxOccurs="1"/>
    <xs:element ref="start-min" minOccurs="0" maxOccurs="1"/>
    <xs:element ref="view-error" minOccurs="0" maxOccurs="unbounded"/>
  </xs:sequence>
</xs:complexType>

On /REST/GET/manage/v2/databases?view=metrics, there is also a meta, but its complexType in meters.xml looks like this:

<xs:complexType name="meta">
  <xs:sequence minOccurs="0" maxOccurs="unbounded">
    <xs:any processContents="skip"/>
  </xs:sequence>
</xs:complexType>

This pattern holds across ones that work and ones that don't. The code that does the recursion is looking for $complexType//xs:element[@ref]. As you can see, the complexType in meters.xml has no elements. I've also seen some that have xs:element, but no @ref attribute.

Looks to me like there's a problem with the way meters.xml is generated or written.

[kcoleman-marklogic]

Thanks, Dave. I'm going to defer to Gordon on this as he's much more familiar than I am with these schemas. Maybe he can rope in Jim or Norm to take a look from that side.

[dmcassel]

@kcoleman-marklogic @gfurbush checking in... do we need RunDMC code changes for this one?

[gfurbush]

Last I heard, Jim Fuller was going to tackle this in his schema. I’m not sure if this task got lost in all of the background noise, however.

[kcoleman-marklogic]

Jim checked in some doc related schema changes this week. I wonder if they're germane to this problem. Perhaps the right way to deal with problem atm is to assign it to Jim. :)

Whatcha think, @xquery?