Add RAML hook to skip tests

[drernie]

Not strictly speaking an abao issue, as it involves changing the RAML file. However, I would like to make an explicit change to the documentation indicating that an API is not properly implemented yet, and have abao use that as the mechanism to skip tests. That way, consumers of the API have fair warning that there is no functional test coverage.

[galkin]

Hi, @drernie. Your issue is an exelect feature request. Do you have some ideas or examples, how it should be written in raml? Abao supports 0.8 raml version.

[drernie]

While the simplest option would be a "skip: true", that would be too abad-specific.

I'd recommend something that indicates how well the documentation matches the implementation. We could call it 'status', but that would be rather ambiguous, so I'd go with a "maturity" keyword that could exist at several levels of the API:

maturity: [untestable, unimplemented, incomplete, complete, stable]

• untestable: the docs or implementation are incoherent, and should be skipped
• unimplemented: the docs reflect an API that do not exist yet; the tests could be run, but would fail; since this is expected, it should not cause the overall tests to fail
• incomplete: the API doc is correct, but may not match all of the implementation.
• complete: the API doc is correct, complete, and matches the implementation
• stable: this API is frozen for the current version, and should not be changed. aboa would treat this the same as complete, but other tools may not.

Hope this helps!

[plroebuck]

Sounds like what you want could be done with RAML-1.0 annotations. We don't support 1.0 yet, but you might want to research the topic and see if it would handle your needs. But I don't see how we could implement such with RAML-0.8, as I don't believe you can extend the older version of RAML at the specification level.

[drernie]

Yeah, it sounds like this would be a perfect candidate for 1.0 annotations. Which, alas, means we'll have to wait until 1.0 is supported by abao: #57