tests: run test against actual schema files, not snapshots

[jkowalleck]

Current implementation of java tests run against a snapshot of the schema files, shipped with org.cyclinedx java package. This is undesired and error-prone, as changes to the schema files in this repo are not checked against the schema example files. Therefore, most java checks in pull requests are just bogus results without any meaning. This must change ASAP.

Use cases:

• a PullRequest is opened, that modifies a schema file. tests must run based on these changes, not some snapshots.
• a PullRequest is opened, that modify some test-resources. tests must run based on these changes.

acceptance criteria

• all test run against the actual schema files in this repo, not against any other snapshots
• optional, but preferred: tests run offline, do not load any data from external sources
• all currently running tests still pass

scope

• test resources: https://github.com/CycloneDX/specification/tree/master/tools/src/test/resources
  • tests for JSON files
  • tests for XML files
  • valid-*.{json,xml} should pass schema validation
  • invalid-*.{json,xml} should fail schema validation
• schema files: https://github.com/CycloneDX/specification/tree/master/schema
  • XSD for XML
  • JSON schema for JSON

targets

• [ ] java tests -- open, uses remote snapshot which is about to be changed
  • see https://github.com/CycloneDX/specification/blob/d37a107d78315675ce2969c10c3ea0b6d92f39a6/tools/src/test/java/org/cyclonedx/schema/JsonSchemaVerificationTest.java#L60-L61
  • see https://github.com/CycloneDX/specification/blob/d37a107d78315675ce2969c10c3ea0b6d92f39a6/tools/src/test/java/org/cyclonedx/schema/XmlSchemaVerificationTest.java#L64-L65
• [x] php tests -- done, uses local versions
  • see https://github.com/CycloneDX/specification/blob/d37a107d78315675ce2969c10c3ea0b6d92f39a6/tools/src/test/php/json-schema-functional-tests.php#L15-L16
  • see https://github.com/CycloneDX/specification/blob/d37a107d78315675ce2969c10c3ea0b6d92f39a6/tools/src/test/php/json-schema-lint-tests.php#L5-L6 https://github.com/CycloneDX/specification/blob/d37a107d78315675ce2969c10c3ea0b6d92f39a6/tools/src/test/php/json-schema-lint-tests.php#L16
  • see https://github.com/CycloneDX/specification/blob/d37a107d78315675ce2969c10c3ea0b6d92f39a6/tools/src/test/php/xml-schema-functional-tests.php#L15-L16
• [x] js tests -- done, uses local versions
  • see https://github.com/CycloneDX/specification/blob/200355550bd40316e9c898a7902cfb61b5b89017/tools/src/test/js/json-schema-functional-tests.js#L24-L25
  • see https://github.com/CycloneDX/specification/blob/200355550bd40316e9c898a7902cfb61b5b89017/tools/src/test/js/json-schema-lint-tests.js#L16

---

caused by #255 related: https://cyclonedx.slack.com/archives/G01PMU9ERJA/p1687363757137109

[jkowalleck]

@CycloneDX/core-team FYI

@CycloneDX/java-maintainers can you help out and fix the tests?

[stevespringett]

The latest schema from the 1.5-dev branch has been incorporated into the latest Core Java 7.4.0-SNAPSHOT

[jkowalleck]

re: https://github.com/CycloneDX/specification/issues/256#issuecomment-1601843289 this does not solve the following purpose: a PR modifies the schema files. tests should run on these changes, not some snapshot. here is such a case: #255

[jkowalleck]

@mr-zepol mentioned he might be able to help out here.