provide these specs as integration tests

[err0r500]

Hi,

Do you think it would be feasible to have a set of tests running against a "real" server in order to confirm it respects the specs ?

It's not the most convenient/fastest way to test but It would allow new implementations in any language (and a way to propagate issues discovery to any of them).

Just an idea but, as a go developper, it would be really useful in order to contribute to gold.

Best,

[kjetilk]

Yes, I think we should have that as a very high priority after the release of v5.0.0, but it is still subject to resource planning within Inrupt to see if we take that on.

[err0r500]

Ok, actually I realize the main solid README refers to

A test suite for testing and validating Solid implementations.

... so the idea may not be new :)

[csarven]

We need test suites generally for:

• [ ] Solid Server
• [ ] Solid Apps

The "solid specs" (standards, protocols, vocabularies, formats, practices...) reuse existing stuff eg. LDP, WAC/ACL. The Test Suite would refer to corresponding specs eg. LDP Test Suite https://w3c.github.io/ldp-testsuite/

[timbl]

This though I think is the issue about the server tests which test any solid server implementation completely from the outside, so independent of implementation. Then we could immediately point it at NSS and Virtuoso and also Gold to see how far behind it is, and of course any new implementations in new languages. Definitely an important part of the ecosystem.

[kjetilk]

Indeed! I'll have a go at it as soon as I have head above water.

[csarven]

@timbl If I understand you correctly, I've misread this issue. So, what I'm saying while has some overlap with the server/unit? tests, it is about conformance with other specs. Created https://github.com/solid/solid-spec/issues/135 . I hope that sufficiently distinguishes things.