Logical validation of Statements

[canweriotnow]

Specifically, re: Interaction Activities -

From all indications, the current is completely valid:

{
    "version": "1.0.0",
    "id": "d1eec41f-1e93-4ed6-acbf-5c4bd0c24269",
    "actor": {
        "objectType": "Agent",
        "name": "Joe Schmoe",
        "mbox": "mailto:[email protected]"
    },
    "verb": {
        "id": "http://adlnet.gov/expapi/verbs/answered",
        "display": {
            "en-US": "answered"
        }
    },
    "object": {
        "objectType": "Activity",
        "id": "http://www.example.com/activities/001",
        "definition": {
            "name": {
                "en-US": "Example Activity"
            },
            "type": "http://adlnet.gov/expapi/activities/cmi.interaction",
            "interactionType": "true-false",
            "correctResponsesPattern": [
               "true"
            ]
        },
    "result": {
        "response": "false",
        "completion": true,
        "success": true
    }
}

Clearly, the response is incorrect, yet "success" is true. The Statement itself is self-contradictory, yet know allowance is made (AFAICT) for an LRS to reject a Statement for being illogical or self-contradictory.

One option would be to require the LRS to logically (not only structurally) validate Statements.

another would be to require or allow the LRS to record the Statement as having been issued, but to instantly issue a second statement voiding the first for being illogical/self-contradictory/impossible, so that both events are part of the record.

Each has its ups and downs; in the first instance, the offending AP could receive an error response and log the error so that the developer has feedback and can patch the issue; in the latter, no data is lost, even if it's bad data.

It seems obvious that allowing illogical statements is asking for trouble once we get past the "dumb" LRS and try to actually leverage xAPI data for some useful purpose (crazy concept, I know). Of course, it could be left to the consumer of data to do sanity checks on data coming from the LRS, but that's rather tantamount to telling people "use this data, but don't trust it", which seems like a great strategy for promoting adoption.

[garemoko]

The main issue I see with this as a spec change is that the success of the experience is a design decision made by the AP. In order to check the AP's marking, the LRS would need to:

• Understand how the verb used affects the relationship between response, correct response pattern and success.
• Know how to the AP wants to handle partially correct answers.

This is a lot of complexity for a fairly narrow case, so I think this is best handled as a recipe that can define the verbs and ways of handling the complexities of question marking (in fact I'd like to move interactions as a whole to a recipe at some point). Can LRS reject statements based on recipes? I think so; IMHO that could sit under the permissions umbrella.

This could make a very helpful and widely adopted recipe, so don't see this as me dismissing what you're saying; I just don't think it should be a requirement in the spec for all LRS.