api
api copied to clipboard
lichess-org
Lack of documentation of optional fields in response bodies
A large number of API responses have fields in the body which may or may not be present, and the fact that they are not guaranteed to be present is nowhere documented. I attempted a fix at one instance of this in #158, but the problem exists in many more places. The UserExtended schema is a good example of this.
{
"id": "georges",
"username": "Georges",
"online": true,
"perfs": {
"chess960": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"atomic": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"racingKings": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"ultraBullet": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"blitz": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"kingOfTheHill": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"bullet": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"correspondence": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"horde": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"puzzle": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"classical": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"rapid": {
"games": 2945,
"rating": 1609,
"rd": 60,
"prog": -22,
"prov": true
},
"storm": {
"runs": 44,
"score": 61
}
},
"createdAt": 1290415680000,
"disabled": false,
"tosViolation": false,
"profile": {
"country": "EC",
"location": "string",
"bio": "Free bugs!",
"firstName": "Thibault",
"lastName": "Duplessis",
"fideRating": 1500,
"uscfRating": 1500,
"ecfRating": 1500,
"links": "github.com/ornicar\r\ntwitter.com/ornicar"
},
"seenAt": 1522636452014,
"patron": true,
"verified": true,
"playTime": {
"total": 3296897,
"tv": 12134
},
"title": "NM",
"url": "https://lichess.org/@/georges",
"playing": "https://lichess.org/yqfLYJ5E/black",
"completionRate": 97,
"count": {
"all": 9265,
"rated": 7157,
"ai": 531,
"draw": 340,
"drawH": 331,
"loss": 4480,
"lossH": 4207,
"win": 4440,
"winH": 4378,
"bookmark": 71,
"playing": 6,
"import": 66,
"me": 0
},
"streaming": false,
"followable": true,
"following": false,
"blocking": false,
"followsYou": false
}
In this sample from the documentation for the https://lichess.org/api/user/{username} endpoint, nowhere it the docs is it stated that any of these fields may not be present, nor when they are not present. Despite this, a great deal of these fields are optional. Among them:
"followable","following","blocking", and"followsYou"never appear in a response to a unauthenticated request."title"does not always appear, as discussed in #158."prov"only appears whentrue, otherwise it is absent."disabled"and"tosViolation"also only appear whentrue, as with"prov"."online"is always present, unlike"prov","disabled", and"tosViolation"."fideRating","uscfRating", and"ecfRating"do not always appear.
as well as presumably several more examples that I have missed. This issue also appears in other schemas. While it may be argued that some of the fields are obviously optional (such as "fideRating", "uscfRating", and "ecfRating" if a user has not set them), deserializing data from the API still requires an unwieldy amount of reverse-engineering to determine which fields in response bodies may be absent and how to interpret their absence.
deserializing data from the API still requires an unwieldy amount of reverse-engineering to determine which fields in response bodies may be absent and how to interpret their absence.
Generally, I'd recommend just handling both cases. I don't think in any of those cases it's possible that the field can be either null or not present.
But ofc it would be nice to document this accurately.
deserializing data from the API still requires an unwieldy amount of reverse-engineering to determine which fields in response bodies may be absent and how to interpret their absence.
Generally, I'd recommend just handling both cases. I don't think in any of those cases it's possible that the field can be either
nullor not present.But ofc it would be nice to document this accurately.
Sorry, I only just realized that I was using null and "not present" in a somewhat confounding manner. The issue I'm running into is not with fields being null, rather it's with fields not being present. Handling both cases in all instances is possible, but I think that in instances where the API's specification offers a stronger guarantee about the data received, we ought to be documenting it. Even if we do ask the user to handle both cases, I think that at a minimum we need to document how to interpret absent fields. For example, if it is a rule across the API that absent bool fields have a default value of false, that needs to be documented.
In fact, doing a bit more reading shows that, at least in the case of the UserExtended schema, a value is present if and only if it is not null. The relevant bit of code in the API is here. All fields are added to create a JSON object, and then all null fields are removed using .noNull. This then becomes a matter of figuring out which fields are allowed to be null, since those are the only ones which may not be present.
Side question. I'm confused about this:
"followable", "following", "blocking", and "followsYou" never appear in a response to a unauthenticated request.
What is an authenticated request? I tried using an API token for a bot with the web:login and bot:play scopes, but that didn't result in these fields being present in the returned /api/user/{username} JSON.
