mlemgroup
Improve robustness of API interaction
Introduction
When the project first started interaction with the API was over WebSocket, with the responses being manually parsed using some help from SwiftyJSON. Later, the project was transitioned over to the HTTP based API, while still manually building requests and parsing.
After discussion it was agreed to build an APIClient and model all our requests and responses to fully mirror the API offered by Lemmy. This has greatly improved the situation as building and performing requests has become much simpler, and we get back a nicely modelled response ready for use by the application.
With that said, I have concerns over our current approach as it exposes us to a lot of unnecessary risk when changes are made to the Lemmy API. As this is still in very active development, these changes are frequent and even in the short time working on Mlem there have been multiple occurrences where changes have been necessary in order to keep the application functional.
The reason we are exposed to so much risk is because we have chosen to mirror the API directly. At the time this was discussed I raised concerns that we were over-modelling, as in many cases we would not need the values, but mapping them means our models will fail if a value (used or not) is changed.
Requirements
In order to improve this situation and ensure we can aim to make the application as robust as possible I would like to begin working on the following:
- Scoped Models - these would be tied to the full API model they relate to but would be cut down to only include the values which specific areas of the application require. This means if a breaking change is made on the API, it would not impact areas which do not require it.
- Integration Tests - A set of tests, likely done in a separate process not directly built within the project which would use our full API models and requests to confirm compatibility with new Lemmy releases. This would allow us to quickly verify as RCs are made available how any changes would impact the application and allow us to make pre-emptive changes where necessary.
If the team support this approach I'd be happy to start working in this area going forward, I spent some time looking into how we could produce a scoped model that would provide some compiler safety built in while working on hiding downvotes when they were disabled.
It should be possible to approach this work in an iterative manner, so if going ahead I'd break it down into various tasks to begin working through.
Originally I thought that the Lemmy API would not change that much, but in hindsight, I was too optimistic.
I think that we should indeed find a way to make a more "battle-hardened" parser against the Lemmy API, one that can easily switch parsing strategies based on the recognised version from the server, to make sure we can always fetch the data we would want, or have feature-flows in place for if a server version is too low to support such a thing.
yeah, we get back the version in the /site call so we could look at allowing certain interactions or features based on that, that'd be nice.
I think as a phase one stripping back what fields each area depends on to the minimum would help mitigate many breakages.
That was the bit I was going to start with, how has the rest of the mapping been going @ShadowJonathan? We wouldn't need them all in place for this, but as more come in there'd be a few changes to make them start working the way I have in my head
Slating this for 1.2 because "API Middleware" in 1.1 should give us a nice platform to address this from
