Better error handling

[ammar92]

Is your feature request related to a problem? Please describe. The input validation seems to be inconsistent or lacking in some areas.

Describe the solution you'd like More input validation and generic errors such as OOIs not found or API errors should be captured in a more generic way.

Describe alternatives you've considered There are a few methods to deal with this:

• Capture specific errors in a custom middleware
• Decorators to act on specific errors
• Make use of the generic error handling Django provides
• Subclass (related) exceptions from Django's builtin exceptions (this goes well with the option above)

[originalsouth]

Also in Octopoes there are many exceptions we don't properly catch and route. I suggest we:

1. Map out the exception hierarchy,
2. Catch and route the exceptions to the desired location (depending on the type),
3. At the appropriate point report them to the user with a useful message (and optional trace back),
4. Test the exceptions through the integration tests infrastructure.

[originalsouth]

#2721 as an example.

[ammar92]

Chain of API calls

We need to tackle these kind of situations as well:

sequenceDiagram
    actor User
    User ->> Rocky: GET "/entity?pk=Hostname|ifconfig.me"
    Rocky ->> Octopoes: Request object "Hostname|ifconfig.me"
    Octopoes ->> XTDB: Give me entity with attribute "Hostname|ifconfig.me"
    XTDB -->> Octopoes: 404: Entity not found
    Octopoes ->> Rocky: 500: Internal Server Error
    Rocky ->> User: ERROR

For example, some frontend views may display crashes or 500 errors due to failed external requests. In this case to the call Octopoes API fails, because the inner exception (request from Octopoes API to XTDB API) fails. However, it's important to note that most of these inner requests, particularly those resulting in status codes 400 to 500, are usually client or user errors, and can be managed. So in this scenario, you'd expect the Octopoes API handling the response correctly, and in turn Rocky to respond on that appropriately.

sequenceDiagram
    actor User
    User ->> Rocky: GET "/entity?pk=Hostname|ifconfig.me"
    Rocky ->> Octopoes: Request object "Hostname|ifconfig.me"
    Octopoes ->> XTDB: Give me entity with attribute "Hostname|ifconfig.me"
    XTDB -->> Octopoes: 404: Entity not found
    Octopoes ->> Rocky: 404: Object not found
    Rocky ->> User: Object "ifconfig.me" not found!

Exception types on API calls

We need to clearly specify on what type and direction the error occurs. E.g. is it a request error (connection error, timeouts or anything that relates to the request or not reaching the external resource) or a response error (the server responded, but with an error). This clarity helps in understanding and handling errors effectively, but also produce nicer error messages in the frontend and logs.