prez
prez copied to clipboard
RDFLib
Listing endpoints contain functions that generate things such as curies and prefixes
These functions that generate things should really be on startup.
To ensure subsequent updates to the data are in sync with these generated things, Prez should supply endpoints to update data. These functions can then run on the new data and ensure these generated things are up to date.
An example of this being problematic:
If the Prez server restarts, any requests to an individual object will result in pydantic validation errors with the server responding with status code 500. These errors are the result of unbound prefixes, which are only generated when a request is made to a listing endpoint such as /v/vocab, /v/collection, etc.
Note, this issue applies to all the Prez subsystems.
how should prez know which URIs to generate prefixes for?
what two things need to be kept in sync?
how should prez know which URIs to generate prefixes for?
Does Prez not have all of the info required at startup? What other info does it need besides the info that's already in the profiles?
what two things need to be kept in sync?
Inevitably, the data in Prez will be updated. Example, the periodic loading of new vocab into the system. Prez should provide an endpoint to perform such updates so that its internal systems and data can be kept in sync with the database.
It doesn't know which URIs have namespaces which don't have prefixes defined on startup e.g. https://linked.data.gov.au/datasets/geofabric/fc/catchments
It doesn't at present, it generates a prefix if it can't find one.
May need to look at using a background task to generate the IRIs that don't have a prez curie identifier. Implementation may use a combination like celery + redis. Would be a good direction to go if we need to cater for large scale installations of Prez. Restarts + horizontally scaling the prez app won't require generating IRIs and storing data locally in memory or on disk as it can be stored in a distributed manner in Redis.
the immediate issue here has been resolved - prefixes can be specified where stability is desired. We can open a separate discussion around common caching as/when there's a need for it