api icon indicating copy to clipboard operation
api copied to clipboard

Bootstrap dump creation activity

Open azaroth42 opened this issue 4 years ago • 2 comments

From IIIF/discovery#81, and enabled by IIIF/discovery#83, it would be possible (without any official change to the spec) to create a bootstrap 'dump' file, in the spirit of the Resource Dump from ResourceSync.

When seeing the creation of one of these, a crawler would know that it can retrieve the dump file, process it, and have all of the resources in the state they were at the time of the dump. This would cut down the number of retrieval requests, and optimize for compression over multiple files.

In the situation where the crawler does not understand the profile, format or type of the dump, it would continue and see the individual references in the stream. Thus the dump should come immediately after any entries for resources that are in the dump, and before any further changes. In a 'refresh' scenario, the creation of the dump would be the end of the refresh process.

The challenge is the alignment of filepaths or other identifiers for the packaged bitstreams with the equivalent URIs. This is solved in ResourceSync with a Resource Dump Manifest, with a mandated filename. The members of the TSG on the 2021-01-06 call did not want to expand the scope of the charter to include solving this issue, but consensus was that enabling such experimentation was important to include in 1.0

azaroth42 avatar Jan 06 '21 19:01 azaroth42

Agreed on 2021-02-03 call to defer until 1.1

azaroth42 avatar Feb 04 '21 18:02 azaroth42

Propose transfer to iiif/api, keeping defer

azaroth42 avatar Oct 12 '22 16:10 azaroth42