Refactor: separation of API from storage

[dlorenc]

Right now our API layer is tightly coupled to the storage layer. The API types directly turn into storage types, even though there is tons of validation and canonicalization first. It makes it hard to understand exactly what gets stored in the log for a given type, and makes the code harder to read and test in general.

I think we can separate these cleanly.

[loosebazooka]

Here's a rundown of the types

Types

rekord

field	api	storage
data	*	*
data.content	*
data.hash	*	*
data.url	*	*
signature	*	*
signature.content	*	*
signature.format	*	*
signature.url	*
signature.publicKey	*	*
signature.publicKey.url	*
signature.publicKey.content	*	*
extraData	*	*

intoto

field	api	storage
content	*	*
content.envelope	*
content.hash	*	*
content.hash.algorithm	*	*
content.hash.value	*	*
publicKey	*	*
extraData	*	*

jar

field	api	storage
signature		*
signature.content		*
signature.publicKey		*
signature.publicKey.content		*
archive.hash	*	*
archive.hash.algorithm	*	*
archive.hash.value	*	*
archive.url	*
archive.content	*
extraData	*	*

rfc3161

field	api	storage
tsr	*	*
tsr.content	*	*
extraData	*	*

rpm

field	api	storage
publicKey	*	*
publicKey.url	*	*
publicKey.content	*	*
package	*	*
package.headers		*
package.hash	*	*
package.hash.algorithm	*	*
package.hash.value	*	*
package.url	*
package.content	*
extraData	*	*

Presumably this split would be enough.

I don't really know enough about the openapi stuff to make a call here. But the internal storage data structures maybe don't require all the code generation and whatever? Just handwritten?

Also VERSION appears to be just a internal storage level marker?

Do we need to care about backwards compatibility?

[haydentherapper]

Good to close this? I don't think we'll be making this change as it would require a large rewrite

[loosebazooka]

Yeah sure.