Atomizer - recognize file types and convert to JSON-AD

[joepio]

Atomizing is about turning non-atomic data into atomic data. This (often) means converting it to JSON-AD, and then sending it / publishing it to an Atomic Server Importer #390.

Ideally, we'd have one application (the Atomizer) that can:

• run as a CLI. Pipe to your locally running Atomic Server to have a highly performant importer.
• run on the server. Upload a file / link to a file and the server will automatically perform conversions / atomizations. Or atomize already uploaded files automagically.
• run in the browser. Let the JS client perform imports. Highly flexible, can even ask user for extra input when needed.

That will be able to:

• Recognize files and file types
• Convert them to JSON-AD
• Send them to an Atomic Server

Implementation

I think a sensible technological approach is to write all of this in Rust, as a new Crate inside this repo. If it's rust, we can easily embed it in Atomic Server. Also, we can still (later) compile it to WASM and run it in the browser.

Filetypes / data types to atomize:

• Files. We already have the File class, which describes a file with some size and some filetype. If the Atomizer does not know what to do with a file, it will simply upload it as-is: as a file.
• Markdown. These can be converted to Articles. We still need a proper model for this.
• Image. Similar to Files, but with extra data, such as EXIF location / camera / aperture / ISO, etc.
• CSV. Parse the headers and convert it to a (new?) Class + Properties.
• HTML. #432

Inspiration:

• See https://github.com/tauri-apps/tauri-plugin-log
• @jonassmedegaard had some really cool ideas about this.