[EPIC] SAVE THIS CONTENT TO GIANT

[hoyla]

[hoyla]

Phil: I’ve started working on this - initially creating an endpoint in giant to store the remote ingest job in postgres, and a new worker to poll the postgres table for jobs and send to the media download service. My next plan is to start editing the media download service to be less tied to the transcription service

[hoyla]

Phil:

To avoid a monster PR, this part of add to giant can be done in 3 stages:

Add a new endpoint to giant which takes a URL and various workspace/collection settings and writes the data to a new ‘RemoteIngest’ table in neo4j/postgres. Have a job that reads this table and pushes messages to SQS accordingly Save to giant part 1 - create new remote ingest job

Update transcription service media download service to deal with messages coming from giant, post results back to giant

Update giant to process finished download jobs and ingest the resulting files