enketo
Audio recording
Explore feasibility.
https://developers.google.com/web/fundamentals/media/recording-audio/
Some details:
- As a user I'd like to be able to record audio and submit them as part of the form without leaving the form page (currently requires recording first and uploading in the EE form)
- As an offline user I'd like to be able to do this without connectivity
- As a survey administrator, I'd like to record entire surveys (or sections of a survey), not only single questions. The goal is to have a parallel audio file that would allow us to compare actual responses with how they were coded by the enumerator.
Are there technical limitations that would prevent large audio files being attached, especially with offline file storage limits? I found libraries like this one that compress audio to mp3, which would lower the file size obviously.
The third issue just does not excite me at all... I expect it to be a big headache. It could be something for a fork perhaps, or for the future Enketo Governance to decide upon.
Some things to look into:
- user permissions
- switching tabs in the browser
- switching apps
- device goes to sleep or other inactivity
- going back to webpage after any of these previous events
- managing file size
- submission
Alright, maybe they can be done in a separate app that integrates/wraps around EE.
The feasibility of items 1 and 2 would be good to know though, if they work they would go part of the way of making item 3 work as well, at least for some users.
Item 1: Definitely doable on a technical level. There are WebAPIs for getting access to audio devices, and accessing data streams from inside the webapp.
Item 2: Using Storage APIs like IndexedDB, it should be possible to store the audio recordings until the app is ready to upload/has uploaded successfully. Note that each browser has different storage limits for each webapp , but they are usually on the order of 5-10 MB. With audio data, this limit might be problematic. Considering we will only be recording voice, we can really cut the size of the audio by recording only mono with lowest bitrate. (45kbps for ogg vorbis (is it lower for mono?)). Back of the envelope calculation gives us approximately 15 minutes of audio fitting in 5MB limit.
Item 3: Perhaps we can upload portions of the recording, during the survey, and then reconstruct them server-side. Perhaps we can detect when the user is not speaking, and only record when a voice is detected. We can tag audio fragments somehow (with clock-time? With currently viewed question/form?) so that we can reconstruct a real-time audio track server-side.
Note that each browser has different storage limits for each webapp , but they are usually on the order of 5-10 MB. With audio data, this limit might be problematic
Enketo already allows attachments for audio, video, or other file formats. So the limit issue would be the same. I was under the impression that offline storage with EE allowed for much larger attachments per survey instance / for all unsubmitted offline instances.
@MartijnR is the browser limit for offline storage documented somewhere?
Yes, we'd use the storage solution of the app, so indexedDb for Enketo Express. Indeed for indexedDb the storage is much larger, and depends on the amount of free storage available on the device. https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API/Browser_storage_limits_and_eviction_criteria#Storage_limits
We'd have to try to reduce the size, and implement/test user permissions to ask for more storage or at least manage the issue somehow or report to the user. We may never have gotten around to doing that in the current Enketo Express. Or if it has been implemented, I have never tested it due to lack of urgency.
I split this issue. The third point (audit) can now be tracked here: https://github.com/enketo/enketo-core/issues/630
I suggest having a look at the file picker and draw widget for inspiration. And of course the generic widget documentation (link prone to change in near future).
Note that the file-manager module in Enketo Core is overridden in its entirety in Enketo Express (it is just mocking to allow widget development with files in Enketo Core).
It would also be very good to check what the audio input does in ODK Collect.
Current state of things:
- Eric doesn't have time for KoBo anymore but did create this HTML5 audio recording proof-of-concept with offline storage: https://github.com/kobotoolbox/html5-audio-recording-demo
- Per-question audio recording
- now can be done within ODK Collect: https://forum.getodk.org/t/adding-native-audio-recording-for-audio-type-questions/29347
- can have quality configured through the XLSForm/XForm definition: https://forum.getodk.org/t/form-spec-proposal-configure-audio-quality-for-internal-recording/31094
- Background audio recording
- now has a form spec: https://forum.getodk.org/t/form-spec-proposal-add-background-audio-recording/31889
- and a (beta) implementation in ODK Collect: https://forum.getodk.org/t/odk-collect-v1-30-beta-background-recording/32411
