record icon indicating copy to clipboard operation
record copied to clipboard

Issues with frequency of iOS stream recordings

Open jamescrosswell opened this issue 2 years ago • 4 comments

Package version 5.0.0-beta.3+1

Describe the bug The app I'm building listens to audio in real time and performs pitch detection on the audio that it receives. On Android and Mac OS that's working fine. However on iOS the data received doesn't match the expected pitch... it's almost like it's slowed down.

The following is a plot of the microphone data received by both Android and iOS devices, while a G3 notes is being played on an instrument. G3 corresponds to about 196.0 Hz. What is being detected on iOS is closer to 177.7 Hz (more like an F3 than a G3)... whereas the Android device measured about 192.3 Hz (close enough for a match).

image

To Reproduce This is obviously a bit tricky to reproduce... I'm mainly posting here to see if there's any obvious reason why the pitch might be off on iOS devices.

Potentially I could try to create an app that played exactly the same recording of a G3 note and also recorded it at the same time... run this on both Android and iOS and then compare the resulting waveforms. Let me know if that's necessary.

Expected behavior I'd expect sample audio data from both Android and iOS to be roughly the same pitch.

Apple Device:

  • Device: iPhone 11
  • OS: iOS 16.5

Android Device:

  • Device: Samsung S22
  • OS: One UI 5.1 (Android 13)

Additional context Add any other context about the problem here.

jamescrosswell avatar Oct 06 '23 03:10 jamescrosswell

Thanks for the report. iOS and macOS platforms share almost the same code in v5, so no, unfortunatly, I don't see any obvious reason at the read of your report.

Are you using the same config on iOS and macOS? And which one is it? I already suppose you are streaming with only one channel. So it could be an internal adjustment of sample rate.

Another difference could be if your macOS is under version 10.15?

Here's the init difference between the platforms https://github.com/llfbandit/record/blob/26edb8fd6fc02ffd397c032d2dcdc81d09b9d32d/record_darwin/darwin/Classes/Recorder.swift#L202-L208

Here are the constraints applied on iOS platform: https://github.com/llfbandit/record/blob/master/record_darwin/ios/Classes/RecorderIOS.swift

llfbandit avatar Oct 06 '23 13:10 llfbandit

Hi @llfbandit... thanks for the quick reply. Looks like a dead end, unfortunately. I'm building everything on macOS 13.6 and my config is the same for both iOS and Android:

static const recorderConfig = RecordConfig(encoder: AudioEncoder.pcm16bits, sampleRate: 44100, numChannels: 1)

I'll see if I can find a way for someone else to easily reproduce the issue.

jamescrosswell avatar Oct 06 '23 23:10 jamescrosswell

Streaming has been redone in record_darwin: ^1.0.1-beta.1. AudioEngine is now used. Maybe it will be better. You will need to add it manually to test it.

llfbandit avatar Nov 03 '23 14:11 llfbandit

Streaming has been redone in record_darwin: ^1.0.1-beta.1. AudioEngine is now used. Maybe it will be better. You will need to add it manually to test it.

Thanks @llfbandit, I have tested it on a couple of other devices since and they don't appear to behave the same. So potentially it's unique to the particular iPhone that I have for testing (which is unfortunate, when I'm building an audio app). I'm building some callibration functionality into a new version of the app so that I can get some extra data, hopefully, from folks who have iPhones and are willing to test the app.

This is a hobby project unfortunately, so I only get to work on it about 1 day / week (working through issues like this is slow going)!

But yes, I'll try out the new record_darwin as well and see if there's any change on the physical phone I have for testing.

jamescrosswell avatar Nov 03 '23 21:11 jamescrosswell

record_darwin 1.1.0 has been released. In this version, AGC and echo cancel are managed. This should be no more an issue I guess. Feel free to re-open if it's not the case.

llfbandit avatar May 21 '24 12:05 llfbandit