Visualization does not work on the new Galaxy devices

[mobilevisuals]

A visualizer can have an internal music player, which means that the app has its own built in player. It can also use the music from external music players( Winamp for instance). Visualization for external players does not work on the new Galaxy devices.

I have played music with external players, while debugging the visualizer apps. The bytes in the updateVisualizerFFT method are always 0 for external players, when a visualizer is created with 0 as an argument.

This results in no visualization, since there is no signal to use for visualization. It still works for internal players. The bytes in the updateVisualizerFFT are not 0 then, so the app has something to use for visualization, even when the visualizer is created with 0 as an argument.

This is caused by an Android code bug in the Digital Signal Processors, which are included in the new Galaxy and some HTC devices.

Several visualizer apps have this warning message:

"NOTE: To Galaxy S/S2/S3/S4, Galaxy Note 1/2, Galaxy Nexus and HTC Rezound owners: The visualizer will sporadically show a flatline with many music players, due to an Android bug related to the built-in DSP in these phones. This is a problem for all music visualizer apps on these phones, as evidenced by user reviews of other apps. Still looking for a work-around, but we'll probably have to wait for a future version of Android to fix it. If you have this problem, I apologize! Please email me and I will refund you."

Almost all visualizer apps on the market have this problem and can’t visualize from external players on the new Galaxy and some HTC devices.

It seems like it is possible to solve this problem, because external visualization works on the Morphyre Music Visualizer. It is right now the only visualizer on the market, where this works.

[h6ah4i]

Hello. I'm developing and publishing "Music Visualizer" app on Google Play store. My app also have the similar issues, so I know some causes of them.

First, visualizer not working issue is not a single issue. There are several reasons that makes visualizer not working.

1. Galaxy devices (except for S4) and HTC Rezound

Sorry, I don't know about this issue on these phones. I have Galaxy S2 (SC-02C, JP model), but visualizer works well on it. But I got some feedback from Galaxy S3 users that my app doesn't work on it.

2. Galaxy S3, S4, Note 3 and derived models (Snapdragon model only)

Snapdragon based Galaxy S4 have visualizer not working issue. My Galaxy S4 (SC-0E, JP model) corresponds to this case. I investigated this issue and found the reason.

It uses hardware acceleration playback mechanism called "Tunnel Player (Tunnel Playback)" when playing music files. If the tunnel player is used, audio capturing process is bypassed. As a result, visualizer does not work at all.

There are several conditions for the tunnel player to be used.

• Snapdragon platform
• "tunnel.decode = true" system property is set
• Audio playback by MediaPlayer class (*1)
• MP3 or AAC playback only
• Only one instance is permitted (can't create multiple tunnel player instances)

So, workarounds are the follows;

• Use Open SL API instead of MediaPlayer class (*2)
• Use OGG format instead of MP3 or AAC
• Put "tunnel.decode = false" in build.prop file (!!! root required)
• Occupy tunnel player instance by dummy silent MP3/AAC file playback. Second MediaPlayer instance will fall back on standard audio playback mechanism. (November 20, 2013 added: I found that MediaPlayer.start() is not required on the dummy instance, only MediaPlayer.prepare() call is needed. sample code is here)

NOTE 1) This issue is already reported on Samsung's developer forum. (But it seems not fixed yet...)

• http://developer.samsung.com/forum/board/thread/view.do?boardName=GeneralB&messageId=238465

NOTE 2) Originally, the tunnel player seems to have been introduced by Code Aurora Forum's developer.

• https://www.codeaurora.org/cgit/quic/la//platform/frameworks/base/tree/media/libstagefright/AwesomePlayer.cpp?id=refs/heads/b2g/ics_strawberry

3. Nexus 4, Nexus 5 and Nexus 7 (2013) (maybe Snapdragon based Nexus only)

Nexus series devices are Google's reference model Android devices. So sometimes they support new features earlier than vendor customized models.

The Google has introduced low-latency audio playback feature since Android 4.1. But it has a side-effect, visualizer doesn't work while music playing by Open SL APIs.

However this issue only occurs on certain Nexus devices. Nexus 10 also supports low-latency playback, but it doesn't have the issue. I think the difference came from base platform difference, Nexus 4 and 7 (2013) are using Qualcomm's Snapdragon SOC, on the other hand Nexus 10 uses Samsung's Exynos SOC. (I don't know that Galaxy Nexus has this issue, but it's interesting because the phone using OMAP SOC.)

Workaround is the follow;

• Use MediaPlayer class based player apps (*1)

ref.) http://source.android.com/devices/latency_design.html

4. Some custom ROMs (Old LiquidSmooth, RootBox and VanirAOSP)

Some custom ROMs have bugs on visualizer implementation. I fixed it on LiquidSmooth, RootBox and VanirAOSP ROM, but I'm afraid that there are other custom ROMs still have this bug.

• https://github.com/LiquidSmooth/android_frameworks_av/pull/1
• https://github.com/LiquidSmooth/android_frameworks_av/pull/2
• https://github.com/Root-Box/frameworks_av/pull/1
• http://vaniraosp.goo.im/#/c/5820/

5. CyanogenMod 10.2 (ADDED on November 18, 2013)

On CyanogenMod 10.2, Visualizer class doesn't work if it created with audio session 0 (= system mixed output).

This issue seems to occur on certain devices only.

[Affected]

• Nexus 7 2013 ("flo")
• Galaxy S Relay G4 ("apexqtmo")

[Not affected]

• Nexus 7 2012 ("grouper")

6. Recent Motorola's phones (Moto X, DROID Mini, ...) (ADDED on November 23, 2013)

Maybe this issue is only occur on JellyBean. After upgrading to KitKat, it seems to be fixed. (A my app user confirmed this, he is using a Moto X phone.)

---

(*1) MediaPlayer class based player apps:

Music Visualizer built-in player, Google Play Music, PlayerPro (without DSP pack), Rocker Player Free, doubleTwist, n7player, Apollo

(*2) Open SL based player apps:

Poweramp, jetAudio, WinAmp, PlayerPro + DSP pack, Rocket Player Premium, GoneMad, Neutron

---

Thanks, Haruki Hasegawa (h6a h4i)

[mobilevisuals]

It seems like there can be no definite solution to this problem until a new software update is released from Samsung. How can we make them aware of this problem and encourage them to solve it as soon as possible?

Music visualization always works on the new iOS devices without any issues. Music visualization should work on all the new Android devices according to the specification, but it does not work on a lot of the new high profile devices, like Samsung's S4. This gives Android a bad reputation and makes people want to buy iPhone instead.

[felixpalmer]

I agree that this is vendor issue. Have you tried contacting Samsung directly and seeing if they have a process for reporting such issues, or even a bug tracker (I know Apple has something like that, for example). I'm not in a position to test myself, as I don't have the device in question.

I'm afraid that I'm not that hopeful here, when a manufacturer ships a device that has a bug in its API, there's only so much you can do. Given how long it takes to get a bug fix from reporting stage to actually being pushed out to devices, sadly I don't think this will be fixed soon. Still worth a try though... :(

[abiemann]

My Samsung Galaxy S4, Model SPH-L720, with Android version 4.3, suffers from this problem. Using MediaPlayer to play an MP3 causes the Visualizer to send a steady stream of 0's in the case of onFftDataCapture(), and a steady stream of -128's in the case of onWaveFormDataCapture()

I've tried one of h6ah4i's suggested work arounds: to play a quiet dummy MP3 before playing the actual music MP3. However, I did not see success. I used the 30second and 60second quiet MP3's found here: http://www.xamuel.com/blank-mp3s/

Maybe those MP3's aren't suitable because there wasn't enough data to enage the tunnel player ?

[h6ah4i]

Hello abiemann. ~~I think the Visualizer issue on SPH-L720 is different from tunnel player's side effect. Because the tunnel player is only used on Snapdragon SOC, SPH-L720 is a Samsung's Exynos SOC model...~~ Oops, SPH-L720 is a Snapdragon model.

I downloaded the 1 min length MP3 from your suggested URL, and tried the workaround with using it. The result was good, it seemed to work on my S4 (SC-04E).

Have you tried Poweramp player already? It's the most easy way to identify the cause is a tunnel player's side effect.

[abiemann]

Hi h6ah4i - I did try PowerAmp and with the Visualizer getting data from the mixer i.e. "audioSession=0" does indeed return real amplitude data instead of 0's.

However, my app uses the MediaPlayer and no amplitude data was returned by the Visualizer API on my S4. On other devices like the Nexus 7, and Nexus 4 there's no problem.

When I attempted the work-around with the quiet MP3, I simply created two MediaPlayer objects: one played the quiet MP3 first (looping was enabled), the second MediaPlayer then played the real music - but again I received no amplitude data from the mixer :-(

[h6ah4i]

@abiemann I made a sample code that shows how to avoid tunnel player. Can you try this?

https://github.com/h6ah4i/tunnel_player_workaround

[mobilevisuals]

I want to test your sample app, to see if it fixes the problem. I only need the APK file to test, I don't need the source and the other files. Can you upload the APK file? abiemann, have you tested the file?

I think that this bug is devastating for the music visualizer business. There is no point in making music visualizers if they can't work on the most popular devices. I am not going to make any more music visualizers until this is fixed. It simply results in too many refunds and complaints from users.

This bug also makes people want to buy iPhone instead, since music visualizing works without problems there.

[mobilevisuals]

Maybe we should contact the Code Aurora project, since they are responsible for this bug?

[abiemann]

Sorry about that long delay - and thank you h6ah4i for your effort ! I've just deployed the "tunnel player workaround" to my Samsung Galaxy S4 (Model SPH-L720, Android 4.3, Hardware version L720.08) and after pressing "play" and selecting a music track, I do see a red waveform on the display that matches the beat of the music ! It looks like the work-around is functional.

[felixpalmer]

@abiemann, great to hear you've found a workaround for the tunnel player. Do you think you could perhaps integrate the same workaround into this project and submit it as a Pull Request, so others can benefit? Thanks

[mobilevisuals]

I have tested on my S4 Mini. The visualization works the first time the project is installed from Eclipse. I test with the music players "Sound picker" and "Choose music track". I close "Tunnel player workaround" and then start it by clicking on the app icon. The visualization also works now for both of the music players.

I restart the S4 Mini and open "Tunnel player workaround". The visualization does not work now, it only shows a flatline.

abiemann, do you get the same problem when you restart your S4?

h6ah4i, is it possible to make the visualization work then the device is restarted? It seems like the restart messes up something.

[h6ah4i]

@mobilevisuals I think it's impossible to make the workaround more reliable. No way to obtain whether the MediaPlayer instance backend is a tunnel player...

[mobilevisuals]

h6ah4i; Do you get the same problem when you restart your S4?

[mobilevisuals]

Felix: Have you tested this on your mobile? Do you get the same problem when you restart it? I want to know if this only happens on the S4 Mini, which is a very compact device. This makes it limited in some ways, it doesn't have the full capacity as the S4.

[h6ah4i]

@mobilevisuals The demo app works properly on my S4 after rebooting.

[mobilevisuals]

That is really good to hear! Have you deployed this code fix in your Visualizer app? It is a big improvement, since S4 is the most popular device now.

Mobiroo tested some of my company's visualizer apps and they found that visualization does not work on the Vanilla build on the Galaxy Note, HTC Evo 3D, Samsung Galaxy S3, and Samsung Galaxy S2.“ It is called Vanilla when not customized from its original form. They found the same problem on the integrated build of Galaxy note, Galaxy S2, and HTC Evo 3D.

Is Galaxy Note derived from S4? I assume that the code fix won't solve the problem on the Nexus devices, HTC Evo 3D, S3 and S2,since these are not derived from S4.

Which part of the code in the project can be used to fix the problem? I can add it to one of my visualizer apps and then send it to Mobiroo for testing. Then we'll know on which devices it works.

[mobilevisuals]

I got a response from Samsung today and they want a description of the problem. Isn't this also caused by a bug in the Digital Signal Processors? If so, how should I describe the DSP connection? This is what I plan to write, can you tell me if this is OK or if should add or remove something?

Music visualization should work on all the new Galaxy devices according to the specification, but it does not work on a lot of the new high profile devices, like the S4. We have investigated the problem and we came to these conclusions:

Galaxy S4 and derived models

These devices use a hardware acceleration playback mechanism called "Tunnel Player (Tunnel Playback)" when playing music files. If the tunnel player is used, audio capturing process is bypassed. As a result, visualizer does not works at all. This issue is already reported on Samsung's developer forum, but it seems not fixed yet. • http://developer.samsung.com/forum/board/thread/view.do?boardName=GeneralB&messageId=238465 Originally, the tunnel player seems to have been introduced by Code Aurora Forum's developer. • https://www.codeaurora.org/cgit/quic/la//platform/frameworks/base/tree/media/libstagefright/AwesomePlayer.cpp?id=refs/heads/b2g/ics_strawberry

Other Galaxy devices

It has been showed by tests conducted by Mobiroo that music visualization do not work on Galaxy Note and S3. Snapdragon based S3 uses LPAPlayer for MediaPlayer class backend. The "LPA" means Low Power Audio, and the LPAPlayer seems very similar to tunnel player, so this is probably causing the problem here. We do not know the reason why music visualization does not work on Galaxy Note. I think this problem is very serious, because it gives Android a bad reputation and makes people want to buy iPhone instead. Music visualization always works on the new iOS devices without any issues. I know several people, who have bought iPhone instead of Android, just because they want music visualization to work. Several developers of music visualizer apps are trying to solve this problem in an open source project. You can find more information here in this thread:

https://github.com/felixpalmer/android-visualizer/issues/5#issuecomment-29638873

[h6ah4i]

@mobilevisuals

I've already introduced this workaround method in my Music Visualizer app since v1.5.0 (about five months ago). But sometimes the workaround fails, because another player app has already gained the tunnel player instance before my app launched. So, even now, some users report me that my app sometimes does not work, or suddenly stopped to work.

I don't know about Galaxy Note, S2, S3 and HTC Evo 3D's issues. At least my app works on my Galaxy S2 (SC-02C with stock Android 4.0.3 ROM) and HTC Evo 3D (ISW12HT with stock Android 4.0.3 ROM). And I know a bit about S3. Snapdragon based S3 uses LPAPlayer for MediaPlayer class backend. The "LPA" means Low Power Audio, and the LPAPlayer seems very similar to tunnel player. Yet, I'm NOT sure the LPAPlayer is the evil.

Which part of the code in the project can be used to fix the problem? Refer to onCreate() method of MediaPlayerLocalService class.

I got a response from Samsung today and they want a description of the problem. Isn't this also caused by a bug in the Digital Signal Processors? If so, how should I describe the DSP connection? This is what I plan to write, can you tell me if this is OK or if should add or remove something?

Sorry, I don't know WHY visualizer doesn't work when tunnel player is used. I haven't read the TunnelPlayer class, ALSA and lower drivers implementations completely. So I can't describe about DSP : (

Are you going to write about THIS issue ticket? I think it provides very useful information for Samsung's developer.

And... I'm not good at English, so please correct my sentences more preferable.

[mobilevisuals]

I have reported to Samsung now and I referred to this thread. Thanks for the S4 code fix! I am going to add it to one of visualizer apps. I hope that it solve this issue for S4, but there are a lot of other devices with problems

It would be good if we could list all the devices, which don't support music visualization. These could then be excluded from the compatibility list on Google play. This would decrease refunds and bad reviews. Do you know any more new devices,which don't support music visualization?

[abiemann]

There's one additional technical detail that I just discovered on my Galaxy S4... If the media volume is set all the way down by the user then the Visualizer data immediately stops coming through. For example, this means that onFftDataCapture() will return an fft[] containing just 0's

This will happen even with the tunnel workaround implemented, and for any audio session ID, and even with the Visualizer scaling mode explicitly set to SCALING_MODE_NORMALIZED.

If the media volume is increased just slightly then the Visualizer data will start coming through again.

[mobilevisuals]

h6ah4i: I try to add your S4 code fix to my app "Astral 3D visualizer", but I can not get it to work. I put the call

mSilentPlayer = createSilentMediaPlayer();

right before I create my own Mediaplayer in onCreate() of the main activity. The silent media player doesn't get started , which makes the tunnel player start. This makes the music visualization not work. I test on my S4 mini, but also on S4 on Samsung Remote labs (or should music visualization not be tested here?). It is the same result on both of these. I debug the code and I see that mp.setDataSource causes an IOException. The values for the arguments are:

FileDescriptor 45 startOffset 60
lengthFile 34478

I then debug tunnel_player_workaround and it is the same values and the same IOException here. I only see a flatline on tunnel_player_workaround now. I uninstall the app and install again, but it is the same thing. I test on S4 on Samsung Remote labs, but it is the same thing here. I haven't changed any code in tunnel_player_workaround. Do you know how I can proceed here?

[h6ah4i]

@mobilevisuals It seems strange. On my S4, ParcelFileDescripter is like the flollowing;

{ParcelFileDescriptor: FileDescriptor[46]} start=60 len=961305}

FileDescriptor value is different, but it's OK, this value is determined on runtime. However len parameter is different, it's a problem. Maybe your MP3 file is broken.

[mobilevisuals]

The MP3 file that I use is 34 478 bytes and has MD5: 1D52337F55B07B6B823F5BFA9C4452FB. Do you have the same on your file? If so, how can the file be broken?

[h6ah4i]

I don't know why your MP3 file is different from mine... Please download it again from the repository.

https://github.com/h6ah4i/tunnel_player_workaround/blob/master/assets/workaround_1min.mp3

% git clone https://github.com/h6ah4i/tunnel_player_workaround.git Cloning into 'tunnel_player_workaround'... remote: Counting objects: 45, done. remote: Compressing objects: 100% (32/32), done. remote: Total 45 (delta 0), reused 45 (delta 0) Unpacking objects: 100% (45/45), done. Checking connectivity... done

% cd tunnel_player_workaround/assets

% dd if=workaround_1min.mp3 of=broken.mp3 bs=1 count=34478 34478+0 records in 34478+0 records out 34478 bytes transferred in 0.119411 secs (288734 bytes/sec)

% ls -la *.mp3 -rw-r--r-- 1 h6ah4i staff 34478 12 4 07:25 broken.mp3 -rw-r--r-- 1 h6ah4i staff 961305 12 4 01:30 workaround_1min.mp3

% md5sum *.mp3 8e2e1af675842886571ffd6452175b80 broken.mp3 14989cdbc0786e8ec326a7483374acad workaround_1min.mp3

[mobilevisuals]

Thanks, I got it to work on the S4 now! I tested on Samsung remote labs. It is possible to test there, but you have to go to the settings and turn up the media volume, because it is set to 0 as default. You will always get a flatline if you don't do that. Have you tested your app on Samsung remote labs? Maybe we should test the visualization on all the devices there? This would make it possible to exclude the incompatible devices, which would reduce the bad reviews and the refunds.

I also tested on S4 Mini, but the music visualization still does not work on tunnel_player_workaround or Astral 3D Music Visualizer . S4 Mini is however a very compact device, with some limitations.

[mobilevisuals]

I am listing the devices, which don't support music visualization. These can then be removed from the compatibility lists on Google Play, to avoid bad reviews and refunds. I checked all the reviews for my visualizer apps and found these incompatible devices:

S4 mini,Nexus 5,Kyocera Rise,LG Spectrum 2,Nexus 4,Nexus 7,Galaxy Note

Galaxy Note was incompatible according to Mobiroo's testing, so I added that one too. Does anyone know any more devices, which don't support music visualization?

Mobiroo says that the integrated build of S3 is compatible, it is only the vanilla build, which is incompatible. My apps have got 6 reviews from S3, where music visualization worked, so I think S3 can be listed as compatible. I have not got any review, where it didn't work on S3.

Mobiroo also says that it doesn't work on neither the Vanilla build nor the integrated build of HTC Evo 3D and Galaxy S2. This is very strange, since my apps only have got reviews from these devices, where music visualization worked.

These devices support music visualization according to the reviews of my apps:

Galaxy Note2 Galaxy Tab3 8.0 EVO 3D Galaxy S3 Event (C5133) Galaxy Mega Galaxy Prevail II ST21a Iconia Tab A100 TF700T Optimus L3 Galaxy Tab2 10.1 XOOM (stingray) C1505 Galaxy S II Galaxy S I

[h6ah4i]

@felixpalmer I added the workaround code in your android-visualizer app. And I've just sent a pull request (https://github.com/felixpalmer/android-visualizer/pull/6).

[cypherdare]

Haruki, thanks for sharing all this info! I've been getting multiple emails a day for the past year because of this.

Do you know if your solution for occupying the tunnel player helps when using audio session 0 to listen to other media players, or is it only helpful for playing your own audio? i.e., is only one tunnel player allowed on the entire device, or only one per app?

[h6ah4i]

@cypherdare The number of tunnel player instance is limited to only one per entire device. Therefore the workaround is effective for sounds from other player apps. But you must note that it's too difficult to maintain the sequence of creation MediaPlayer instances among different processes. So the workaround is not perfect, and sometimes it won't work as expected.