audioMotion.js icon indicating copy to clipboard operation
audioMotion.js copied to clipboard

Loading metadata with `p-limit`

Open Borewit opened this issue 5 months ago • 1 comments

This PR is build on top of:

  • #89

I took a bunch of metadata files, spin op the audioMotion.js in server mode. With the changes of #89 it metadata loaded slowly, massive parallel loading of the playlist, which took ~6 seconds to load all metadata.

First I disabled parallel fetching in the addSongToPlayQueue, retrieveMetadata , function chain. The parallel loading is fact not done in retrieveMetadata, which is kind of mind fuck, as this method is responsible for the entire queue. Yet if a bunch of files are added to the queue, this function called for each item again and again. Hence #89 makes the parallel loading worse, and therefor performance goes down the drain. After disabling all parallel processing, putting awaits all over the place. The increased the performance to ~1100 ms!

Then I re-enabled parallel loading, now using p-limit, to avoid the number of parallel processing, and to slowly tear down the requirement to keep calling retrieveMetadata way to ofton. Resulting to loading the metadata in 600ms.

Technical debt

  • I cut loose the promise add metadata to queue. https://github.com/hvianna/audioMotion.js/blob/471e9474760e3c494167c17f78c7d0bea3afc446/src/index.js#L1258 I maybe I should say, I kept as is on dev branch. There are several ways to resolve this, but it requires some refactoring. One solution could be to push things which require metadata in a queue. Then we consume this queue by the the metadata retrieveMetadata, which maybe could be a worker / parallel processing thingy.

Alternative solution

Update: I created #104 as probably a better alternative them, as that one queues items

Borewit avatar Jul 20 '25 13:07 Borewit

Published this version on Netlify: https://audiomotion.netlify.app/

Borewit avatar Jul 20 '25 14:07 Borewit