Implement Audio (Table provided mechanical audio)

[freezy]

Audio is currently completely absent, even on parse level, i.e. the audio streams are skipped. What needs to be done:

• [x] Parse the audio data from the VPX file
• [x] Make them accessible by name in the table object
• [ ] Add a JavaScript API to play them, with the parameters Visual Pinball provides, namely:
  • loop count
  • volume
  • pan,
  • random pitch
  • pitch,
  • use same (?)
  • restart
  • front rear fade

We might be able to stream the audio directly from the table blob. To check. A good high-level audio library seems to be Howler.

[neophob]

How does VPX implements audio? is the ROM used for feedback?

[freezy]

A .vpx file ships with a bunch of .wav or MP3 files, and the table script plays them. That's for mechanical sound effects, the actual ROM sound isn't handled at all, that's done by PinMAME.

[neophob]

Ok so this is about handling the mechanical sound effects then, right?

[freezy]

This issue, yes, entirely.

[freezy]

I've added the parsing to feature/sound. What's left to do is implement the scripting API:

TODO - Reference

There's another way of playing sounds however, which is through external files in VP's Music folder. I think that's what PlayMusic uses, but I'll verify. In this case we would need a special provider whose implementation changes depending on the platform or environment.

[freezy]

Note that the table loader doesn't actually load the sound yet - it just parses its metadata and saves the position where the sound data is stored within the stream. All the data is in PinSoundData, while access goes through PinSound, which can be found in the Table's sounds property.

It might be a good idea to link the data via a Blob to an URL with createObjectURL, so it's accessible to any sound library. Like it's currently done with textures as well.

[freezy]

About pre-loading, here's how it currently works with textures:

• The public VPX-JS API to create the 3D scene is generateTableNode(). It's currently called like that because it creates a node-base scene and I didn't want to use three.js lingo (better suggestions welcome).
• I would plug into the same method. It's basically what vpweb calls when the table is loaded. You could also create another method, but then you somehow got to make sure the host app doesn't forget to call it.
• For textures I'm using two different loaders, ThreeTextureLoaderNode and ThreeTextureLoaderBrowser. This is because Node.js does additionally optimize and convert the images and uses the fs API, while the browser does no optimizations and uses an URL loader. You'll probably have to do the same, because createObjectURL() doesn't exist in Node.js (just stub it with nothing in Node.js).

So your pre-loading would loop through the table's sound properties, potentially pre-process them with volume and whatnot (check what VP does), create object URLs to the data, and save the link to it in the PinSound object.

Your Howler implementation can then work with those URLs as if they were normal remote URLs.