ebiten icon indicating copy to clipboard operation
ebiten copied to clipboard

audio: float32-value streams in addition to int16-value streams

Open hajimehoshi opened this issue 2 years ago • 16 comments

Fix Oto first.

hajimehoshi avatar Jun 23 '22 17:06 hajimehoshi

These exposed API would be affected

package audio

func (c *Context) NewPlayer(src io.Reader) (*Player, error)
func (c *Context) NewPlayerFromBytes(src []byte) (*Player, error)
func NewInfiniteLoop(src io.ReadSeeker, length int64) *InfiniteLoop
func NewInfiniteLoopWithIntro(src io.ReadSeeker, introLength int64, loopLength int64) *InfiniteLoop
func Resample(source io.ReadSeeker, size int64, from, to int) io.ReadSeeker
package mp3

func Decode(context *audio.Context, src io.Reader) (*Stream, error)
func DecodeWithSampleRate(sampleRate int, src io.Reader) (*Stream, error)
func DecodeWithoutResampling(src io.Reader) (*Stream, error)
package vorbis

func Decode(context *audio.Context, src io.Reader) (*Stream, error)
func DecodeWithSampleRate(sampleRate int, src io.Reader) (*Stream, error)
func DecodeWithoutResampling(src io.Reader) (*Stream, error)
package wav

func Decode(context *audio.Context, src io.Reader) (*Stream, error)
func DecodeWithSampleRate(sampleRate int, src io.Reader) (*Stream, error)
func DecodeWithoutResampling(src io.Reader) (*Stream, error)

Adding option arguments to all the above APIs would not be feasiable 🤔

hajimehoshi avatar Nov 06 '22 10:11 hajimehoshi

Though it is possible to add an option without breaking compatibility, it is not good to confuse users. Let's postpone this to v3, when we can break compatibility.

hajimehoshi avatar Nov 06 '22 13:11 hajimehoshi

I revisited this and realized that changing this at v3 might be too agressive. Users would be hesitated to migrate to v3 if v3 starts to support only floats. Though it is inevitable to add some confusions, what about adding these APIs?

package audio

func (c *Context) NewPlayerF32(src io.Reader) (*Player, error)
func (c *Context) NewPlayerF32FromBytes(src []byte) (*Player, error)
func NewInfiniteLoopF32(src io.ReadSeeker, length int64) *InfiniteLoop
func NewInfiniteLoopWithIntroF32(src io.ReadSeeker, introLength int64, loopLength int64) *InfiniteLoop
func ResampleF32(source io.ReadSeeker, size int64, from, to int) io.ReadSeeker
package mp3

type StreamF32 struct { /* ... */ }

func DecodeF32(src io.Reader) (*StrreamF32, error)
package vorbis

type StreamF32 struct { /* ... */ }

func DecodeF32(src io.Reader) (*StreamF32, error)
package wav

type StreamF32 struct { /* ... */ }

func DecodeF32(src io.Reader) (*StreamF32, error)

I don't plan to provide automatic-resampling versions of Decode functions as this is problematic in terms of execution cost. Resample(F32) should work.

hajimehoshi avatar Sep 11 '23 16:09 hajimehoshi

So, you want to keep both formats even for v3, or would you rather switch to only F32 format? Whatever the case, I think that F32 should become the "main" format, so I think that creating audio/v2 may be a more desirable transition path.

(EDIT: I don't know if you want to keep L16 only for back-compatibility or if it's also to offer it as a potentially more performant option)

tinne26 avatar Sep 21 '23 13:09 tinne26

I'd like to switch to F32 in the future. I have not determined to keep S16 in v3.

hajimehoshi avatar Sep 21 '23 13:09 hajimehoshi

Yeah audio/v2 sounds a good idea.

EDIT Or audiof32, maybe? :)

hajimehoshi avatar Sep 21 '23 13:09 hajimehoshi

So, if we create audio/v2, this would be like this. The API seems almost the same, but io.Reader treats float32 data instead of int16.

// audio/v2
func Resample(source io.ReadSeeker, size int64, from, to int) io.ReadSeeker

type Context
func CurrentContext() *Context
func NewContext(sampleRate int) *Context
func (c *Context) IsReady() bool
func (c *Context) NewPlayer(src io.Reader) (*Player, error)
func (c *Context) NewPlayerFromBytes(src []byte) *Player
func (c *Context) SampleRate() int

type InfiniteLoop
func NewInfiniteLoop(src io.ReadSeeker, lengthInBytes int64) *InfiniteLoop
func NewInfiniteLoopWithIntro(src io.ReadSeeker, introLengthInBytes int64, loopLengthInBytes int64) *InfiniteLoop
func (i *InfiniteLoop) Read(b []byte) (int, error)
func (i *InfiniteLoop) Seek(offset int64, whence int) (int64, error)

type Player
func (p *Player) Close() error
func (p *Player) IsPlaying() bool
func (p *Player) Pause()
func (p *Player) Play()
func (p *Player) Position() time.Duration
func (p *Player) Rewind() error
func (p *Player) SetBufferSize(bufferSize time.Duration)
func (p *Player) SetPosition(offset time.Duration) error
func (p *Player) SetVolume(volume float64)
func (p *Player) Volume() float64
// audio/v2/vorbis
type Stream
func Decode(src io.Reader) (*Stream, error)
func (s *Stream) Length() int64
func (s *Stream) Read(p []byte) (int, error)
func (s *Stream) Seek(offset int64, whence int) (int64, error)
// audio/v2/wav
type Stream
func Decode(src io.Reader) (*Stream, error)
func (s *Stream) Length() int64
func (s *Stream) Read(p []byte) (int, error)
func (s *Stream) Seek(offset int64, whence int) (int64, error)

hajimehoshi avatar Dec 16 '23 18:12 hajimehoshi

Compared to text/v2, the API is not drastically changed, so I doubt it would be worth introducing a new package. Another concern is, audio.Context and audio/v2.Context cannot coexist, while text and text/v2 are available at the same time (though I don't recommend it).

So, I slightly prefer the first idea (adding *F32 versions to the current packages) https://github.com/hajimehoshi/ebiten/issues/2160#issuecomment-1714185922

hajimehoshi avatar Dec 16 '23 18:12 hajimehoshi