libvgm icon indicating copy to clipboard operation
libvgm copied to clipboard

The "utils" package should be split up.

Open ValleyBell opened this issue 3 years ago • 9 comments

Currently, including the utils sub-library in a project pulls lots of other stuff that you may not need.

For example, the audio library uses utils for threading stuff, but this also pulls the DataLoader (incl. zlib) and string conversion (incl. iconv) functions in, which it does not need.

Currently, there are 3 categories:

  • file/memory loading
  • threads/synchronization
  • string codepage conversion

I want at least different CMake targets for those. Having separate pkg-config entries should be useful as well.

ValleyBell avatar Nov 26 '20 08:11 ValleyBell

Alternatively, what about pulling items from utils directly into the libraries that need it?

For example, I think threading is only required by the audio library, that could just be moved into there.

Is DataLoader / string conversion used outside of the player library?

jprjr avatar Dec 11 '20 15:12 jprjr

Another idea, libvgm could be split up. Maybe form a new org or use vgmrips, and move emu, player, etc into their own repos.

I've started learning a bit more about modern CMake and how to have it auto-register "packages" on the system, which makes it really nice to use for devs. I figured out how to use CMake with Visual Studio and have it auto-detect the various sub-libraries I had built, I did zero config of Visual Studio with finding external references, adding include directories, etc.. It's pretty cool. I just opened the folder, it detected the CMakeLists, and find_package worked exactly as it should, it was great.

I say that because I've always gotten the feeling that dealing with external dependencies on Visual Studio is rough (I barely use it so anything besides opening a project and clicking build, I consider rough), so I'd understand if breaking this up sounds like it could be a pain - but with a few changes to the cmake files it'd be pretty close to seamless. Devs could just clone down the new repos, open it in visual studio, build. CMake would take care of resolving the dependency between the player and emu libraries.

jprjr avatar Dec 11 '20 20:12 jprjr

Another idea, libvgm could be split up. Maybe form a new org or use vgmrips, and move emu, player, etc into their own repos.

I wouldn't mind this, specifically, the "audio" component could be completely split from libvgm, leaving it only with the "emu" and "player" components.

It's good that CMake works for you, I have had the opposite experience when using CMake with MinGW packages. Specifically, the library finder scripts never work for me, and that it doesn't recognize the proper place to install programs. This is why I have been pushing for ways to use the library that don't require CMake.

superctr avatar Dec 11 '20 20:12 superctr

Are you cross-compiling or using mingw on windows? I can only speak from the cross-compiling experience, but once I setup my toolchain file with:

set(CMAKE_FIND_ROOT_PATH /path/to/x86_64-w64-mingw32)
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

(along with the usual cross-compiling settings) everything started working great. Don't know if any of that could be a help?

I think MXE includes that as a script, too, which I remember working really well.

jprjr avatar Dec 11 '20 21:12 jprjr

Alternatively, what about pulling items from utils directly into the libraries that need it?

For example, I think threading is only required by the audio library, that could just be moved into there.

Is DataLoader / string conversion used outside of the player library?

I'd like to keep the possibility of using e.g. the string conversion stuff in a program that only needs the emu library but not the player.
example: You want to write a PMD player and decode + display tags. Then you won't need the player library (only emu and maybe audio), but you'd like to easily convert the tags from CP932 to UTF-8.

For in_vgm I also want to be able to use the mutex functions (for thread safety) despite not needing any of the audio parts, as Winamp does audio output by its own.

Another idea, libvgm could be split up. Maybe form a new org or use vgmrips, and move emu, player, etc into their own repos.

I actually planned to move it to vgmrips once libvgm becoming "stable". (= C-API for the player library)

While preferring to keep all the current components in the same repo, I'm not completely against it. We could also just keep it in the same repo, but split it into separate sub-projects per folder. Similar to as it is now, but with more explicit separation regarding installation and configuration files.

ValleyBell avatar Dec 12 '20 09:12 ValleyBell

I've been lurking around the ongoing development and discussion, excuse me for dropping in like this.

Another idea, libvgm could be split up. Maybe form a new org or use vgmrips, and move emu, player, etc into their own repos.

I wouldn't mind this, specifically, the "audio" component could be completely split from libvgm, leaving it only with the "emu" and "player" components.

I would definitely appriciate the audio components being split for future reuse.

It's good that CMake works for you, I have had the opposite experience when using CMake with MinGW packages. Specifically, the library finder scripts never work for me, and that it doesn't recognize the proper place to install programs. This is why I have been pushing for ways to use the library that don't require CMake.

I would say offering support for both a 'modern' system as CMake or Meson and a 'traditional' system as (auto)make would seem a good solution to me.

DeltaRazero avatar Dec 12 '20 14:12 DeltaRazero

I 100% agree with splitting audio out. Having it in isn't the biggest deal (you can just not build it). My interest is primarily in writing plugins for other players that handle audio and synchronization for me.

I do want to voice, I used to really dislike cmake, it is frustrating how there's not a whole lot of good documentation on being a cmake user, most docs are on writing cmake files, and cross-compiling on automake is pretty easy - you just pass LDFLAGS and C(XX)FLAGS for finding your libraries and includes, set the appropriate CC, and you're probably good to go.

Once I discovered the toolchain file and how to tell cmake "here's where my stuff is" it got so much better. I'd imagine using a toolchain file on Windows is pretty crucial.

Still haven't figured out how to tell Meson "here's my stuff" so I'm not a big fan.

jprjr avatar Dec 12 '20 15:12 jprjr

I'm fine with the audio split. However I'll leave it to you to come up with a library name. :P (I can still do the splitting by myself though.)

While at it, the utils package could become a separate repo as well.

CMake is going to stay the primary build system, as this is what I know best. I'm okay with having support for one or maybe two additional build systems, but I'll have to rely on contributions there.

ValleyBell avatar Dec 13 '20 08:12 ValleyBell

you might benefit from a separate, reusable file stream library that wraps stdio, memory buffers, transparent gzip read/write and recurses through directories and archives such as zip files

vampirefrog avatar Jun 16 '24 07:06 vampirefrog