schellingb
Support for .zip and .dosz archives in playlist
I have a DOS game, and stored the whole disk content in a dedicated directory.
I also archived the directory in a zip archive, and copied the .zip archive to a .dosz file, as described here: https://docs.libretro.com/library/dosbox_pure/#zip-files-can-be-renamed-to-dosz
This is what I've observed so far after enabling in RetroArch the dosbox-pure core.
Import content functionality
In retroarch I've used the import content functionality for creating a DOS playlist:
- start.exe in the folder is found and recognized
- start.exe in zip is found and recognized
- start.exe in dosz is not found
That would be the first issue; the import content functionality does not recognize .dosz
Start game from generated playlist
The "import content" functionality generated a playlist
- unzipped
start.exestarts without issues - zipped
start.exedoes not start, on the console, the message "illegal start command" is shown - nothing to test since the
.doszentry does not appear in the playlist
From the logs of retroarch, I've found the following command:
[ERROR] [Environ] SET_MESSAGE_EXT: Unable to open content file: /opt/data/games/dos/Krynn/champions_of_krynn/champions_of_krynn.zip#champions_of_krynn/START.EXE
That would be the second issue; the import content functionality creates "invalid" entries for .zip archives or the dosbox-pure core misinterprets the playlist entries for .zip archives.
Start game manually
Instead of using the playlist, I tried at last to start the game manually
- unzipped
start.exestarts - selecting the
.ziparchive provides an option. if I select load archive, the games starts; if I select "browse archive", and then start.exe it does not start, it shows the illegal start command - starting
.doszstarts(!) the game directly
Conclusion
.zip and .dosz file are supported but the integration with the playlists does not work as expected.
The way this core is meant to be used is to start the .dosz directly and use the auto start feature (see that part of the documentation) of the start menu. You designate auto start by pressing right on a given .exe file. The core has no control over what RetroArch's import/scan feature finds. For example if RetroArch finds "SETUP.EXE" of a game but not "GAME.EXE" it might always run the setup program. So that's why the option for renaming to .dosz exists in the first place, to avoid RetroArch messing with the content. Which, as you figured out, works :-)
I also assumed that the missing detection for dosz is somewhere else, but I wanted to describe the whole situation.
Do you happen to know where the fact that .dosz are not recognized should be reported?
The fact that .zip archives in the playlist do not work...
Is this something that the core has control over?
What do you mean .dosz is not recognized? The core lists .dosz as a supported file extension so RetroArch will ask to open them with DOSBox Pure (and only DOSBox Pure because no other core supports .dosz). With .zip it's more complicated because RetroArch can load console ROMs even from inside .ZIP so it might list the .ZIP file itself or a specific file inside the .ZIP. The core does not run during playlist management or directory scanning so it has no control over that.
How are you creating your playlists? Try using Manual Scan with "Scan Inside Archives" set to off.
What do you mean .dosz is not recognized?
I meant that the game is not recognized while importing the content, as explained at the beginning. The uncompressed game and the game in the zip archive are added to the playilist.
How are you creating your playlists?
With "Import Content", then "Scan Directory".
I can also use "Scan File" and select the .dosz file, but the game is not recognized/added to the playlist.
Try using Manual Scan with "Scan Inside Archives" set to off.
If I do a manual scan, with "Scan Inside Archives" disabled, System name set to "DOS", and core set to DOSBOX-pure, then
- the .dosz file is added to the playlist
- all metadata is missing (title, year, album art, ... )
is there a setting I am missing?
If I do a manual scan, with "Scan Inside Archives" disabled, System name set to "DOS", and core set to DOSBOX-pure, then
- the .dosz file is added to the playlist
- all metadata is missing (title, year, album art, ... )
is there a setting I am missing?
This is normal for a manual scan, or at least this happened when i've used it in the past for other cores. I now make my playlists manually being as the Import Scan feature is not reliable, as you've found out.
Inside the RetroArch folder is a folder named playlists which contains .lpl files for each playlist. These files can be opened with any text editor. It's fastest to copy an existing DOS.lpl file and edit that. I then put the game year in the title and make my own box art/screenshots, because RetroArch's downloaded thumbnails are often poor quality.
You can even make custom playlist icons. For example, if you make a playlist named "Windows.lpl" (for Windows 95/98 games) then the icons would be named Windows.png and and Windows-content.png and placed inside RetroArch\assets\xmb\monochrome\png if you're using the default theme.
Some new info about my zip archive.
Instead of having the content directly in the zip archive, I had zipped the folder with the game in it. This makes a relevant difference.
The path of the executable in the archive champions_of_krynn.zip#champions_of_krynn/START.EXE
Changing the archive so that the path is champions_of_krynn.zip#START.EXE, changes the behavior; dosbox-pure can start the playlist entry generated by the import content functionality.
I'm not sure what the most sensible improvement is; it is certainly confusing, as it is possible to start the game, just not with the playlist entry created by retroarch.
@PoloniumRain thank you for the suggestion; I am aware how to write playlists and already do so.
I'm reporting the issue because I would like to reduce the setup ("manually" adding missing games, images, other metadata ...) for the games I have to zero.
I think there's some confusion here.
The .dosz file extension is not meant to work with the RetroArch "Import Content" scan feature. As @schellingb mentioned earlier:
The way this core is meant to be used is to start the .dosz directly
Which would be: Main Menu > Load Content.
And this is why things are done this way:
if RetroArch finds "SETUP.EXE" of a game but not "GAME.EXE" it might always run the setup program. So that's why the option for renaming to .dosz exists in the first place, to avoid RetroArch messing with the content. Which, as you figured out, works :-)
So it is by design that .dosz files do not work with Import Content > Scan Directory/Scan File.
For example, you'll find that if you scan a folder that has many DOS games that are either inside folders or .zip files, then open up the created DOS.lpl playlist file, the path for some games will occasionally link to random files. It will not always point to the actual game executable file. One game that i know will not work is the QDOS version of Quake, where the created playlist file will have a path that points to PDIPX.COM, or some other incorrect file, instead of the correct file which is QDOS.EXE. So each time you run the game from the playlist, the game wont start.
As you've also discovered, the scan will also not work with .zip files depending on the directory layout inside the zip.
So being as .dosz files do not work with the scan at all, then this problem doesn't exist for them.
Here is the RetroArch database file for DOS games. This is what RetroArch uses to create a playlist when a scan is run. The core has no control over this. You can see that it lists the .exe files for each DOS game, which are typically correct, but there's tons of DOS games that are missing. When a game is missing, or the directory structure isn't correct, the scan just seems to link to any random file, which isn't even always an executable file.
The way to solve all these problems would be for someone to update that RetroArch database file with all DOS games and get it to also work with subdirectories, and good luck with that lol. There's probably well over 8000 DOS games and then there's patches, mods, new games and backports that are still released...
The .dosz file extension is not meant to work with the RetroArch "Import Content" scan feature. As @schellingb mentioned earlier:
I see no mention that it is not meant to work, and little reason why the import content scan feature should not be able to recognize .dosz archive (if it recognizes the zip archive) and create a matching entry in the playlist.
But for that I opened a separate issue: https://github.com/libretro/libretro-database/issues/1586 (I hope it is the correct repository)
The way to solve all these problems would be for someone to update that RetroArch database file with all DOS games and get it to also work with subdirectories, and good luck with that lol.
This seems completely unrelated to this issue.
- The game (except in
.dosz) is recognized correctly. - The game starts correctly when not zipped, or when zipped with an "appropriate" structure.
As you've also discovered, the scan will also not work with .zip files depending on the directory layout inside the zip.
This is IMHO something worth improving (although I guess with low prio), either with a better error message, or by supporting more archive structures.
Given the file champions_of_krynn.zip#champions_of_krynn/START.EXE
this works:
{
"path": "/opt/data/games/dos/Krynn/champions_of_krynn/champions_of_krynn.zip",
"label": "Champions of Krynn (1990)",
"core_path": "/home/fekir/.config/retroarch/cores/dosbox_pure_libretro.so",
"core_name": "DOS (DOSBox-Pure)",
"crc32": "B86758DC|crc",
"db_name": "DOS.lpl"
},
but this does not
{
"path": "/opt/data/games/dos/Krynn/champions_of_krynn/champions_of_krynn.zip/champions_of_krynn#START.EXE",
"label": "Champions of Krynn (1990)",
"core_path": "/home/fekir/.config/retroarch/cores/dosbox_pure_libretro.so",
"core_name": "DOS (DOSBox-Pure)",
"crc32": "B86758DC|crc",
"db_name": "DOS.lpl"
},
On the other hand, given champions_of_krynn.zip#START.EXE
this works
{
"path": "/opt/data/games/dos/Krynn/champions_of_krynn/champions_of_krynn.zip",
"label": "Champions of Krynn (1990)",
"core_path": "/home/fekir/.config/retroarch/cores/dosbox_pure_libretro.so",
"core_name": "DOS (DOSBox-Pure)",
"crc32": "B86758DC|crc",
"db_name": "DOS.lpl"
},
and this too:
{
"path": "/opt/data/games/dos/Krynn/champions_of_krynn/champions_of_krynn.zip/#START.EXE",
"label": "Champions of Krynn (1990)",
"core_path": "/home/fekir/.config/retroarch/cores/dosbox_pure_libretro.so",
"core_name": "DOS (DOSBox-Pure)",
"crc32": "B86758DC|crc",
"db_name": "DOS.lpl"
},