beetle-psx-libretro icon indicating copy to clipboard operation
beetle-psx-libretro copied to clipboard

[Bounty] Increase PSX Memory

Open barisyild opened this issue 4 years ago • 47 comments

Increase PSX Memory feature can help remove some limitations with a patch in heap memory supported games.

Additional memory must be located in dma.

Development Consoles has 8mb dma but retail consoles has 2mb dma

There are even people who have modified their console on the internet to use 8mb dma. (useless for homebrew development or heap memory increased games)

Resources:

SCPH-5502 8MB Modification: http://www.psxdev.net/forum/viewtopic.php?t=660 https://retrocomputing.stackexchange.com/questions/13551/how-much-memory-did-the-playstation-development-kit-have

Note:

Preferably, it would be nice if the amount of RAM could be adjusted.

Bounty Link: https://www.bountysource.com/issues/90799133-feature-request-increase-psx-memory

barisyild avatar Mar 30 '20 15:03 barisyild

Where should the additional memory be located?

simias avatar Mar 30 '20 18:03 simias

Additional memory must be located in dma.

Development Consoles has 8mb dma but retail consoles has 2mb dma

There are even people who have modified their console on the internet to use 8mb dma. (useless for homebrew development or heap memory increased games)

Resources:

SCPH-5502 8MB Modification: http://www.psxdev.net/forum/viewtopic.php?t=660 https://retrocomputing.stackexchange.com/questions/13551/how-much-memory-did-the-playstation-development-kit-have

barisyild avatar Mar 30 '20 19:03 barisyild

By modifying the games, the ram limit of the compatible games can be increased. (Will patch remove some limitations)

Example of Retail Crash Team Racing (SCES-02105)

image

barisyild avatar Mar 30 '20 19:03 barisyild

Ah, so instead of having 2MB mirrored 4 times we'd have 8MB proper. That's straightforward to implement but it'll break savestates hard.

simias avatar Mar 30 '20 19:03 simias

Out of curiosity what does CTR do differently when it has 8MB of RAM?

simias avatar Mar 30 '20 19:03 simias

For now, I think of just removing 3 limitations.

  • The Oxide character cannot be used in adventure mode or some levels due to the ram restriction.
  • 2P and 4P modes are very limited, these restrictions can be removed. (We will patch low quality characters and low quality levels)
  • It may be possible to use the boss characters in the OnlineCTR project.

barisyild avatar Mar 30 '20 19:03 barisyild

I did a quick hack to enable 8MB in beetle: https://github.com/simias/beetle-psx-libretro/tree/dev/ram-8m

It's completely hardcoded and almost certainly won't work with the dynarec but it could be a starting point.

I don't know if it works but it doesn't seem not to work (i.e. it doesn't crash when I start a game).

simias avatar Mar 30 '20 20:03 simias

I couldn't try it because there was no compiler on my computer, according to my observations ram did not increase.

barisyild avatar Mar 31 '20 00:03 barisyild

I did a quick hack to enable 8MB in beetle: https://github.com/simias/beetle-psx-libretro/tree/dev/ram-8m

It's completely hardcoded and almost certainly won't work with the dynarec but it could be a starting point.

I don't know if it works but it doesn't seem not to work (i.e. it doesn't crash when I start a game).

Doesn't work, RAM is still mirrored 4 times. Look at Cheat Engine results:

Dynarec ON: https://i.imgur.com/ffUmm0v.png Dynarec off (Beetle interpreter): https://i.imgur.com/FT0c4GJ.png

rapierx avatar Mar 31 '20 02:03 rapierx

Doesn't work, RAM is still mirrored 4 times. Look at Cheat Engine results:

I couldn't compile it. which tool do you use?

barisyild avatar Mar 31 '20 02:03 barisyild

I couldn't compile it. which tool do you use?

MSYS2 on Windows 10, following https://docs.libretro.com/development/retroarch/compilation/windows/#building-cores

rapierx avatar Mar 31 '20 02:03 rapierx

@rapierx when compiling with HAVE_LIGHTREC=1 (which adds the dynarec on/off options you described, and so I am assuming you compiled with it) the ram is 2MB mirrored 4 times still. Try compiling without that and see if the change worked with cheat engine, the dynarec code will need some additional changes to support 8MB.

ZachCook avatar Mar 31 '20 06:03 ZachCook

Without Lightrec, looks like RAM mirroring is disabled. I can't find any practical difference in performance. For example, I can't put more enemies on stages, game still hangs.

https://i.imgur.com/eF4oWAw.png

Usually Beetle uses memory addresses similar to Gameshark addresses on actual console, like 8007D954 or 6007D954. With this commit, memory addresses are random; in the image 8007D954 changes to 18873994.

rapierx avatar Mar 31 '20 12:03 rapierx

I'm not sure how to interpret these CheatEngine screenshots, I'm not familiar with this tool.

The patch won't work with dynarec enabled, I haven't touched that (well, I did update a few calls in the main code but I'm sure that it's not enough). It's meant to be a prototype to see if it works as expected using the interpreter.

Without Lightrec, looks like RAM mirroring is disabled. I can't find any practical difference in performance. For example, I can't put more enemies on stages, game still hangs.

The games almost certainly needs to be patched to make use of the additional RAM. As such without patched game this change won't do anything besides breaking games that rely on the RAM being mirrored somehow.

I can only provide 64bit Linux builds, no Windows, sorry.

simias avatar Mar 31 '20 14:03 simias

I'm not sure how to interpret these CheatEngine screenshots, I'm not familiar with this tool.

If I use Cheat Engine with Beetle to find game addresses, I usually find 4 addresses equally spaced apart, like in the images posted in https://github.com/libretro/beetle-psx-libretro/issues/634#issuecomment-606352840. https://i.imgur.com/ffUmm0v.png shows 60117ac0, 60317ac0, 60517ac0, 60717ac0. With RAM mirroring disabled I find just one address.

rapierx avatar Mar 31 '20 14:03 rapierx

@simias Is there a way to enable Printf logs, I found a ram test and want to test it.

RAMTEST.zip

barisyild avatar Mar 31 '20 14:03 barisyild

Without Lightrec, looks like RAM mirroring is disabled. I can't find any practical difference in performance. For example, I can't put more enemies on stages, game still hangs.

https://i.imgur.com/eF4oWAw.png

Usually Beetle uses memory addresses similar to Gameshark addresses on actual console, like 8007D954 or 6007D954. With this commit, memory addresses are random; in the image 8007D954 changes to 18873994.

Games are usually programmed to use 2MB ram. If you find the swap area as in the example I have shown, you can increase the ram rate.

image

barisyild avatar Mar 31 '20 15:03 barisyild

I am also in favor of the memory limitation being updated to 8mb. Crash Team Racing has a lot of room for potential if it has a little more memory (removed features, modding potential). Every other PS1 game in existence will have a potential ability for modding if there is extra room to store data

niko1point0 avatar Apr 01 '20 01:04 niko1point0

I couldn't compile it. which tool do you use?

MSYS2 on Windows 10, following https://docs.libretro.com/development/retroarch/compilation/windows/#building-cores

I'm tried on 2 computer, giving following error.

image

barisyild avatar Apr 01 '20 13:04 barisyild

It probably means that both these ifdefs are false:

https://github.com/libretro/beetle-psx-libretro/blob/84044f186847f52b84655edfe3e53e3b20146572/libretro-common/cdrom/cdrom.c#L481-L487

Judging by the flags, it looks like the 2nd one should pass on windows (I assume WIN32 is not strictly 32bits but just a historical name?).

You could stick a #define _WIN32 at the top of the file and see if that helps, but I suppose the real problem is that your build environment doesn't define the proper macros which will probably create other issues.

simias avatar Apr 01 '20 13:04 simias

It probably means that both these ifdefs are false:

https://github.com/libretro/beetle-psx-libretro/blob/84044f186847f52b84655edfe3e53e3b20146572/libretro-common/cdrom/cdrom.c#L481-L487

Judging by the flags, it looks like the 2nd one should pass on windows (I assume WIN32 is not strictly 32bits but just a historical name?).

You could stick a #define _WIN32 at the top of the file and see if that helps, but I suppose the real problem is that your build environment doesn't define the proper macros which will probably create other issues.

Could there be an error in the command I compile because many similar errors occur. (Compiles the .so library without problems)

image

make platform=win

barisyild avatar Apr 01 '20 14:04 barisyild

I am also in favor of the memory limitation being updated to 8mb. Crash Team Racing has a lot of room for potential if it has a little more memory (removed features, modding potential). Every other PS1 game in existence will have a potential ability for modding if there is extra room to store data

The problem is that we can't just increase the memory by default because it would reduce the accuracy of the emulator. On real Playstations what they seem to have done is wire the memory controller the same way as on the 8MB prototypes but with the two most significant bits of the address bus "in the air".

In practice that means that the 2MB of RAM are mirrored 4 times on the first 8MB of the address space. So for instance if you write a value at address 0x00000000 you can read it back at 0x00200000, 0x00400000 and 0x00600000. If we replace this mirrors with actual RAM we change that behavior, and that may introduce compatibility issue with weird/bugged games. I actually don't know if such games exist, but I don't know that they don't exist.

So this chance needs to be configurable, which makes it a bit trickier, especially when it comes to savestates and maybe the various dynarecs.

simias avatar Apr 01 '20 14:04 simias

I am also in favor of the memory limitation being updated to 8mb. Crash Team Racing has a lot of room for potential if it has a little more memory (removed features, modding potential). Every other PS1 game in existence will have a potential ability for modding if there is extra room to store data

The problem is that we can't just increase the memory by default because it would reduce the accuracy of the emulator. On real Playstations what they seem to have done is wire the memory controller the same way as on the 8MB prototypes but with the two most significant bits of the address bus "in the air".

In practice that means that the 2MB of RAM are mirrored 4 times on the first 8MB of the address space. So for instance if you write a value at address 0x00000000 you can read it back at 0x00200000, 0x00400000 and 0x00600000. If we replace this mirrors with actual RAM we change that behavior, and that may introduce compatibility issue with weird/bugged games. I actually don't know if such games exist, but I don't know that they don't exist.

So this chance needs to be configurable, which makes it a bit trickier, especially when it comes to savestates and maybe the various dynarecs.

This feature can be added to the menu as an option. We have accomplished this with a PCSXR that has now been modified. (Emulator is slow because it was builded in debug mode)

https://www.youtube.com/watch?v=KO3ayiXBB5w

barisyild avatar Apr 01 '20 14:04 barisyild

Could there be an error in the command I compile because many similar errors occur. (Compiles the .so library without problems)

It's very much possible but I only ever build for linux so I really don't know anything about your build environment. I know that many devs here use Windows however, hopefully they'll be able to help better.

simias avatar Apr 01 '20 14:04 simias

Could there be an error in the command I compile because many similar errors occur. (Compiles the .so library without problems)

It's very much possible but I only ever build for linux so I really don't know anything about your build environment. I know that many devs here use Windows however, hopefully they'll be able to help better.

Finally I managed to compile! After the memory is 8mb, a value repeats 4 times in memory and 8mb space cannot be used.

image

When I increased the PCSX-R memory to 8MB, I was able to use all the memory and in no way the same value was found in memory 4 times.

barisyild avatar Apr 01 '20 22:04 barisyild

Could you share the test you're running?

simias avatar Apr 02 '20 11:04 simias

Could you share the test you're running?

pcsxr-master-8mb-source.zip

*Emulator requires plugin you can download and use the appropriate plugins from the internet.

Also you can verify ram with following ps1 executable. RAMTEST.zip

And you may patch CTR (SCUS_944.26) for 8MB ram usage (ISO Patch Recommended)

DC 01 0C 00 00 00 00 D0 F9 00 0C 20 DC 01 0C 00 00 00 00 D0 F9 00 0C 80

barisyild avatar Apr 02 '20 14:04 barisyild

@barisyild See my comment above about compiling without HAVE_LIGHTREC=1, as that uses a different memory allocation method that has not been changed to support 8MB

ZachCook avatar Apr 05 '20 20:04 ZachCook

@barisyild See my comment above about compiling without HAVE_LIGHTREC=1, as that uses a different memory allocation method that has not been changed to support 8MB

Do you want me to compile with HAVE_LIGHTREC = 0?

barisyild avatar Apr 05 '20 20:04 barisyild

Yes, or just not setting it at all, as 0 is default in makefile I think, You currently need to if you want to test the 8MB changes

ZachCook avatar Apr 05 '20 20:04 ZachCook