dolphin-memory-engine icon indicating copy to clipboard operation
dolphin-memory-engine copied to clipboard

[NEEDS REBASE] Add support for other string encodings

Open campital opened this issue 4 years ago • 8 comments

This is a modification of the memory engine to support UTF-16 and UTF-32 strings.

Here is a tentative list of changes:

  • Can now search and display other string encodings
  • Buttons to select between UTF-8, UTF-16, and UTF-32 in the search properties
  • Support for encodings in the search result layout
  • Support for encodings in the memory watch layout
  • New JSON key "stringWidth" that has an int value
  • Context menu for switching between string encodings in MemWatchEntrys
  • New drop-down menu in DlgAddWatchEntry

I could have missed some other part of integration (I did notice the CSV export but it looks like it doesn't currently save some other watch properties), but I think I got most of it.

Edit: I am unable to test this on Windows, and I also used wstring_convert, which will be deprecated soon (I didn't know if an external library or a homemade implementation would be appropriate).

campital avatar Jul 25 '20 02:07 campital

The build failure seems to be caused by a bug in MSVC. I think I'll have to drop codecvt and wstring_convert and use something else.

campital avatar Jul 25 '20 02:07 campital

Update: I have just successfully compiled my latest commit and tested it on Visual Studio 2019 with Windows SDK 10.0.18362.0. I think the compilation errors in the CI are due to a bug in earlier MSVC versions.

campital avatar Jul 31 '20 02:07 campital

I have rewritten the unicode converter so that it does not use the deprecated header. I have also tested my changes on Linux and MSVC Windows and AppVeyor is now successfully building.

campital avatar Aug 01 '20 04:08 campital

If I ever port this to dolphin, I will consider this, but because it's been so long, I can't properly review this until I get back to this project, sorry, but I do like the features!

aldelaro5 avatar Aug 05 '20 20:08 aldelaro5

Ok so I did some tests and I found 2 issues:

  • the add watch dialogu doesn't offer me the option to change the encoding when I select the string type
  • The length shown for the string doesn't represent reality: if I have a string where the width per char is 4 bytes, the watcher shows me string[4], but it's actually string[1], 4 is just the number of bytes the string takes.

These have to be fixed and I just got 2 major bugfix and a requested pr in so I can't put this in the next release meaning this will have to wait when I have free time to look at BOTH this pr and the macos one (because tbf, that one I need to do a lot of tests that are a bit tedious).

So correct these and I might recheck the pr later, it did seemed to work at first glance which is good :D

aldelaro5 avatar Mar 12 '21 06:03 aldelaro5

How does everything look now?

I modified most of the memory manipulation code to interpret the length parameter as an array length rather that the size in bytes (raw memory operations were not changed, though). This makes it easier to have a UTF-16 string with a length of 1, while it actually uses 2 bytes of memory.

The memory watch entry GUI has also been updated to show 'utf8[len]", "utf16[len]", and "utf32[len]" rather than "string[len]". Also, you can now input the string length in DlgAddWatchEntry.

campital avatar Mar 21 '21 22:03 campital

@campital I know this is very late, but if you rebase your changes I'll look into getting this in.

dreamsyntax avatar Oct 31 '23 23:10 dreamsyntax

Ultimately decided this will not be present in the final Qt 5 build. I will take a look at this again post Qt 6 release.

dreamsyntax avatar Nov 02 '23 23:11 dreamsyntax