text issue: Diz assembly outputs characters in strings that Asar can't handle

[binary1230]

this might be an asar bug but, we can fix it in Diz.

in exported assembly marked as text, there can be stuff like null chars etc in the quoted string,

fix is likely to breakup the strings and output those characters as individual bytes when we detect them.

textttttttttttttt!!!!!!!!!!!!!

[binary1230]

CC @PlazmaCube

[binary1230]

example of extra stuff in the quoted string:

even though maybe some of these chars aren't actually text and might be part of a data table or something, Diz should be able to handle it gracefully (certainly the nulls, at minumum)

we could separately also break up text spacing based on the nulls so it's a little more human-friendly

forgot, CC @rdrpenguin04 as well

[binary1230]

here's the research:

1. The actual error is I think Asar is bugging out on just the null characters in the string, which, Diz really should be escaping. So I think that'll be the real fix on our end.
2. There's still some niceties Diz could do a better job with splitting up lines of text etc.

Here's the quoted string parsing code in Asar, with all the macros expanded. the lines marked with "!!!!!" comments are the errors we're [legitimately] hitting:

if (!confirmquotes(line)) { 
  thisline = i; thisblock = line; asar_throw_error(0, error_type_null, error_id_mismatched_quotes); line[0] = '\0'; 
}

confirmquotes() calls dequote() here: https://github.com/RPGHacker/asar/blob/f428b1631d3b1e7b921dfc0b5f14b4851ee3b5bb/src/asar/libstr.cpp#L297

and this is what that function looks like expanded:

bool confirmquotes(const char * str)
{
	for (int i=0;str[i];)
	{
		if (str[i]=='"')
		  do { 
			 i++; 
			 while (str[i]!='"') { 
			   if (!str[i]) 	// <----- bails if NULL. this is our issue
				   return false;
			   i++; 
			 } 
			 i++; 
		  } while(0); 
		else if (str[i]=='\'') 
		  do { 
			i++; 
			while (str[i]!='\'') { 
			  if (!str[i]) 		// <----- bails if NULL. this is our issue
				  return false;
			  i++;
			} 
			i++; 
		  } while(0)
		else 		// external
			i++;	// external
	}
	return true;
}

I think if Diz looks full null chars and outputs 'db $00' after any string previously containing them, we should be good here.

[PlazmaCube]

Hm, looks pretty complicated. Hope debugging it works out!

[binary1230]

following up, i believe I "fixed" this in the Diz output with: https://github.com/DizTools/Diz.LogWriter/commit/e93214fd6d13df279b371ca9a7b45e3b9f59d107 and https://github.com/DizTools/Diz.LogWriter/commit/a7d32209266b9880494940b90c4d6193a9fcb5a0

basically, Diz won't try and output strings with certain special chars in them anymore (like null, \r, \n) and will just stop printing the string and insert a byte into the disassembly with those special chars. then it will resume printing the string if it can.

I'm going to mark this closed for now but please re-open if anything is still an issue