32-bit codeview/.pdb local variable address is off by 4

[mirek-fidler]

This is a little bit hard to demonstrate by a testcase, so instead I am posting workaround I have used to make our debugger work with llvm-mingw toolchain for 32-bit outputs (64-bit does not seem to be affected) - look at line with "// Workaround for supposed clang/win32 issue" comment:

struct Pdb::LocalsCtx {
	adr_t                       frame;
	VectorMap<String, Pdb::Val> param;
	VectorMap<String, Pdb::Val> local;
	Pdb                        *pdb;
	Context                    *context;
};

BOOL CALLBACK Pdb::EnumLocals(PSYMBOL_INFO pSym, ULONG SymbolSize, PVOID UserContext)
{
	LocalsCtx& c = *(LocalsCtx *)UserContext;

	if(pSym->Tag == SymTagFunction)
		return TRUE;

#if 1
	DLOG("------");
	DDUMP(pSym->Name);
	DDUMPHEX(pSym->Flags);
	DDUMP(pSym->Flags & IMAGEHLP_SYMBOL_INFO_PARAMETER);
	DDUMP(pSym->Flags & IMAGEHLP_SYMBOL_INFO_REGISTER);
	DDUMP(pSym->Flags & IMAGEHLP_SYMBOL_INFO_REGRELATIVE);
	DDUMP(pSym->Flags & IMAGEHLP_SYMBOL_INFO_FRAMERELATIVE);
	DDUMP(pSym->Register == CV_ALLREG_VFRAME);
	DDUMP(pSym->Register);
	DDUMP(pSym->Scope);
	DDUMP(pSym->Value);
	DDUMP(pSym->ModBase);
	DDUMPHEX((adr_t)pSym->Address);
#endif

	bool param = pSym->Flags & IMAGEHLP_SYMBOL_INFO_PARAMETER;
	Val& v = (param ? c.param : c.local).Add(pSym->Name);
	v.address = (adr_t)pSym->Address;
	if(pSym->Flags & IMAGEHLP_SYMBOL_INFO_REGISTER)
		v.address = pSym->Register;
	else
	if(pSym->Flags & IMAGEHLP_SYMBOL_INFO_REGRELATIVE) {
		if(pSym->Register == CV_ALLREG_VFRAME) {
		#ifdef CPU_64
			if(c.pdb->win64)
				v.address += c.pdb->GetCpuRegister(*c.context, CV_AMD64_RBP);
			else
		#endif
				v.address += (adr_t)c.pdb->GetCpuRegister(*c.context, CV_REG_EBP) - 4 * c.pdb->clang; // Workaround for supposed clang/win32 issue
		}
		else
			v.address += (adr_t)c.pdb->GetCpuRegister(*c.context, pSym->Register);
	}
	else
	if(pSym->Flags & IMAGEHLP_SYMBOL_INFO_FRAMERELATIVE)
		v.address += c.frame;
	
	LLOG("LOCAL " << pSym->Name << ": " << Format64Hex(v.address));
	c.pdb->TypeVal(v, pSym->TypeIndex, (adr_t)pSym->ModBase);
	if(param && v.udt && v.ref == 0 && c.pdb->win64) { // dbghelp.dll incorrectly does not report pointer for (copied) value struct params
		v.ref++;
		v.reference = true;
	}
	v.reported_size = pSym->Size;
	v.context = c.context;
	return TRUE;
}

Not sure whether this is my misunderstanding of dbghelp.dll API, but as far as I can see toolchain output is simply off by 4 bytes. With above workaround, debugger works fine for .pdb/code output from Visual C++ compiler as well as llvm...

Not sure this is the right place to report this, but hopefully you will know what to do :)

[mirek-fidler]

Compiler commandline:

C:\upp\bin/clang\bin\i686-w64-mingw32-c++.exe -c  -I"C:\u\upp.src\upptst" -I"C:\u\upp.src\uppsrc" -I"C:\upp/bin/clang\include" -I"C:\upp\bin\SDL2\include" -I"C:\upp\bin2\openssl\include" -I"C:\upp\bin\mysql\include" -I"C:\upp\bi
    n/pgsql/x86/include" -I"C:/upp/out/upptst/PdbTests/CLANG.Debug.Debug_Full.Gui.Main" -DflagGUI -DflagMAIN -DflagCLANG -DflagDEBUG -DflagDEBUG_FULL -DflagBLITZ -DflagWIN32 -mthreads -g2 -static   -fexceptions -D_DEBUG -O0 -g -
    gcodeview -fno-limit-debug-info -x c++ "C:\u\upp.src\upptst\PdbTests\main.cpp"  -o "C:/upp/out/upptst/PdbTests/CLANG.Debug.Debug_Full.Gui.Main\main.o"

[mstorsjo]

Well, I don't really know the PDB internals at all, neither do I know dbghelp.dll, so I don't really have much to add at this point. I'd suggest filing a bug at https://bugs.llvm.org, then hopefully someone else who knows about the PDB generation can help.

Does the issue disappear/change if you compile with or without optimization? Are you able to test the same code with clang in msvc mode, if it works better there? (I've been told that at least the Visual Studio IDE/debugger makes a few assumptions about the C++ ABI that doesn't hold up in the itanium ABI used in mingw mode, but I'm not sure if that affects things if you're just using dbghelp.dll.)

[mirek-fidler]

A little update for future reference / anybody else who could deal with this, after playing with this a bit more, I think the problem is this. The start of 32 bit compiled function looks

004018D0  push ebp 
004018D1  mov ebp,esp  <-----
004018D3  push esi 
004018D4  and esp,byte -0x8    <-----
004018D7  sub esp,0xc80 
004018DD  mov eax,esp 

I believe that the issue is that compiler emits debug info relative to aligned value of esp, while in ebp register is still unaligned value (in the function code, clang then always uses esp based adresses). So the final fix in our debugger looks like this:

			{
				adr_t ebp = (adr_t)c.pdb->GetCpuRegister(*c.context, CV_REG_EBP);
				if(c.pdb->clang)
					ebp &= ~(adr_t)7;  // Workaround for supposed clang/win32 issue
				v.address += ebp;
			}