ghidra
ghidra copied to clipboard
NationalSecurityAgency
PDB parsing error - PrimitiveTypeApplier seen where AbstractComplexTypeApplier expected for record number
Ghidra 10.3.3 and 10.4 (I haven't tried any other versions) are having problems parsing my PDB file, but WinDBG loads it correctly.
Any tips?
PDB Universal> Issue processing PDB file: c:\symbols\pa\pavlov.pdb\441BA19C80994975BEF464CC285080531\pavlov.pdb:
ghidra.app.util.bin.format.pdb2.pdbreader.PdbException: PrimitiveTypeApplier seen where AbstractComplexTypeApplier expected for record number TYPE[0]
The PDB file is 54MB but I only see 27MB being read, if I look via Procmon. This might be a red herring, but thought I would point it out in case there is some size limitation on PDBS.
And here are some other logs from application.log:
2023-11-24 10:28:09 DEBUG (SymbolServerService) SymbolServerService: querying c:\symbols for pavlov.pdb, 441ba19c-8099-4975-bef4-64cc28508053, 1, 0, ???
2023-11-24 10:28:09 DEBUG (SymbolServerService) SymbolServerService: got 1 results from c:\symbols
2023-11-24 10:28:09 DEBUG (SymbolServerService) SymbolServerService: found 1 matches
2023-11-24 10:28:09 DEBUG (SymbolServerService) SymbolServerService: getting symbol file: c:\symbols\pa\pavlov.pdb\441BA19C80994975BEF464CC285080531\pavlov.pdb
2023-11-24 10:28:09 DEBUG (SymbolServerService) SymbolServerService: local file now: c:\symbols\pa\pavlov.pdb\441BA19C80994975BEF464CC285080531\pavlov.pdb
2023-11-24 10:28:09 INFO (PdbUniversalAnalyzer) PDB analyzer parsing file: c:\symbols\pa\pavlov.pdb\441BA19C80994975BEF464CC285080531\pavlov.pdb
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_SHQUERYRBINFO
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/tagMETAHEADER
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/tWAVEFORMATEX
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/waveformat_tag
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_IMAGE_LOAD_CONFIG_DIRECTORY64
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/tagPDW
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_SHFILEOPSTRUCTA
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_SHFILEOPSTRUCTW
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_IMAGE_RELOCATION
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/tagBITMAPFILEHEADER
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/std/basic_streambuf<wchar_t,std::char_traits<wchar_t>_>
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/std/basic_streambuf<char,std::char_traits<char>_>
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/std/error_category
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/tagPDA
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_IMAGE_AUX_SYMBOL/<unnamed-type-Section>
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/DLGTEMPLATE
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/DLGITEMTEMPLATE
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/MiniZipCloseArchiveHelper
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/ZipAdapter
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/std/_Ref_count_base
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/Logger
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/Counter
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/std/_Pad
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/avAvatar/ScanIpcMsg
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/Buffer
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/boost/detail/sp_counted_base
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/ZipAdapter
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/Buffer
2023-11-24 10:28:11 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/CommonError
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/DebugHelper
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/AtomS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/CalDAVS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/CertMgr
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/FileMailerS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/FTPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/HTMLMailerS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/HTTPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/IMAPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/IPDaemonS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/IPPortS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/JSONS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/LDAPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/NNTPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/POPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/RESTS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/RSSS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/SMPPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/SMTPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/SNPPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/SOAPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/TelnetS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/WebDAVS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/WebFormS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/WebUploadS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/XMPPS
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/B9QNode
2023-11-24 10:28:12 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/RuleList
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/ATL/_stdcallthunk
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/B9StreamWriter
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_MINIDUMP_IO_CALLBACK
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_SHChangeDWORDAsIDList
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/Channel
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/B9Serializable
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/ATL/COleDateTimeSpan
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/ATL/COleDateTime
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/DirectoryRecord
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_Cb7ZipArchiveInfo
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_lb_addr
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/ZIP_HEADER
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/boost/asio/detail/win_thread/func_base
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/YR_ARENA_FILE_BUFFER
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_RTL_SCALABLE_MRSW_LOCK_COUNTER
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_PPL_LOOKASIDE_LIST_ARRAY
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_USB_ENDPOINT_DESCRIPTOR
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/_USB_CONFIGURATION_DESCRIPTOR
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/Mem_Range_s
2023-11-24 10:28:13 WARN (CppCompositeType) PDB STRUCTURE reconstruction failed to align /pavlov.pdb/IO_Des_s
To Reproduce Steps to reproduce the behavior: 1.Setup up PDB symbol server 2. Analyze file with Universal PDB
Expected behavior PDB data applied to functions so that I can see the actual function names.
For example, WinDBG does not have a problem:
0:000> lm
start end module name
00d20000 013ba000 Pavlov (private pdb symbols) c:\symbols\Pa\Pavlov.pdb\441BA19C80994975BEF464CC285080531\Pavlov.pdb
Screenshots If applicable, add screenshots to help explain your problem.
Attachments If applicable, please attach any files that caused problems or log files generated by the software.
Environment (please complete the following information):
- Windows 11 23H2 - Microsoft Windows [Version 10.0.22621.2715]
- Java Version: 17.0.4.101
- Ghidra Version: 10.3.3, 10.4
- Ghidra Origin: official Github distro
- Compiler version: VS 2015, Update 3
Additional context Add any other context about the problem here.
I'd like to understand the PDB situation that is causing this, but we did not provide enough details or a trace with the error. I believe that the the issue is arising from when trying to define the containing class of the class/struct it is processing, but there is not containing class (a NOTYPE).
There are a few possible short-term solutions to work around this issue. Ultimately, I believe this code will be changing/different for the next major release, but I can try to introduce a fix in a patch release if I can identify a solution.
I understand that you are using a pre-built distribution, but if you can build from source you could make a change: AbstractComplexTypeApplier has a method getComplexApplier around line 37. It makes a call to getApplierSpec, but should probably be calling getApplierOrNoTypeSpec instead. If you could test this, then I could try to get this into a patch release.
You could use PDB MSDIA Analyzer instead of the PDB Universal Analyzer, but this also means you will need to make sure the msdia140.dll is registered in your Windows system. The the Ghidra pdb.exe native application is built with VS 2017, I believe the msdia140.dll that comes with VS 2015 should work. Read the Ghidra help regarding use of the PDB MSDIA and you should also find a README_PDB.html document in one of the Ghidra distribution document directories, which should tell you how to register the DLL.
You could load just the PDB public symbols, but you will only be able to get data type information from mangled symbols (from the Demangler) and not rich PDB datatype information. To do this, instead of using the PDB Universal Analyzer, you would instead do File->Load PDB..., select the PDB and for the "Control" option, select "Public Symbols Only." Then run other analysis other than PDB Universal.
Thank you very much for the suggestions. I will try suggestion 2 at the moment. I don't think I have the bandwidth to build Ghidra at the moment.
Sounds good. I believe that solution (1) is OBE in the master branch with a recent code push (d4861c46ac6bcb30e79da483c66b5e78fca47abc), and hopefully your issue is not seen again when that release is made.
I registered the VS2015 msdia140.dll and things definitely work better. I do see a lot of symbols. Though, I still do see some error messages, like:
Problem parsing or applying PDB information: org.xml.sax.SAXParseException; lineNumber: 1397161; columnNumber: 109; XML document structures must start and end within the same entity.
java.io.IOException: Problem parsing or applying PDB information: org.xml.sax.SAXParseException; lineNumber: 1397161; columnNumber: 109; XML document structures must start and end within the same entity.
at ghidra.app.util.bin.format.pdb.PdbParser.applyTo(PdbParser.java:436)
at ghidra.app.plugin.core.analysis.PdbAnalyzer.parsePdb(PdbAnalyzer.java:109)
at ghidra.app.plugin.core.analysis.PdbAnalyzer.added(PdbAnalyzer.java:95)
at ghidra.app.plugin.core.analysis.AnalysisScheduler.runAnalyzer(AnalysisScheduler.java:186)
at ghidra.app.plugin.core.analysis.AnalysisTask.applyTo(AnalysisTask.java:39)
at ghidra.app.plugin.core.analysis.AutoAnalysisManager$AnalysisTaskWrapper.run(AutoAnalysisManager.java:686)
at ghidra.app.plugin.core.analysis.AutoAnalysisManager.startAnalysis(AutoAnalysisManager.java:786)
at ghidra.app.plugin.core.analysis.AutoAnalysisManager.startAnalysis(AutoAnalysisManager.java:665)
at ghidra.app.plugin.core.analysis.AutoAnalysisManager.startAnalysis(AutoAnalysisManager.java:630)
at ghidra.app.plugin.core.analysis.AnalysisBackgroundCommand.applyTo(AnalysisBackgroundCommand.java:58)
at ghidra.framework.plugintool.mgr.BackgroundCommandTask.run(BackgroundCommandTask.java:102)
at ghidra.framework.plugintool.mgr.ToolTaskManager.run(ToolTaskManager.java:334)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1397161; columnNumber: 109; XML document structures must start and end within the same entity.
at ghidra.xml.ThreadedXmlPullParserImpl.checkForException(ThreadedXmlPullParserImpl.java:122)
at ghidra.xml.ThreadedXmlPullParserImpl.waitForNextElement(ThreadedXmlPullParserImpl.java:168)
at ghidra.xml.ThreadedXmlPullParserImpl.hasNext(ThreadedXmlPullParserImpl.java:154)
at ghidra.app.util.bin.format.pdb.ApplySymbols.applyTo(ApplySymbols.java:54)
at ghidra.app.util.bin.format.pdb.ApplyTables.applyTo(ApplyTables.java:42)
at ghidra.app.util.bin.format.pdb.PdbParser.applyTo(PdbParser.java:404)
... 12 more
Caused by: org.xml.sax.SAXParseException; lineNumber: 1397161; columnNumber: 109; XML document structures must start and end within the same entity.
at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1251)
at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:637)
at ghidra.xml.ThreadedXmlPullParserImpl$ContentHandlerRunnable.run(ThreadedXmlPullParserImpl.java:267)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
... 1 more
---------------------------------------------------
Build Date: 2023-Sep-28 1301 EDT
Ghidra Version: 10.4
Java Home: C:\Program Files\Eclipse Adoptium\jdk-17.0.4.101-hotspot
JVM Version: Eclipse Adoptium 17.0.4.1
OS: Windows 11 10.0 amd64
Those messages might be due to limitation of SAX parser. See totalEntitySizeLimit in #1207.
I’d still like your feedback of whether the PDB Universal of the next Ghidra release works for you.
@snowkoan Release 11.0 went out late December. It is my expectation that if you try PDB Universal in that release, you should no longer observer the error that you initially reported in this ticket.
I believe this issue will no longer occur due to d4861c46ac6bcb30e79da483c66b5e78fca47abc which makes the issue OBE. If you have issues with the new code, please open a new ticket.
Hello, sorry for the late notice. I tested with Ghidra 11.0.1 and the PDB does load fine now. Thanks for your help!