Velocidex
Crash on Windows 11 24H2 ESE db file
:wave: Hello, I just wanted to open a small issue I recently encountered when reviewing an ESE database on Windows 11 version 24H2.
I get a crash when I try to run the go-ese example binary, using the latest commit:
eseparser.exe catalog --long_values C:\Users\bob\Desktop\qmgr.db
panic: runtime error: makeslice: len out of range
goroutine 1 [running]:
www.velocidex.com/golang/go-ese/parser.(*LongValue).Buffer(0xc000351540)
C:/Users/bob/Downloads/go-ese/parser/long_values.go:18 +0x45
www.velocidex.com/golang/go-ese/parser.(*Catalog).Dump(0xc0000083d8, {0x0?, 0x0?, 0x0?})
C:/Users/bob/Downloads/go-ese/parser/catalog.go:857 +0xa19
main.doCatalog()
C:/Users/bob/Downloads/go-ese/bin/catalog.go:28 +0xa5
main.init.0.func1({0xe5bc7c, 0x7})
C:/Users/bob/Downloads/go-ese/bin/catalog.go:39 +0x51
main.main()
C:/Users/bob/Downloads/go-ese/bin/main.go:32 +0x167
I tried an older Windows 11 ESE db and did not have any issues
eseparser.exe catalog --long_values "C:\Users\bob\Projects\artemis\core\tests\test_data\windows\ese\win11\qmgr.db"
[MSysObjects] (FDP 0x4):
Columns
0 ObjidTable Signed long Flags 1
1 Type Signed short Flags 1
2 Id Signed long Flags 1
3 ColtypOrPgnoFDP Signed long Flags 1
4 SpaceUsage Signed long Flags 1
5 Flags Signed long Flags 1
6 PagesOrLocale Signed long Flags 1
7 RootFlag Boolean Flags 0
8 RecordOffset Signed short Flags 0
9 LCMapFlags Signed long Flags 0
10 KeyMost Unsigned short Flags 0
11 LVChunkMax Signed long Flags 0
12 Name Text Flags 1
13 Stats Binary Flags 0
14 TemplateTable Text Flags 0
15 DefaultValue Binary Flags 0
16 KeyFldIDs Binary Flags 0
17 VarSegMac Binary Flags 0
18 ConditionalColumns Binary Flags 0
19 TupleLimits Binary Flags 0
20 Version Binary Flags 0
21 SortID Binary Flags 0
22 CallbackData Long Binary Flags 0
23 CallbackDependencies Long Binary Flags 0
24 SeparateLV Long Binary Flags 0
25 SpaceHints Long Binary Flags 0
26 SpaceDeferredLVHints Long Binary Flags 0
27 LocaleName Long Binary Flags 0
Indexes
Id:
Name:
RootObjects:
....
My own Rust based ESE parser had errors when reading these newer ESE db files. It looks other parsers also had issues with the Windows 24H2 release.
I'm not 100% sure what is causing go-ese to crash, but I was able to fix what was triggering errors in my parser (I was not correctly handling large AvailablePageTags). But I'm unsure if that is what is happening here too (based on error I don't think so?).
Attached the qmgr.db file that triggers the crash qmgr.zip
I used go-ese a lot when developing my own ESE parser, so I just wanted to give you a heads up about this issue. Let me know if more info is required
Thanks for reporting this. It seems to be a bug where the AvailablePageTag value is unusually large. Then when we parse the tags we run into the values. I added some extra checks for this condition. I suspect the AvailablePageTag is actually added with some flags - can you please elaborate on what your fix was like to your parser?
After these checks the parser returns a lot more data but maybe some of this data was removed from the table but the tags are still there? This may not be a bad thing to be able to read deleted values.
I tried to read the file you provided with nirsoft esedatabaseview and it crashed. I then upgraded to the latest and it didnt crash but only showed one row for the Files and Jobs table but now with the fix in place I can see many rows for this table. It is possible these rows are deleted which is why they are not shown by the nirsoft tool - it is kind of useful to see them though.
My fix was similar, adding check for large tag data sizes. Though i only get one row back similar to the esedatabaseview tool. Thats cool ur getting more data back!
It might be deleted rows? qmgr.db is used for BITS Jobs. Deleted Jobs can definitely be found/recovered from the file.
These are the extra rows I'm getting in that file
https://github.com/Velocidex/go-ese/blob/master/fixtures%2FWindowsQmgr.golden#L50
They might be rubbish so I'm not sure but they look reasonably consistent. Maybe deleted?
They appear to be valid BITS components. Some data can be extracted from it
FileInfo { file_id: "", filename: "wct4A28.tmp", full_path: "C:\\Users\\bob\\AppData\\Local\\Temp\\wct4A28.tmp", tmp_fullpath: "C:\\Users\\bob\\AppData\\Local\\Temp\\BIT4A28.tmp", drive: "C:\\", volume: "\\\\?\\Volume{537fb036-a3cd-47ab-9a2c-4ba87cc2c588}\\", url: "https://g.live.com/1rewlive5skydrive/OneDriveProductionV2?OneDriveUpdate=1b0712196fafba9acc43c1e5bbc8", download_bytes_size: 1739, transfer_bytes_size: 1739, files_transferred: 0 }
Maybe some of the tags are defunct?