SHA-1 of VDF converted to text in v28 and v29

[romatthe]

Thanks so much for this repo, it has helped me out quite a bit in dealing with the AppInfo format.

Sorry for bothering with such a seemingly silly question: but there's one thing I still can't quite figure out. How is the SHA-1 hash of the 6th AppSection field is generated? The readme states:

20bytes  - SHA1 // of text appinfo vdf, as seen in CMsgClientPICSProductInfoResponse.AppInfo.sha

This leads me to a few questions: I assume this means that the binary vdf blob (the 9th field in the AppSection format) should be converted to a text representation, and then the SHA-1 hash of that text representation should be calculated.

However:

• Is there a description somewhere of the text-based format?
• What text encoding is used?
• Is this a tightly packed text representation or is it pretty printed? With indentation? Spaces or tabs as indentation? How many spaces, how many tabs?
• I assume this means that the strings from the string table (at the end of the AppInfo file) should be used in the text representation?

More importantly, what is this mysterious CMsgClientPICSProductInfoResponse.AppInfo.sha that's being referred to here? I genuine can't figure out what this is supposed to refer to.

I've been scouring the web for some information on any of this, but I can only find scattered pieces of code, most of it from previous version of the format without the hashes. If there's some example code somewhere or a bit of documentation, that would be really cool.

Thanks!

[xPaw]

CMsgClientPICSProductInfoResponse is the protobuf when Steam client requests appinfo from the server. That SHA is used for downloading the appinfo from CDN via http. SteamKit has code for this.

[ymgve]

CMsgClientPICSProductInfoResponse.AppInfo.sha is the SHA1 hash of the raw CMsgClientPICSProductInfoResponse.AppInfo.buffer in the same message, excluding the last byte, which is always \x00

Likewise, CMsgClientPICSProductInfoResponse.PackageInfo.sha is the SHA1 hash of the raw CMsgClientPICSProductInfoResponse.PackageInfo.buffer, excluding the first four bytes which seem to always be \x01\x00\x00\x00

(This assumes CMsgClientPICSProductInfoRequest.meta_data_only is False)

[romatthe]

Thank you both for clarifying. I was completely unaware this was referring to the PB definitions.

One final question: I quickly whipped up some code over lunch to convert the binary VDF format to a textual VDF representation, so I could then calculate the correct SHA1 of that textual representation. The end goal is to be able to alter the AppInfo VDF and write back the correct checksum.

Right now, without altering the VDF contents, I am able to correctly generate the textual VDF representation for about 500 of the 5500 apps in my local appinfo.vdf. (In other words, when I generate the textual VDF representation of a binary VDF, and subsequently take the SHA1 hash of those, I get a result that matches the entry in my local appinfo.vdf for 500 of the 5500 entries.).

It's hard to see any immediate issues with my code, and the results look reasonable, so it's extremely hard to debug. Is there a tool out there that can correct convert a binary VDF to a textual VDF representation? That way I could diff that against my own results and perhaps successfully debug some of it.

Again, sorry with bothering you this, but I simply cannot find any more concrete information anywhere.

[xPaw]

I doubt you're expected to get correct hash out, why aren't you using the second hash?

[romatthe]

The end goal is to be able to alter the AppInfo VDF and write back the correct checksum.

When altering the the VDF contents, I'd need to regenerate both the first and the second hash. The hash based on the binary VDF is very easy, of course. The first one is obviously the one I'm struggling so much with.

Perhaps some additional clarification because I might be seriously confusing you, for which I apologize. The end goal is to alter the appinfo.vdf for the local Steam installation (the one in appcache/) to alter the way the titles of games are displayed and sorted in the Steam client. So I want to put the appinfo.vdf back together in a way that Steam will accept (though I cannot confirm if the Steam client actually checks these).

This was easy to do with the v27 format and worked reasonably well, but the v28 and v29 format introduced the additional complexities with the hashes.