coh2stats icon indicating copy to clipboard operation
coh2stats copied to clipboard

Questions about getting data from api and interpreting it

Open liaoyuzh opened this issue 3 years ago • 6 comments

So far I only know these api methods from a forum https://pastebin.com/raw/8CNjUJA8. However, I am more curious about the other methods available to users. Where can I find documentation, list of methods etc to request data by myself?

In addition, if available, I want to know if the following data are available for each match:

Also, by inspecting a match info data, I notice the

options: "eNozYPvGAAGMDEyMIHpj9qa5gt18FlBhBjlGORCJB4OAEBCbFMQXJWbmFMcn5qXE56aWJOYwDAYAAHUmDME=",
slotinfo:
    "eNrjYWBg4GB4tVOGAQQYGTABM4zxHwpA7BdAtYwgDiMDJ2Om60QOkGBxMCtYIYQESlRAJXRvMKFKLL4iLgliZTxnQ5UohOrQQ9eRAZVg+8KMKhEOlZiLLlEJldiCKsGIAhEA5jN0MGWfOkowMCIFExO2oBFjRAqaV+15/FiD5gVUwhjdoy+hEkboEn+vHRUEsTjeo3mUoQOiAyNoWKASGEHDCJXAGjScjMI+k4XBPvFjRZfmZOSD6n38ACXqCAekYJgeaqAhAUa8aYyT0SfaDuz3DEgyRbiIkzHs2GawVOl1tPDygOo5LsqAKqEHlTiDLmEHlZAUR5OwgErIoEvk37IXAbHuiTNQksbEH1uCaRYsaYwBW9DoIqcx9znRIljTWBpU4gm6s3OgEs/QJTyhEt6f0VKMF1RCGl2HN1QCI2gCoRKSmEHDyWgckQCW7ENNRCApa0+IVJ44RlRnQ40kOen9RwNYxZAkGTGlGUnXQhtbAJrRNIc=",

are encoded in base64. But a regular base64 decoder doesnt interpret it correctly. It just doesnt make sense.

So in general, it would be greatly appreciated if you can provide some helps/documentation on such where I can get these data and how to interpret. Thanks!

liaoyuzh avatar Apr 25 '21 16:04 liaoyuzh

Hi Liaoyuzh

As mentioned this is not official web-site / not associated with Relic in any way. I don't have access to any documentation or anything else. I am not aware of more API requests than https://pastebin.com/raw/8CNjUJA8

getRecentMatchHistory
getLeaderBoard2

Are the 2 main API functions this site is build on. And as you might noticed, they are not ideal but that's what we need to work with - and I am doing a lot of data transformation after I get it from the Relic API.

My general approach to finding this stuff is to go at http://www.companyofheroes.com/leaderboards , and just "debug / reverse-engineer" how the site works / what goes in the requests and what comes back. And than just testing / finding the information on my own.

This topic might be interesting to you, as it might have some information which I was able to get https://www.coh2.org/topic/107278/coh2-global-stats

And than just go trough the source code I have here, I recommend this file, it should have all the API requests I am doing on the BE and also related information to the data which comes out. https://github.com/petrvecera/coh2ladders/blob/master/packages/functions/src/libs/coh2-api.ts

As for the data:

options: "eNozYPvGAAGMDEyMIHpj9qa5gt18FlBhBjlGORCJB4OAEBCbFMQXJWbmFMcn5qXE56aWJOYwDAYAAHUmDME=",
slotinfo:
    "eNrjYWBg4GB4tVOGAQQYGTABM4zxHwpA7BdAtYwgDiMDJ2Om60QOkGBxMCtYIYQESlRAJXRvMKFKLL4iLgliZTxnQ5UohOrQQ9eRAZVg+8KMKhEOlZiLLlEJldiCKsGIAhEA5jN0MGWfOkowMCIFExO2oBFjRAqaV+15/FiD5gVUwhjdoy+hEkboEn+vHRUEsTjeo3mUoQOiAyNoWKASGEHDCJXAGjScjMI+k4XBPvFjRZfmZOSD6n38ACXqCAekYJgeaqAhAUa8aYyT0SfaDuz3DEgyRbiIkzHs2GawVOl1tPDygOo5LsqAKqEHlTiDLmEHlZAUR5OwgErIoEvk37IXAbHuiTNQksbEH1uCaRYsaYwBW9DoIqcx9znRIljTWBpU4gm6s3OgEs/QJTyhEt6f0VKMF1RCGl2HN1QCI2gCoRKSmEHDyWgckQCW7ENNRCApa0+IVJ44RlRnQ40kOen9RwNYxZAkGTGlGUnXQhtbAJrRNIc=",

I would love to know what it is. The API is returning much more data than Relic displays on their website, I don't think they are processing this data. It looks like base64 but could be different encoding algorithm too. It might be binary/hex data directly transformed into base64.

I hope this helps, cheers

petrvecera avatar Apr 26 '21 09:04 petrvecera

@petrvecera That helps a lot! A few more questions:

  1. I'm able to get commander choices of each player from the getRecentMatchHistory api call, however I dont get access to which commander they lock in repectively in that game, right? Because I want take a look into how comander choice affects winratio in different time period.
  2. Whats the max api calls I can request? per day? what are the restrictions on that?

again, thanks for the help and very solid works of your project :)

liaoyuzh avatar Apr 27 '21 02:04 liaoyuzh

I'm able to get commander choices of each player from the getRecentMatchHistory api call, however I dont get access to which commander they lock in repectively in that game, right? Because I want take a look into how comander choice affects winratio in different time period.

Yes I was not able to find out this information. It could be in the slot info, idk :(

  1. idk, I've encountered errors 429 (too many requests) sometimes https://github.com/petrvecera/coh2ladders/issues/11 I never tried to push really hard on the API. In general my app is doing around ~5k requests/ day right now.

I don't want to cause issues on the Relic side - so they don't block this kind of access. Because this API is used also in-game. When you look at the player profiles and their matches. It calls this API.

One thing which I know is that the API is located in AWS Northern Virginia datacenter. So being close to that can also help in case you need to worry about the response time.

petrvecera avatar Apr 27 '21 12:04 petrvecera

@liaoyuzh

Those extra fields are zlib

"eNq9k0uTmkAQx/NZuK6J8hStymEAUVwRZMVXKocpGFARcAEfJJXvnjCD7lrM7jF9Ybrpnl91939YrvXjN3PM0mB3QEYSpN92PtPnZU7ocFyLyQtY7NLE0Jg+22IKBOPq2GkxAfTe/cigh6ojVx2TaILO6IDTKs+Ehbedl0ec8bW6ZhcjG2V6BmNkvuC8Xe4g6Jf4sop5ynE4RgXUYAGZPgMAUNqVXSLFkJOxDipTesJlDYi59ReMgcNOr9sZcexcq+P6LUGdDY6suCYFw3mihCSuzG4ZkTKaomVdUHLAoDDOq0PNMHuh2mSMrrzqLbAzygRwaTLM9vCVFGhnDVD62PjTsmZsrFynMERfSkmB4li3ux8YRb7XCduZh26TAduJWBeUFhhSZtXlSnNFPJHKUJ3upC5YUBmby2l89bAziARKHwP+vg/LDpv7eG8algH42L4zf1pNTbOsLEms+KBp7k3T7IOmO2+aFv6PpqOTW5A9qeHo3sp9hko157C3Pel4NtqJDyl6mRuLeEaW/eTLlF2OyuOTTDQ5DKi6t/dlpOK4iqiM1fpXERJG0hNoDGc5SU3s2BNEY7BTfanNSJSnvV/flZKLQETbo/Yhda2aodsBlbF/DTQcV56pjJf4uawf5Zw+K9c6ylYtOQpj8Kg6u1qk/Jks/+ny55e/Z7SMtQ=="

decoded

12,[{"profileInfo.id":3824022,"stationID":1,"teamID":0,"factionID":1,"raceID":2,"rankLevel":0,"rankMatchTypeID":-1,"timePerFrameMS":0,"isReady":1,"status":0,"metaData":"AAAB/////wkBI8nJFAAAAAB94wYAAAAAAAUAAAAAAAAJAR1NxhQAAAAAJPsDAAAAAAAFAAAAAAAACQEp15YUAAAAAGTnBgAAAAAABQAAAAAAAAkBHNeWFAAAAAAy2AIAAAAAAAUAAAAAAAAJARvXlhQAAAAAM9gCAAAAAAAFAAAAAAAACQHx3CcVAAAAAHr4AwAAAAAABQAAAAAAAAkBM/GqFAAAAADvDAYAAAAAAAUAAAAAAAAJAZdNyhQAAAAAZOsFAAAAAAAFAAAAAAAACQH5d6oUAAAAABROBQAAAAAABQAAAAAAAAkBMtsjFwAAAAARTgUAAAAAAAUAAAAAAAAJAa/n5BQAAAAAyOAGAAAAAAAFAAAAAAAACQE72yMXAAAAAA5OBQAAAAAABQAAAAAAAAkBCR7LFAAAAAAVTgUAAAAAAAUAAAAAAAAJAZwuJxcAAAAAEk4FAAAAAAAFAAAAAAAACQE315YUAAAAAOPgBgAAAAAABQAAAAAAAAAAAAAAAAAAAAD/////AAAAAAAAAAAAAAAAAAAAAAAAAAA="},{"profileInfo.id":1186615,"stationID":2,"teamID":1,"factionID":0,"raceID":4,"rankLevel":0,"rankMatchTypeID":-1,"timePerFrameMS":0,"isReady":1,"status":0,"metaData":"AAAB/////wkBkuUtFwAAAACgHAAAAAAAAAUAAAAAAAABAAkBg9huFgAAAADu3gYAAAAAAAUAAAAAAAAJATIVmQwAAAAA+d8GAAAAAAAFAAAAAAAACQHyp+8VAAAAAGfnBgAAAAAABQAAAAAAAAkBPjykCgAAAACe3gYAAAAAAAUAAAAAAAAJAXYztgwAAAAAn94GAAAAAAAFAAAAAAAACQHRWLoMAAAAAPLeBgAAAAAABQAAAAAAAAkB1NFWDQAAAABQ3wYAAAAAAAUAAAAAAAAJAdU6nw4AAAAAR98GAAAAAAAFAAAAAAAACQH67OoMAAAAAFPfBgAAAAAABQAAAAAAAAkB1jqfDgAAAABK3wYAAAAAAAUAAAAAAAAJASmKyAwAAAAATd8GAAAAAAAFAAAAAAAACQHUOp8OAAAAAD/fBgAAAAAABQAAAAAAAAEAAAAAAAAAAAAAAP////8AAAAAAAAAAAAAAAAAAAAAAAAAAA=="}]

petrvecera avatar Mar 08 '22 14:03 petrvecera

@liaoyuzh

Those extra fields are zlib

"eNq9k0uTmkAQx/NZuK6J8hStymEAUVwRZMVXKocpGFARcAEfJJXvnjCD7lrM7jF9Ybrpnl91939YrvXjN3PM0mB3QEYSpN92PtPnZU7ocFyLyQtY7NLE0Jg+22IKBOPq2GkxAfTe/cigh6ojVx2TaILO6IDTKs+Ehbedl0ec8bW6ZhcjG2V6BmNkvuC8Xe4g6Jf4sop5ynE4RgXUYAGZPgMAUNqVXSLFkJOxDipTesJlDYi59ReMgcNOr9sZcexcq+P6LUGdDY6suCYFw3mihCSuzG4ZkTKaomVdUHLAoDDOq0PNMHuh2mSMrrzqLbAzygRwaTLM9vCVFGhnDVD62PjTsmZsrFynMERfSkmB4li3ux8YRb7XCduZh26TAduJWBeUFhhSZtXlSnNFPJHKUJ3upC5YUBmby2l89bAziARKHwP+vg/LDpv7eG8algH42L4zf1pNTbOsLEms+KBp7k3T7IOmO2+aFv6PpqOTW5A9qeHo3sp9hko157C3Pel4NtqJDyl6mRuLeEaW/eTLlF2OyuOTTDQ5DKi6t/dlpOK4iqiM1fpXERJG0hNoDGc5SU3s2BNEY7BTfanNSJSnvV/flZKLQETbo/Yhda2aodsBlbF/DTQcV56pjJf4uawf5Zw+K9c6ylYtOQpj8Kg6u1qk/Jks/+ny55e/Z7SMtQ=="

decoded

12,[{"profileInfo.id":3824022,"stationID":1,"teamID":0,"factionID":1,"raceID":2,"rankLevel":0,"rankMatchTypeID":-1,"timePerFrameMS":0,"isReady":1,"status":0,"metaData":"AAAB/////wkBI8nJFAAAAAB94wYAAAAAAAUAAAAAAAAJAR1NxhQAAAAAJPsDAAAAAAAFAAAAAAAACQEp15YUAAAAAGTnBgAAAAAABQAAAAAAAAkBHNeWFAAAAAAy2AIAAAAAAAUAAAAAAAAJARvXlhQAAAAAM9gCAAAAAAAFAAAAAAAACQHx3CcVAAAAAHr4AwAAAAAABQAAAAAAAAkBM/GqFAAAAADvDAYAAAAAAAUAAAAAAAAJAZdNyhQAAAAAZOsFAAAAAAAFAAAAAAAACQH5d6oUAAAAABROBQAAAAAABQAAAAAAAAkBMtsjFwAAAAARTgUAAAAAAAUAAAAAAAAJAa/n5BQAAAAAyOAGAAAAAAAFAAAAAAAACQE72yMXAAAAAA5OBQAAAAAABQAAAAAAAAkBCR7LFAAAAAAVTgUAAAAAAAUAAAAAAAAJAZwuJxcAAAAAEk4FAAAAAAAFAAAAAAAACQE315YUAAAAAOPgBgAAAAAABQAAAAAAAAAAAAAAAAAAAAD/////AAAAAAAAAAAAAAAAAAAAAAAAAAA="},{"profileInfo.id":1186615,"stationID":2,"teamID":1,"factionID":0,"raceID":4,"rankLevel":0,"rankMatchTypeID":-1,"timePerFrameMS":0,"isReady":1,"status":0,"metaData":"AAAB/////wkBkuUtFwAAAACgHAAAAAAAAAUAAAAAAAABAAkBg9huFgAAAADu3gYAAAAAAAUAAAAAAAAJATIVmQwAAAAA+d8GAAAAAAAFAAAAAAAACQHyp+8VAAAAAGfnBgAAAAAABQAAAAAAAAkBPjykCgAAAACe3gYAAAAAAAUAAAAAAAAJAXYztgwAAAAAn94GAAAAAAAFAAAAAAAACQHRWLoMAAAAAPLeBgAAAAAABQAAAAAAAAkB1NFWDQAAAABQ3wYAAAAAAAUAAAAAAAAJAdU6nw4AAAAAR98GAAAAAAAFAAAAAAAACQH67OoMAAAAAFPfBgAAAAAABQAAAAAAAAkB1jqfDgAAAABK3wYAAAAAAAUAAAAAAAAJASmKyAwAAAAATd8GAAAAAAAFAAAAAAAACQHUOp8OAAAAAD/fBgAAAAAABQAAAAAAAAEAAAAAAAAAAAAAAP////8AAAAAAAAAAAAAAAAAAAAAAAAAAA=="}]

Impressive! How did you manage to find it out?

liaoyuzh avatar Mar 12 '22 18:03 liaoyuzh

Impressive! How did you manage to find it out?

Actually a friend of mine gave me a hint. We were talking about some stuff and I mentioned that I have some binary data from the API, which is from a game and it's not hash. He got idea that it could be compression and with that zlib came to mind as it's used a lot in the gaming industry.

Unfornatelly inside there is additional data which is not really human readable - that looks like some "flags" and you would have to know exactly what means what.

petrvecera avatar Mar 12 '22 21:03 petrvecera