hackenproof data : missing some inscope values

[swdbo]

Hello !

It seems hackenproof data are not working properly anymore.

Example : On https://github.com/arkadiyt/bounty-targets-data/blob/master/data/hackenproof_data.json "/codex/codex-exchange" is empty but https://hackenproof.com/codex/codex-exchange shows in scope domains.

[arkadiyt]

Hey @swdbo, aren't those domains listed here? https://github.com/arkadiyt/bounty-targets-data/blob/master/data/hackenproof_data.json#L672-L687

edit: ah, it seems it was missing on this commit at the time that you opened the issue: https://github.com/arkadiyt/bounty-targets-data/blob/0b56bbe2f2a2117d58cf1e751afeb10900692ef8/data/hackenproof_data.json. I'll try and debug this a bit

[swdbo]

Hello @arkadiyt,

Yes, sorry I should have linked the exact commit. I don't see any difference in the html structure. Could it just be the server that rates limiting you ?

[arkadiyt]

Could it just be the server that rates limiting you ?

I don't know - I've been trying on and off to reproduce this issue and can't

[arkadiyt]

I've added some retries to hackenproof and I believe this should be resolved now. However I was never able to consistently reproduce it in the first place so it's hard to say with certainty - I'm going to close this issue out but if you see the same problem again please let me know or reopen it and I can try to dig in further