"We are experiencing a lot of load"

[DinkydauSet]

Dear creators of Dogbolt, I noticed this message about load on your website. I just wanted to let you know that I and two others are writing a paper about decompiler benchmarking (testing the quality of decompilation results), and we are using your service to test some decompilers we don't have access to, so I have a feeling that we could be responsible for that load. I hope you don't mind, as we are really dependent on your service right now. We plan to submit a few more batches of executables over the next few weeks and then we are done.

[ltfish]

Putting on my researcher’s hat: If your research adds significantly more load than usual for decompiler explorer, you should seriously consider running a self-hosted instance of decompiler explorer and performing as many experiments as you want there. Otherwise I see it as an abuse of free public resources that a company (Vector35) provides to the community, and I don’t think that reflects well on you or your research group.

While I appreciate your notice, “we need to overload your service more for an upcoming conference due” is not a great excuse.

[jbcjsc]

Dear creators,

Thank you for the reply. I am one of the other two and I've been doing the batches.

First of all: we mean you no harm. We have, and had, no intention of overloading. If we did, we apologise.

We are three bachelor students currently working on our thesis. We had no idea our work would be flooding your resources, but admittedly, we might have been warned, had we read the FAQ section on your site more carefully. To determine whether we really were the cause of the trouble, I'll tell you what we've done. I started a batch of 1.500 binaries on April 18, 7.17 CET. It finished April 19th, 13.33 CET. If this was the period of the commotion, we were the likely culprits.

We are looking forward to hearing from you.

Kind regards,

Jaap & Reijer & Kesava

[psifertex]

Thanks for reaching out. Were you actually able to get results for 1500 binaries in that period? We have rate-limiting that should have stepped into block that much automation. Are you sure many of the results aren't invalid? That said, even with the rate-limiting this sort of automation is absolutely prone to causing service degradation for others and I would encourage you to seek other options.

If you're interested in an academic research license for the two commercial tools in question I highly suggest you reach out directly to the respective companies: V35, HexRays

We (Vector 35) have in the past sponsored some research with free licenses, and even our student discount can bring the price of a license well within an educational organization's budget. Hex-Rays likewise has some educational discount options available as well.

[negasora]

We limit to 60/hr so 30 hours is more than enough to submit 1500 bins

[jbcjsc]

Dear Dogbolt-people,

Sorry for the late reply. We are currently very near the deadline of handing in our bachelor thesis, so tension is high and time is tight.

In reply to your question: we did manage to get results. As one of your team noticed, we were below the 60/hr limit.

Thank you for the advice on license-possibilities, but at the moment, time is running out and we are devoting pretty much all our time to all those just-before-deadline-activitties. We lack time and resources to go deeper into that at the moment. We would very much like to run a final of our binaries using your service. As we wrote before: we do not want to cause harm, so, if we limit to 40/hr (1 every 1.5 minute), would that be ok with you? Are there time slots (perhaps during the night) that we should preferably use?

Kind regards,

Jaap

[negasora]

40/hr seems reasonable for now - hopefully we'll figure out why ghidra's perf is so bad and it won't end up with a giant backlog so easily. Thanks for the heads up and good luck with your research!

[DinkydauSet]

Thanks. We have Ghidra running locally so we won't add to that queue at least. We have also found Ghidra to be pretty slow. I just timed retdec and ghidra with a random executable: retdec took 4 seconds, ghidra 62 seconds.

[negasora]

Sadly rn every decompiler runs for every binary so it'll still clog up ghidra but :shrug: