micropython icon indicating copy to clipboard operation
micropython copied to clipboard

docs: The old forum is down

Open peterhinch opened this issue 10 months ago • 20 comments

Documentation URL

No response

Description

https://forum.micropython.org/ produces a 404

Code of Conduct

Yes, I agree

peterhinch avatar Mar 15 '25 12:03 peterhinch

Does anyone know what the tech backend of the forum was/is? That would be a first step to assess the options to migrate the content somewhere else.

Josverl avatar Mar 15 '25 15:03 Josverl

It works for me, if you do a reload...

It uses phpBB.

dpgeorge avatar Mar 15 '25 21:03 dpgeorge

Ive set up a simple monitor to check availability from different (Azure) locations, and there is something flakey to say the least.

Image

Test Details

Azure Application Insights. Standard Web site availability tests - All regions - 15minutes

Image

Josverl avatar Mar 16 '25 13:03 Josverl

Yes, it's 404 here (on two different browsers). It sometimes reappears briefly. See also https://github.com/orgs/micropython/discussions/16937

peterhinch avatar Mar 16 '25 19:03 peterhinch

responses are improving:

  • availability is up to 100% since ~03:00 CET
  • Latency is down from >15 secs to ~4.4 secs

Image

now has something changed ? or are the polls waking up the site / cache layers ?

Josverl avatar Mar 17 '25 07:03 Josverl

Maybe an Act of Damien.

robert-hh avatar Mar 17 '25 08:03 robert-hh

I had a look at the server logs, and it seems like a DDoS attack maybe? Or just a lot of invalid/random traffic.

For micropython.org the stats for the requests and status codes are:

Time start:     2025-04-06 00:10:17
Time end:       2025-04-07 04:35:34
Total seconds:  102317.0
Total requests: 612715
200 : 573459 requests = 93.59%
206 : 828 requests = 0.13%
301 : 8315 requests = 1.35%
302 : 1680 requests = 0.27%
304 : 9790 requests = 1.59%
403 : 67 requests = 0.01%
404 : 13066 requests = 2.13%
416 : 101 requests = 0.01%
499 : 5257 requests = 0.85%
500 : 152 requests = 0.02%

That seems normal, about 6 requests per second, 94% of them return 200.

But for forum.micropython.org over the same time period it doesn't look good:

Time start:     2025-04-06 00:10:17
Time end:       2025-04-07 04:37:46
Total seconds:  102449.0
Total requests: 2623174
200 : 1011032 requests = 38.54%
206 : 35 requests = 0.001%
301 : 13656 requests = 0.52%
302 : 564 requests = 0.021%
304 : 66 requests = 0.002%
400 : 2 requests = 7.62e-05%
403 : 1840 requests = 0.07%
404 : 1100584 requests = 41.95%
499 : 495395 requests = 18.88%

That's a huge number of 404 and 499 responses.

Those 404/499 requests seem to be randomly generated phpBB URL that don't resolve to anything.

For now I've disabled the forum, although these random requests keep coming in (so it's definitely not real people accessing the site for those 404/499s).

dpgeorge avatar Apr 07 '25 05:04 dpgeorge

I am responsible for 10 HTTP gets every 15 minutes for the Azure App Insights probes, 15m is the max interval I can set for forum and search , and I check from different continents as there were significant differences in availability. I suspect the forum may be under an investigative attack looking for vulnerabilities in handling these strange urls.

and you are correct that during the recent period there have been issues again. happy to share mor detailed timing if that helps you.

Image

Josverl avatar Apr 07 '25 05:04 Josverl

Possibly related : https://app.opencve.io/cve/?vendor=phpbb

CVE-2001-1471 | 1 Phpbb | 1 Phpbb | 2025-04-03 | 8.8 High

prefs.php in phpBB 1.4.0 and earlier allows remote authenticated users to execute arbitrary PHP code via an invalid language value, which prevents the variables (1) $l_statsblock in prefs.php or (2) $l_privnotify in auth.php from being properly initialized, which can be modified by the user and later used in an eval statement.

Note the update date of the CVE, and the issues occurring on and since that day. I suggest you treat this as a security incident.

Josverl avatar Apr 07 '25 05:04 Josverl

Thanks @Josverl for the extra info. That plot does indicate that the attack started abruptly, with a good correlation of the date of that CVE being updated.

The forum uses a much more recent version of phpBB so shouldn't be impacted by that CVE. Still, the traffic is annoying and slowing down some other micropython sites.

dpgeorge avatar Apr 07 '25 05:04 dpgeorge

I can offer to run Azure Front Door + WAF Standard on my expense to help filter out that unwanted traffic if that would help.

Josverl avatar Apr 07 '25 07:04 Josverl

I can offer to run Azure Front Door + WAF Standard on my expense to help filter out that unwanted traffic if that would help.

No, don't do that, it won't help.

dpgeorge avatar Apr 07 '25 12:04 dpgeorge

@Josverl are you able to share recent plots for forum latency?

Also, are you constantly accessing http://micropython.org/ks/test.html ? Someone is...

dpgeorge avatar May 31 '25 02:05 dpgeorge

Currently traveling, I'll check when I get home. IIRC The checks access a page every 15 minutes from 5 different locations globally. So about 20 requests/hour.

Update (waiting at the gate) : I had turned off the availability checks to avoid extraneous traffic - so can't share recent history. I have re-enabled it for the following URLs :

  • https://forum.micropython.org/
  • https://forum.micropython.org/search.php

Josverl avatar May 31 '25 07:05 Josverl

the esp8266 tutorial uses that URL, so that is possibly running on a lot of boards. docs/esp8266/tutorial/network_tcp.rst : response = http_get("http://micropython.org/ks/test.html")

Josverl avatar May 31 '25 13:05 Josverl

The access to http://micropython.org/ks/test.html is coming from a single IP address, around 5 times per second.

dpgeorge avatar May 31 '25 13:05 dpgeorge

Can you block or delay that IP?

Josverl avatar May 31 '25 14:05 Josverl

@dpgeorge the last 24 hours i've seen no issues

Image

Josverl avatar Jun 01 '25 12:06 Josverl

Thanks @Josverl for the data. At least availability is not being impacted.

I can block the IP that's requisting ks/test.html, but would be interesting to know who it is.

dpgeorge avatar Jun 02 '25 02:06 dpgeorge

always hard to tell with just an IP. Rate limiting would be best - but probably comes at a significant cost. Ill keep the monitors running for now and notify you in case of alerts.

Josverl avatar Jun 02 '25 11:06 Josverl

I had to disable the old forum again, it was getting a huge amount of spam traffic, again.

dpgeorge avatar Jun 26 '25 13:06 dpgeorge

Did you receive the e-mail alert I set up, or did you find out yourself ?

Image

Josverl avatar Jun 27 '25 10:06 Josverl

Did you receive the e-mail alert I set up, or did you find out yourself ?

Both. I got lots of emails (and still getting them) and also noticed the server was very slow.

dpgeorge avatar Jun 27 '25 10:06 dpgeorge

I updated the config to send fewer alerts, and paused the alerting for now

Josverl avatar Jun 27 '25 11:06 Josverl