Forum has been very slow 2 days now...

[smatovic]

I had from 1999 til 2010 the profession of an web-programmer and Linux system-administrator, and reached with the second tuning step of the TC server my limit, my knowledge is meanwhile outdated. Hence I suggest to the founders to seek for skilled volunteers or to engage a professional for tuning Apache modules and iptables against crawlers and bots.

I think you'll be impressed how helpful today's top chatbots are with knotty technical issues where the knowledge is widely known. It's definitely worth a try.

Well, my knowledge is outdated, I reached my limit, and I have no time to work in.

--
Srdja

[hgm]

It seems mod_evasive is already enabled in the current server configuration. But the parameters it uses (18 times the same file in a 2-sec interval, or 128 accesses in total) is not effective against the current DDoS attack. Since the query string is considered part of the filename, and the attackers use different sid for every access, only the total limit on the number of accesses can be effective, no matter how we configure mod_evasive. And an hour ago, when it was hardly possible at all to access TalkChess, there were only about 50 accesses per second by all attackers combined. The individual attackers only do some 5 requests per minute.

It would be better to simply reject all requests for index.php. I don't think there is a legitimate reason to request that. But of course the attackers could switch strategy when they discovers their requests no longer work, and could start requesting something else, which cannot be distinguished from a legitimate request.

The attacker's IP addresses all start with 43, though, with rather limited variation of the second byte of the address. So perhaps the quota should be applied to the entire 43 IP range, and ban that entire range when exceeded.

Anyway, it seems we need to create our own dedicated version of mod_evasive. So I will have to learn how to write and install mods for the Apache server. Not sure when I can get to doing that...

[towforce]

...It would be better to simply reject all requests for index.php. I don't think there is a legitimate reason to request that.

I look at that page at every visit: it's a quick way to see updates to the infrequently used forums. I hope it can be kept.

Maybe the page can be cached and just updated every 5 minutes?

index.php

[Joerg Oster]

...It would be better to simply reject all requests for index.php. I don't think there is a legitimate reason to request that.

I look at that page at every visit: it's a quick way to see updates to the infrequently used forums. I hope it can be kept.

Maybe the page can be cached and just updated every 5 minutes?

index.php

I had also bookmarked the address with index.php included. Changed that now ...

[hgm]

If I look at the forum home page, the request appears in the log as a request for /, not for /index.php. Since the server already has to do significant work to figure out that / is a directory, and that it has to serve /index.php in it, rather than /index.html or /index.cgi, I assume it would do this only after the decision to not reject the request is made. In the log of a test session where I accessed the home page several times, there never appeared a request for /index.php fully spelled out, and certainly not with an sid in the query string.

But rejecting these requests only is a quick and dirty solution, as the attacker could simply shift to requesting another file. And it would require modification of the Apache mod_evasive module anyway. And once I know how to do that, it will be very easy to recognize DDoS attacks and the participating attackers in a more fundamental way.

[hgm]

The source code of mod_evasive I had did not work for Apache2.4, ehich altered some variable names. But I found a source code that is suitable for this version. And I managed to install it. This only required the command

Code: Select all

sudo apsx -i -a -c mod_evasive.c

where the mod_evasive.c is in my user directory. I did this after disabling the mod_evasive that was already in use through

Code: Select all

sudo a2dismod mod_evasive

I then restarted the server through

Code: Select all

sudo systemctl restart apache2.service

Since I am able to post this message, these actions apparently have not wrecked the server.

Of course this does not improve anything yet, as the mod_evasive that I now installed is likely identical to the one that was already in use. There is only one thing that is a bit weird. Apache handles its modules by having for each one a .load file in /etc/apache2/mods-available/*.load, which then contains a command to load an executable from /usr/lib/apache2/modules/mod_*.so . The original evasion.load referred to mod_evasion20.so, but the one I now installed added a file evasion20.load, referring to mod_evasion.so .

[smatovic]

Turns out, after login you are redirected to /index.php?sid=xyz and "Bord Index" links to /index.php.

/index.php is a valid request, one shall not block it by default.

--
Srdja

[hgm]

OK, I see. First I just discriminated against "GET /index.php", because the r->uri in the struct r passed to the module did only contain the path part, not the query string. But then I could not process my notifications anymore, which does generate a request for index.php with a query string.

Then I found out the passed struct also has a field r->unparsed_uri, and I had it test for index.php?sid= . That solved the problem with the notifications, but as you mentioned interferes with logging in (or out, for that matter). So no luck there, and I removed all my patches.

The good news, however, is that I apparently now am able to make arbitrary modifications to mod_evasive, which do actually work as designed in Apache. So now it is just a matter of writing some code to recognize a DDoS attack, and the clients participating in it, to reject their requests. (BTW, even though forum response seems good today, the log shows we are still under attack. Just not so heavily; only some 20 requests per second.)

I also learned that the standard mod_evasive doen ignore the query string, so that in principle it can be configured to block the originators of all these index.php?sid=... requests. But the problem is that the individual attackers do not submit that many requests. It is just that there are so many of those. So lowering the threshold for blocking to stop them could very well interfere with normal forum use, when a user browses through several threads using viewtopic.php?t=...

[hgm]

I now made some changes to mod_evasive, so that it will be more effective in recognizing a DDoS attack. But the threshold for this now seems a bit low: when I try to load a discussion while flushing the browser cache, some of the files initially do not get loaded, and I lose the formatting, or the piece images in pgn4web. For use with normal caching the forum seems to work fine, though.

When you get blocked (resulting in 403 "forbidden" responses), you can just wait 10 sec, and then refresh the page. For me this usually solves the problem.

Let me know about any weirdness you now experience when using the forum.

[smatovic]

Yesterday evening the website looked broken and I had 403 errors, today it looks fine, no errors, and is fast....

--
Srdja