Apache Ultimate Bad Bot Blocker for CWP and CWP PRO !

Hi,

First of all i wish to congratulate all the programmers of both CWP (Centos Web Panel) and CWP (Centos Web Panel) PRO for the excellent work they do and for the excellent product they have created, truly excellent from all points of view.

Having said that, i ask you to consider the implementation in CWP (Centos Web Panel) of Apache Ultimate Bad Bot Blocker for security purpouse.

This is the official link of Apache Ultimate Bad Bot Blocker: https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker

This security tools block this:

Apache Ultimate Bad Bot Blocker Version: V3.2019.10.1180

- Bad Referrer Count: 6774
- Bad Bot Count: 556
- Fake Googlebots: 214

... and plus more ...

The Ultimate Apache (2.2 > 2.4+) Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic from ever reaching your web sites.

PLEASE SEE - Definition of Bad Bots:

Bad bots are:
Bad Referrers
Bad User-Agent Strings
Spam Referrers
Spam Bots and Bad Bots
Nuisance or Unwanted Bots
Sites Linked to Lucrative Malware, Adware and Ransomware Clickjacking Campaigns
Vulnerability scanners
Gambling and Porn Web Sites
E-mail harvesters
Content scrapers
Link Ranking Bots
Aggressive bots that scrape content
Image Hotlinking Sites and Image Thieves
Bots or Servers linked to viruses or malware
Government surveillance bots
Botnet Attack Networks (Mirai)
Known Wordpress Theme Detectors (Updated Regularly)
SEO companies that your competitors use to try improve their SEO
Link Research and Backlink Testing Tools
Stopping Google Analytics Ghost Spam
Browser Adware and Malware (Yontoo etc)

I know you are always very busy developing CWP but i hope you can consider this suggestion of mine which i believe would be appreciated by all users of CWP (Centos Web Panel) and CWP (Centos Web Panel) PRO !

Thanks.

Hi,

please respond to my question.

Thanks in advance for the support.

  Hi Admin and Support Team of CWP (Centos Web Panel) and CWP (Centos Web Panel) PRO, please respond to my question, thanks.

Also this is the latest and stable version of Apache Ultimate Bad Bot Blocker Version: V3.2019.12.1186

This is the official link of Apache Ultimate Bad Bot Blocker: https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker

Thanks in advance for the support.

Hi, probably later in the next year.

Hi, probably later in the next year.

  Hi "Igor S.,

This is excellent news !

Thanks in advance for the support.

I've got this working on CWP

WARNING: this works ONLY on Apache-2.4, not if you have Nginx (as a proxy)

mkdir /usr/local/apache/custom.d/badbot
wget https://raw.githubusercontent.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/master/Apache_2.4/custom.d/globalblacklist.conf -O /usr/local/apache/custom.d/globalblacklist.conf
wget https://raw.githubusercontent.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/master/Apache_2.4/custom.d/blacklist-ips.conf -O /usr/local/apache/custom.d/blacklist-ips.conf
wget https://raw.githubusercontent.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/master/Apache_2.4/custom.d/bad-referrer-words.conf -O /usr/local/apache/custom.d/bad-referrer-words.conf
wget https://raw.githubusercontent.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/master/Apache_2.4/custom.d/blacklist-user-agents.conf -O /usr/local/apache/custom.d/blacklist-user-agents.conf
hostname|sed 's/\-/\\-/g'|sed 's/\./\\./g'|sed 's/$/ good_ref/'|sed 's/^/SetEnvIfNoCase Referer ~*/' > /usr/local/apache/custom.d/whitelist-domains.conf
for i in /usr/local/apache/conf.d/vhosts/*conf; do echo $i|sed '/\.ssl\.conf/d'|sed 's/\/usr\/local\/apache\/conf.d\/vhosts\//SetEnvIfNoCase Referer ~*/'|sed 's/\.conf$/ good_ref/'|sed 's/\-/\\-/g'|sed 's/\./\\./g' >> /usr/local/apache/custom.d/whitelist-domains.conf; done

/scripts/list_users|awk '{print $3}'|uniq|grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}"|sed 's/^/Require ip /' > /usr/local/apache/custom.d/whitelist-ips.conf
When this is done you have to add the following lines to the bottom of /usr/local/apache/conf/httpd.conf.
You can edit the file directly or in your Panel (WebServers Conf Editor)
Important: this have to be at the bottom, AFTER all the virtualhosts are loaded.


<Location "/">
    AuthMerging And
    Include custom.d/globalblacklist.conf
</Location>

Then restart apache, and check the httpd-logs
I've made a cron-file you can download here: https://www.webservice.be/update-apacheblocker.sh
CAVEAT: after adding a domain you must run the cron-file.
Don't know if theres a way you can do this auto in CWP, so run the script at least once a day.
Or create a cron that checks if something in usr/local/apache/conf.d/vhosts is changed.

Can you explain why this is any better than CSF blocking via modsecurity rules?
(Admittedly, a few custom rules need defined to cater for more bad/rogue bots.) 
To have both will surely Load a server unnecessarily. 

Can you explain why this is any better than CSF blocking via modsecurity rules?

I'm not telling it's better, but off all the solutions it's surely one off the most simple to use.
Add the frequent updates and its light CPU/MEM-foot makes it a good tool for me.

On other (non-CWP) servers we use our own IP-blocks, Fail2ban and for all ngix proxy-servers we use Rate Limiting.
Keeping track of the bots that come and go is impossible to do.
On the server where I installed this blocker the Comodo WAF modsec-rules where active. However the load of the server simply dropped by half after installing this blocker.

Here's a real nice blog about this topic:
https://blog.sqreen.com/detect-block-bad-bots/

Thanks for sharing. Certainly food for thought.

i got the following error

mkdir: cannot create directory ‘/usr/local/apache/custom.d/badbot’: No such file or directory