Author Topic: Something is intercepting bots and loading a different page for them.  (Read 34 times)

0 Members and 1 Guest are viewing this topic.

Offline
*
I’m facing a very strange issue.
Something on the server is intercepting all web requests from bots that have "bot," "crawler," or "spider" in their user-agent string (there may be others, but these are the ones I've noticed). I spotted this because social sharing links stopped loading correctly, and my already meager AdSense earnings plummeted.
The strangest part is that it isn't a standard block; instead, when a bot is detected, it loads a basic HTML page that says "Welcome Loading..."
I’ve checked everything: there are no filters in Nginx or PHP, and ModSecurity is disabled. I reviewed the Nginx and Apache configuration files and found nothing unusual. I even searched the entire server for that "Welcome" file—or any file containing that text—but couldn't find it.
What could be happening? I would appreciate any help. Thanks.
 

Offline
**
Did you search your sites root folder for any suspicious files or code, redirects or recent changes?

What do the logs tell you? You can view in the UI (File Management > CWP->Log Viewer or below)

Apache Main Access Logs: /usr/local/apache/logs/access_log
Apache/Nginx Domain Logs: /usr/local/apache/domlogs/DOMAIN.COM
Nginx Access Logs: /var/log/nginx/access.log

IMO you should also keepModSec enabled (other than disabling for troubleshooting) you are on PHP 7.3 which you might choose to upgrade if your sites allow.
Web Design, Development & Web Hosting
https://6sense.com.au

Offline
***
I’m facing a very strange issue.
(...)

You are a very brave man.
We all need to be constantly paranoid to keep any web server secure.
It is practically unfeasible these days to keep a web server exposed to the internet—using closed-source software like CWP—without ModSecurity, Coraza, or similar tools.
Avoid doing this unless you know exactly what you are doing or want to deal with big major headaches involving clients.
Therefore, enable ModSecurity and use a WAF like Comodo (specifically with the latest updates available for the 'sminozzi' user) or OWASP CRS4, and always monitor your logs to keep your rules up to date.

Regards,
Netino