Author Topic: BING killing my server  (Read 147 times)

0 Members and 1 Guest are viewing this topic.

Offline
*
BING killing my server
« on: June 04, 2019, 03:51:39 AM »
Hi,
Is there a way to prevent search engine crawlers such as BING to kill my server?
It has happened twice this week.
It took me a while to figure out what was going on, but when BING crawls one on my sites, the load average of the server goes to the roof. It is the same as having a denial of service attack as it crawls hundreds of pages at the same time.
Then the server goes back to normal but the bind service doesn't recover and remains down.
I have to connect to the server and start bind (named) manually.
I will install monit in the server to solve this problem but the crawler error will persist.
Any ideas in how to deal with it?
Thanks in advance.

Offline
*
Re: BING killing my server
« Reply #1 on: June 04, 2019, 11:47:52 AM »
they are killing websevers also, many issues recently because of that, maybe block ip ranges with csf

check connections
sh /scripts/net_show_connections

then block ip ranges, this ip is only example:
csf -d 54.104.23.0/24
VPS & Dedicated server provider with included FREE Managed support for CWP.
http://www.studio4host.com/

*** Don't allow that your server or website is down, choose hosting provider with included expert managed support for your CWP.