Control Web Panel
WebPanel => CentOS 7 Problems => Topic started by: Starburst on February 22, 2020, 09:12:58 AM
-
I use Nagios XI to monitor 3 servers.
Currently 2 are Pro and 1 isn't.
Nagios monitors all processes and report fine on all 3.
But when the daily cron is run on the 2 Pro CWP servers it stops reporting correctly to Nagios unless that server is rebooted.
The Non-Pro CWP server is fine.
Any suggestions?
-
Try to check if Nagios server accessible from the server and check the firewall's whitelist.
-
Try to check if Nagios server accessible from the server and check the firewall's whitelist.
Nagios XI server is white listed, everything works fine until in the morning when the cron.daily runs.
Then on the 2 server running CWPPro stop reporting the check_all_procs to Nagios correctly. But 1 server not running Pro continues OK.
The 2 servers running Pro, when rebooted, start reporting correctly again.
Just trying to narrow down what is running at that time causing the problem.
Thanks,
David
-
Igor? Anyone?
-
this are 3rd party scripts, we didn't support it. Check the nagios log why it is not working
-
Sandeep,
With all due respect this isn't a Nagios problem according to the logs.
Something is running in the cron.daily that is causing the problem according to the timestamps in the logs.
And funny enough the 2 servers running CWP Pro are having the problem, while another CWP Non-Pro server is not.
Reboot the server(s) running CWP Pro and it runs fine for 23 hours 55 mins until cron.dail runs again, and then server has to be rebooted to start functioning again.
I'm just trying to narrow down exactly what CWP runs that could be causing this.
-
Out of curiosity, does the service shut down after an update.
Do you really need to restart the entire server, or can you just restart the service...
-
Out of curiosity, does the service shut down after an update.
Do you really need to restart the entire server, or can you just restart the service...
After cron.daily runs it stops sending the correct # of processes running, 6 instead of about 140, xinetd is still running and all other monitors are working correctly.
I've tried restarting the xinetd service, but doesn't do anything.
Only thing that bring back CWPPro reporting correctly is a server reboot.
And again, it's only happening on the CWPPro servers.
The Non-Pro server is OK.
Hence why I've been trying to figure out what runs differently on CWPPro, from looking in /etc/cron.daily @ CWP and other in that directory, I don't see anything.
And it's very weird that the logs show it stops reporting correctly at the same time as that.
I'm tring a couple things different on 1 of the CWPPro servers tonight, but at this point I'm just shotgunning.
-
Would munin not suffice as an alternative? I have had it running fine for months on various servers/VPS.
Just a thought.
-
Would munin not suffice as an alternative? I have had it running fine for months on various servers/VPS.
Just a thought.
Never seen munin before, but have used MRTG & RRDTool.
I like Nagios because it will send out notifications if a service stops responding like WhatsUp Gold.
RRDTool is great for monitoring bandwidth usage on network cards, switches and routers.
I had used it for the 95%tile.
-
I started using MRTG just over 20 years ago. ;)
I use the basics of munin on all servers, plus DNS monitoring on my nameservers. It is possible to set up alerts though I don't utilise that aspect ( I should! ).
-
I started using MRTG just over 20 years ago. ;)
I use the basics of munin on all servers, plus DNS monitoring on my nameservers. It is possible to set up alerts though I don't utilise that aspect ( I should! ).
MRTG & RRDTool is great for bandwidth monitoring.
Nagios is great for service monitoring (http, ftp, smtp, pop/imap, etc) if a service stops responding it will send an email & text alert.
The each have their areas they are good in.
-
How about extending Monit?
https://www.howtoforge.com/tutorial/server-monitoring-with-munin-and-monit-on-centos/
(Though note the CWP Pro possible restriction.)
-
How about extending Monit?
https://www.howtoforge.com/tutorial/server-monitoring-with-munin-and-monit-on-centos/
(Though note the CWP Pro possible restriction.)
What's nice with Nagios, is you can monitor multiple servers.
Had a problem with the Dovecot service dying, and it still was showing active on the server, but an external check showed it non-responsive.
-
I presume that you've tried a cron with
/etc/rc.d/init.d/nagios reloador
/usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfgand set it to run after that cwp one has done it's thing.
Perhaps add it to the end of /etc/cron.daily/cwp ? Though I suspect that won't work - better to look at how long the cwp one runs (approximately) and execute a timed cron, using crontab.
-
I presume that you've tried a cron with
/etc/rc.d/init.d/nagios reloador
/usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfgand set it to run after that cwp one has done it's thing.
Nagios is running on another box.
I tried restarting the nagios service on that box when I noticed this.
Nothing changed.
Again everything is reporting correctly, except the 2 CWPPro servers after the cron.daily runs.
Only fix right now, is to restart the server CWPPro is running on after the cron.daily runs.
Then all is back normal.
-
Is Monit running on the CWP Pro ones? Something must be killing the nagios process/daemon..
I'd be looking at the cron logs on both server types, to try to identify the culprit.
-
Saw the chat back & forth.
It could be something in that area that is proprietary, and this is fine not to disclose.
I don't see how this thread if off topic with a problem I've narrowed down to CWWPro though.
Maybe no one else has monitored processes running, which isn't a deal breaker for me to disable, if it comes to that.
-
Is Monit running on the CWP Pro ones? Something must be killing the nagios process/daemon..
I'd be looking at the cron logs on both server types, to try to identify the culprit.
Nagios isn't running on the CWPPro server, Monit is (tried to disable it, no difference).
xinetd service is still running, wasn't killed.
Trying to restart xinetd service doesn't change anything.
-
Do you have to open particular ports in the firewall for nagios?
You might want to check "iptables -L" before and after the problem occurs (the cwp cron task). If there is a difference then use /etc/csf/csfpost.sh to add in the missing entries. It may explain why you need a reboot for nagios to begin functioning again.
-
Do you have to open particular ports in the firewall for nagios?
You might want to check "iptables -L" before and after the problem occurs (the cwp cron task). If there is a difference then use /etc/csf/csfpost.sh to add in the missing entries.
Nagios uses port 5666 and is open.
As Nagios is running fine 23 hr 55mins each day, and only stops after the cron.daily runs.
xinetd is still running, and restarting the service doesn't work.
And I don't feel like having to reboot servers each day after cron.daily runs.
Checked the logs files and don't see anything.
I'll just chaulk it up to that processes can't be monitored on CWPPro servers.
-
Do you have to open particular ports in the firewall for nagios?
You might want to check "iptables -L" before and after the problem occurs (the cwp cron task). If there is a difference then use /etc/csf/csfpost.sh to add in the missing entries. It may explain why you need a reboot for nagios to begin functioning again.
Thanks for your help ejsolutions.
But I'm giving up on this bug for right now.
-
Thanks for your help ejsolutions.
You're welcome.
Good luck.
-
we've enterprise grade nagios monitoring for our clients and they doesn't have any such issues, check the /var/log/messages and nagios log for the issues.