Control Web Panel

WebPanel => CentOS 7 Problems => Topic started by: leebrimlow on October 26, 2024, 03:25:17 PM

Title: CPU high usage after mariadb upgrade 10.11
Post by: leebrimlow on October 26, 2024, 03:25:17 PM

After upgrading from 10.4 to 10.11  , i got high cpu usage and all my websites not working.
the panel became too slow.

any help please ?

CPU Model: AMD Ryzen 5 3600 6-Core Processor
CPU Details: 12 Core (2200 MHz)
Distro Name: CentOS Linux release 7.9.2009 (Core)
Kernel Version: 3.10.0-1160.119.1.el7.x86_64
Platform: x86_64 Dedicated


(https://up6.cc/2024/10/172995629800951.png)

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: cyberspace on October 26, 2024, 09:31:31 PM

I just recommend you to start from the investigation of MariaDB log file, other system logs, if there is nothing then check SQL queries "show processlist;", check RAM usage, etc.

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: Starburst on October 26, 2024, 10:17:44 PM

What is your Top 4 processes showing?
Can you post a screenshot of that.

This resembles the CPU usage problem from ClamAV.

Also curious why you are using a past EOL OS on a AMD Ryzen server.

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: leebrimlow on October 26, 2024, 10:38:07 PM

(https://up6.cc/2024/10/172998225154411.png)


(https://up6.cc/2024/10/172998225156862.png)

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: cyberspace on October 27, 2024, 01:07:13 AM

It looks like high cpu usage is caused by high i/o wait. Does your server use hard disks not  ssd/nvme disks ?

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: leebrimlow on October 27, 2024, 06:49:22 AM

No, it's NVME, that only happened after upgrading the MariaDB

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: cyberspace on October 27, 2024, 08:32:21 AM

Check logs of the SQL service, if there is nothing then try to optimize MySQL (enable caching if disabled).

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: leebrimlow on October 27, 2024, 08:34:43 AM

wha is the location of the mysql Log file please?

i searched in /var/log/ , but there is no mysql or mariadb log at all

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: cyberspace on October 27, 2024, 09:56:46 AM

If logging is disabled then do following:

1. Open the file:

Code: [Select]

/etc/my.cnf.d/server.cnf2. Add the lines:

Code: [Select]

general_log = On
general_log_file = /var/log/mysqld.log
log_error = /var/log/mysqld_error.log
under [server]

3. Restart MySQL/MariaDB.

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: leebrimlow on October 27, 2024, 12:58:15 PM

i restarted the mariadb upgrade again and stuck at this step for more than an hour !!!


(https://up6.cc/2024/10/173003388552411.png)

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: cyberspace on October 27, 2024, 01:52:28 PM

Have you checked logs ?

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: leebrimlow on October 27, 2024, 02:19:41 PM

 i got this:

2024-10-27 13:13:01 0 [Note] Starting MariaDB 10.11.9-MariaDB-log source revision 0e8fb977b00983d98c4c35e39bc1f36463095938 server_uid B9+4bhdCXZju+t0OtX1U29Rbx8s= as process 3964
2024-10-27 13:13:01 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
2024-10-27 13:13:01 0 [Note] InnoDB: Number of transaction pools: 1
2024-10-27 13:13:01 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2024-10-27 13:13:01 0 [Note] InnoDB: Using Linux native AIO
2024-10-27 13:13:01 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
2024-10-27 13:13:01 0 [Note] InnoDB: Completed initialization of buffer pool
2024-10-27 13:13:01 0 [Note] InnoDB: File system buffers for log disabled (block size=4096 bytes)
2024-10-27 13:13:02 0 [Note] InnoDB: End of log at LSN=834146431144
2024-10-27 13:13:02 0 [Note] InnoDB: 128 rollback segments are active.
2024-10-27 13:13:02 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
2024-10-27 13:13:02 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
2024-10-27 13:13:02 0 [Note] InnoDB: log sequence number 834146431144; transaction id 490699252
2024-10-27 13:13:02 0 [Note] Plugin 'FEEDBACK' is disabled.
2024-10-27 13:13:02 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2024-10-27 13:13:02 0 [Warning] 'thread-concurrency' was removed. It does nothing now and exists only for compatibility with old my.cnf files.
2024-10-27 13:13:02 0 [Note] Server socket created on IP: '0.0.0.0'.
2024-10-27 13:13:02 0 [Note] Server socket created on IP: '::'.
2024-10-27 13:13:04 0 [Note] InnoDB: Buffer pool(s) load completed at 241027 13:13:04
2024-10-27 13:13:04 0 [Note] /usr/sbin/mariadbd: ready for connections.
Version: '10.11.9-MariaDB-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MariaDB Server
2024-10-27 13:13:04 14 [Warning] Aborted connection 14 to db: 'unconnected' user: 'unauthenticated' host: 'localhost' (This connection closed normally without authentication)
2024-10-27 13:23:16 1608 [Warning] Aborted connection 1608 to db: 'unconnected' user: 'netdata' host: 'localhost' (Got an error writing communication packets)
2024-10-27 13:59:18 4465 [Warning] InnoDB: Could not free any blocks in the buffer pool! 8064 blocks are in use and 0 free. Consider increasing innodb_buffer_pool_size.
2024-10-27 14:43:10 9867 [Warning] InnoDB: Could not free any blocks in the buffer pool! 8064 blocks are in use and 0 free. Consider increasing innodb_buffer_pool_size.

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: cyberspace on October 27, 2024, 02:38:32 PM

It could cause high disk i/o usage:

Code: [Select]

2024-10-27 13:59:18 4465 [Warning] InnoDB: Could not free any blocks in the buffer pool! 8064 blocks are in use and 0 free. Consider increasing innodb_buffer_pool_size.
Consider to optimize MySQL/MariaDB performance:
https://mariadb.com/kb/en/optimization-and-tuning/

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: leebrimlow on October 27, 2024, 10:06:30 PM

installed CWP with Almalinux 8 , can not install PHP , upgrade mariadb or do anything!! everything is blocked !! Why !!


(https://up6.cc/2024/10/173006670455271.png)

(https://up6.cc/2024/10/173006670457972.png)


(https://up6.cc/2024/10/173006670459763.png)

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: cyberspace on October 27, 2024, 10:13:50 PM

There are a lot of errors "can't resolve". Make sure the DNS resolver on your VPS/Server is operating fine.

Title: Re: CPU high usage after mariadb upgrade 10.11
Post by: Starburst on October 28, 2024, 06:05:04 AM

You PHP-FPM is what has the load from your screenshots.

I think your server is messed up, PHP-FPM should NOT show up on a dnf update, this is built by CWP.

So I'm guessing you installed PHP and MariaDB on their own and not from CWP.

So you have LAMP installed before CWP, which is advised not to do as it can corrupt the server.

Your best option is redo the server WITHOUT LAMP, just the basic image from your provider.

Then let CWP install everything, and then compile PHP from within CWP, not the PHP repos.

After CWP has installed everything, then you can upgrade MariaDB following one of the guides out there, like:
https://kb.starburstservices.com/control-web-panel-cwp/control-web-panel-cwp-admin-tutorials/upgrade-mariadb-10-x-to-10-11-in-cwp-on-almalinux-8/ (https://kb.starburstservices.com/control-web-panel-cwp/control-web-panel-cwp-admin-tutorials/upgrade-mariadb-10-x-to-10-11-in-cwp-on-almalinux-8/)

But if this is a new server, best bet is to use the AlmaLinux 9.x (usually 9.4) image for the provider.

If you want, you can do the reimage, and I can login to the bare system and get everything installed for you.
Just PM your server details.