how to debug mariadb crash?

Hi

on my server mariadb is often being crashed and not restarted.

how to find a problem why it's crashed and is there any watchdog which will restart the service if any problem?

status shows the following

[root@cwp2 scripts]# service mariadb status
Redirecting to /bin/systemctl status mariadb.service
Б≈▐ mariadb.service - MariaDB 10.1.38 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           Б■■Б■─migrated-from-my.cnf-settings.conf
   Active: active (running) since Tue 2019-04-09 11:49:09 EEST; 11min ago
     Docs: man:mysqld(
           https://mariadb.com/kb/en/library/systemd/
  Process: 7664 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
  Process: 7616 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ]   && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, status=0/SUCCESS)
  Process: 7611 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
 Main PID: 7630 (mysqld)
   Status: "Taking your SQL requests now..."
   CGroup: /system.slice/mariadb.service
           Б■■Б■─7630 /usr/sbin/mysqld

Apr 09 11:59:11 cwp2 mysqld[7630]: 2019-04-09 11:59:11 140547070831360 [ERROR] Got error 127 when reading table './geekshop_shop/oc_product'
Apr 09 11:59:17 cwp2 mysqld[7630]: 2019-04-09 11:59:17 140547070831360 [ERROR] Got error 127 when reading table './geekshop_shop/oc_product'
Apr 09 11:59:33 cwp2 mysqld[7630]: 2019-04-09 11:59:33 140547070831360 [ERROR] Got error 127 when reading table './geekshop_shop/oc_product'
Apr 09 11:59:36 cwp2 mysqld[7630]: 2019-04-09 11:59:36 140547070831360 [ERROR] Got error 127 when reading table './ida_sport/cache'
Apr 09 11:59:36 cwp2 mysqld[7630]: 2019-04-09 11:59:36 140547070831360 [ERROR] Got error 127 when reading table './ida_sport/cache'
Apr 09 11:59:43 cwp2 mysqld[7630]: 2019-04-09 11:59:43 140547070831360 [ERROR] Got error 127 when reading table './geekshop_shop/oc_product'
Apr 09 11:59:49 cwp2 mysqld[7630]: 2019-04-09 11:59:49 140547070831360 [ERROR] Got error 127 when reading table './geekshop_shop/oc_product'
Apr 09 11:59:57 cwp2 mysqld[7630]: 2019-04-09 11:59:57 140547070831360 [ERROR] Got error 127 when reading table './geekshop_shop/oc_product'
Apr 09 12:00:04 cwp2 mysqld[7630]: 2019-04-09 12:00:04 140547070831360 [ERROR] Got error 127 when reading table './geekshop_shop/oc_product'
Apr 09 12:00:11 cwp2 mysqld[7630]: 2019-04-09 12:00:11 140547088923392 [ERROR] Got error 127 when reading table './geekshop_shop/oc_product'

cwp has two monitoring tools, recommend is cwppro monit, but this is not the solution!

A database must be fully stable and it can't crash so you must find and resolve the issue, in some cases that could mean export all with mysqldump and then reinstall database and import all data back again.

I think too that is is must be stable of course, my question was how to understand why it is being crashed. Right now process was dead again but can locate no info why in logs (

Hi, are you sure you checking correct logs? Too often it crashes due not enough RAM.
Check the MariaDB configuration file for the log file location.

Which logs to check is a my main question )

journalctl -u mariadb shows

Apr 11 06:05:19 cwp2 systemd[1]: mariadb.service: main process exited, code=killed, status=9/KILL
Apr 11 06:05:19 cwp2 systemd[1]: Unit mariadb.service entered failed state.
Apr 11 06:05:19 cwp2 systemd[1]: mariadb.service failed.
Apr 11 06:05:24 cwp2 systemd[1]: mariadb.service holdoff time over, scheduling restart.
Apr 11 06:05:50 cwp2 systemd[1]: Stopped MariaDB 10.1.38 database server.
Apr 11 06:07:49 cwp2 systemd[1]: Starting MariaDB 10.1.38 database server...
Apr 11 06:07:49 cwp2 systemd[1]: mariadb.service start-pre operation timed out. Terminating.
Apr 11 06:09:21 cwp2 systemd[1]: mariadb.service stop-final-sigterm timed out. Skipping SIGKILL. Entering failed mode.
Apr 11 06:09:22 cwp2 systemd[1]: Failed to start MariaDB 10.1.38 database server.
Apr 11 06:09:22 cwp2 systemd[1]: Unit mariadb.service entered failed state.
Apr 11 06:09:22 cwp2 systemd[1]: mariadb.service failed.

I've added 4gb extra memory...

Hi, the path to log must be set in /etc/my.cnf.d/server.cnf
Also, check if all files in /var/lib/mysql owned by mysql. The command below will change the owner to correct:

Code: [Select]

chown -R mysql:mysql /var/lib/mysql