mysqld: Can't lock aria control file

Hi,

I upgraded MariaDB from 10.1 to 10.3 following the instructions here - https://www.mysterydata.com/update-upgrade-to-mariadb-10-3-on-vestacp-cwp-centos-7/

After upgrading the mysql server is working perfectly without issues but when I'm logged into CWP GUI as root I do see MySQL server is marked as stopped. Even if I try starting/restarting mariadb it doesn't start.

Here's the output from the terminal.

Code: [Select]

[root@server1 ~]# systemctl status mariadb
● mariadb.service - MariaDB 10.3.23 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           └─migrated-from-my.cnf-settings.conf
   Active: inactive (dead)
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
[root@server1 ~]#

After restarting mariadb I do see the following -

Code: [Select]

May 29 12:29:41 server1.domain.com systemd[1]: Starting MariaDB 10.3.23 database server...
May 29 12:29:42 server1.domain.com mysqld[1950]: 2020-05-29 12:29:42 0 [Note] /usr/sbin/mysqld (mysqld 10.3.23-MariaDB) starting as process 1950 ...
May 29 12:29:42 server1.domain.com mysqld[1950]: 2020-05-29 12:29:42 0 [Warning] Could not increase number of max_open_files to more than 16364 (request: 32183)
May 29 12:29:42 server1.domain.com mysqld[1950]: 2020-05-29 12:29:42 0 [ERROR] mysqld: Can't lock aria control file '/var/lib/mysql/aria_log_control' for exclusive use, error: 11. Will retry for 30 seconds


May 29 12:07:06 server1.domain.com mysqld[1892]: 2020-05-29 12:07:06 0 [Note] InnoDB: 10.3.23 started; log sequence number 6504894; transaction id 4175
May 29 12:07:06 server1.domain.com mysqld[1892]: 2020-05-29 12:07:06 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
May 29 12:07:06 server1.domain.com mysqld[1892]: 2020-05-29 12:07:06 0 [Note] InnoDB: Buffer pool(s) load completed at 200529 12:07:06
May 29 12:07:06 server1.domain.com mysqld[1892]: 2020-05-29 12:07:06 0 [Note] Plugin 'FEEDBACK' is disabled.
May 29 12:07:06 server1.domain.com mysqld[1892]: 2020-05-29 12:07:06 0 [ERROR] Aria engine is not enabled or did not start. The Aria engine must be enabled to continue as mysqld was configured with...ria-tmp-tables
May 29 12:07:06 server1.domain.com mysqld[1892]: 2020-05-29 12:07:06 0 [ERROR] Aborting
May 29 12:07:08 server1.domain.com systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
May 29 12:07:08 server1.domain.com systemd[1]: Failed to start MariaDB 10.3.23 database server.
May 29 12:07:08 server1.domain.com systemd[1]: Unit mariadb.service entered failed state.
May 29 12:07:08 server1.domain.com systemd[1]: mariadb.service failed.
Hint: Some lines were ellipsized, use -l to show in full.


May 29 12:31:00 server1.domain.com mysqld[1950]: 2020-05-29 12:31:00 0 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
May 29 12:31:01 server1.domain.com mysqld[1950]: 2020-05-29 12:31:01 0 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
May 29 12:31:01 server1.domain.com mysqld[1950]: 2020-05-29 12:31:01 0 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
May 29 12:31:02 server1.domain.com mysqld[1950]: 2020-05-29 12:31:02 0 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
May 29 12:31:02 server1.domain.com mysqld[1950]: 2020-05-29 12:31:02 0 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
May 29 12:31:03 server1.domain.com mysqld[1950]: 2020-05-29 12:31:03 0 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
May 29 12:31:03 server1.domain.com mysqld[1950]: 2020-05-29 12:31:03 0 [Note] InnoDB: Check that you do not already have another mysqld process using the same InnoDB data or log files.
May 29 12:31:04 server1.domain.com mysqld[1950]: 2020-05-29 12:31:04 0 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11



I'm also seeing multiple mysqld process is running.

Code: [Select]

[root@server1 ~]# ps aux | grep mysqld
root      1244  0.0  0.0 115540  1708 ?        S    12:04   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/server1.domain.com.pid
mysql     1407  0.1  4.6 1309456 87888 ?       Sl   12:04   0:00 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=/var/lib/mysql/server1.domain.com.err --pid-file=/var/lib/mysql/server1.domain.com.pid
mysql     1892  0.1  2.7 652428 51596 ?        Ssl  12:05   0:00 /usr/sbin/mysqld
root      1992  0.0  0.0 112780   732 pts/0    S+   12:06   0:00 grep --color=auto mysqld


So I stopped mysql and then could successfully start mariadb but the same issue continues everytime I reboot. I tried deleting aria_log_control manually which didn't fix this. Can anyone please tell me why this is happening and is there something I can do to fix this? Any suggestion would be highly appreciated. Thanks for reading.

One thing I forgot to mention, I upgraded MariaDB on multiple servers and experiencing the same with all of them.

https://lmgtfy.com/?q=Could+not+increase+number+of+max_open_files+to+more+than+16364
Voila!
https://stackoverflow.com/questions/34686072/cannot-increase-open-files-limit-in-mariadb-10-on-centos7

Hi,

I have already tried increasing the number of max_open_files and it's not the issue I believe. As I'm seeing it's just a warning. Shouldn't make the whole database server down.

systemctl stop mysqld
ps -ef | grep mysqld
if anything results.
killall mysqld

systemctl start mysql

Hi,

Thanks for the suggestion.

Yes, the above steps brings back the mysql server online I mean killing the mysql processes and restarting mysql and mariadb. The issue was continuing after every reboot. However after upgrade I forgot to start mariadb on boot which was causing issues. "systemctl enable mariadb" fixed the issue and everything running smooth now. Thanks again.

systemctl stop mysqld
ps -ef | grep mysqld
if anything results.
killall mysqld

systemctl start mysql

Thank you. This helped.

Any idea why this happens? I had to run this second time in less than a month

I guess you tried to run the MariaDB server too many times. Try to disable MariaDB monitoring if in use.