Change
Close db connections
Issue description
Using a store opens a connection to the MySQL database. When the PHP process finishes, PHP automatically closes the connection. But during unit tests or during long lived PHP processes with multiple stores in general the number of open db connections increases over time. At some point the maximum number of connections is reached and then there is simply no continuation. In case of the nightly tests this is noticeable since the log file has no entry from 2300 to 0700.
Developer comments
Generally the following rule should apply: where an instance of the PHP class %Store% is created, the db connection should be closed best in the same function or in the same class. In some cases this not necessary or even should be avoided, since database access is necessary in shutdown. These cases are %AdminCommand->execute% and %topincs.php%.
Switched to persistent connections during unit tests. After some digging it became clear that the timeouts and stalls were most likely created by old threads which hugged the same table.
The switch to persistent connections was reverted. It did not make the permature termination of the unit tests go away. There is some specific tests responsible for the failure and also only under special circumstances.
Reusing the same mysqli instance during unit tests, so like persistent, but with more control. Sometime when [13655, removing global state], it was introduced that there is many more db connections created instead of a single global connection. This is most likely the reason for the instability during the unit tests.
The long period of inactivity was caused by a metadata lock in MySQL.
I am no longer convinced that the problem of non-terminating unit tests is caused by Topincs. This might be a case of wrongful attribution. I think now that the behavior my MariaDB changed in a subtle way causing all of the sudden metadata locks where before it just worked fine. The metadata lock was caused by %truncate table% statement where the actual table was different every time. Once the lock was released (after many hours), execution of the tests continued and eventually finished successfully. With adding a WAIT clause to the truncate table statement and repeating the query, for which there is already code present, the problem seems fixed.
This resurfaced after switching to trixie and MariaDB 11.8. The variable [https://mariadb.com/docs/server/server-management/variables-and-modes/server-system-variables, lock\_wait\_timeout] should help so that at least the tests finish. Most likely there can be a pattern observed over days, because the same tests will fail again and again while other will have no problem. This will eventually lead to the root cause of the whole issue.
Switched back from connection reuse, "quasi" persistent, to dedicated open/close during unit tests. In production this was always the case. With persistent connections it is too risky, since during unit testing databases are permananetly setup and torn down in the same location (lots of DML) and since Topincs uses a abstract table schema (same for all domains) the likelihood of a metadata lock conflict for a table is higher. What i still do not understand why the locks always happend at the same few unit tests while the majority of the tests showed no problem despite following the same process. Only for the same five tests the metadata lock happens on every single run.
This is finally resolved. It was most likely caused by some change in MariaDB from 10.5 to 11.8 tightening the grip on DML changes. Anyway open/close is cleaner like also mentioned in [https://stackoverflow.com/questions/19801139/mysql-permanently-getting-waiting-for-table-metadata-lock, this article].
|
Work sessions7
| Start |
2025-06-29T14:55:23
|
| End |
2025-06-29T18:03:11
|
| Participant |
Robert Cerny
|
| Start |
2025-06-30T08:00:00
|
| End |
2025-06-30T09:24:09
|
| Participant |
Robert Cerny
|
| Start |
2025-07-02T08:51:50
|
| End |
2025-07-02T13:52:00
|
| Participant |
Robert Cerny
|
| Start |
2025-07-03T09:00:00
|
| End |
2025-07-04T11:00:00
|
| Participant |
Robert Cerny
|
| Start |
2025-07-05T10:00:00
|
| End |
2025-07-05T15:05:32
|
| Participant |
Robert Cerny
|
| Start |
2025-09-05T16:00:00
|
| End |
2025-09-05T18:13:02
|
| Participant |
Robert Cerny
|
| Start |
2025-09-06T11:13:20
|
| End |
2025-09-06T14:11:14
|
| Participant |
Robert Cerny
|
|
We are sorry
This page cannot be displayed in your browser. Use Firefox, Opera, Safari, or Chrome instead.