Database storage troubleshooter on Adobe Commerce

This article provides a step-by-step troubleshooting guide for diagnosing and resolving database storage issues on Adobe Commerce. It helps identify root causes related to a lack of space in /tmp and /data/mysql, inode exhaustion, and configuration changes such as non-default MySQL tmpdir. Each step presents a diagnostic question and directs you to relevant commands, logs, or documentation based on your answer.

Description description

Environment

Adobe Commerce on Cloud Infrastructure

Issue/Symptoms

  • Site outage or downtime caused by database storage issues.
  • Errors such as No space left on device (28).
  • Unable to SSH into a node due to full /tmp mount.
  • High disk usage in /tmp or /data/mysql directories.
  • Customers unable to add products to cart.
  • Database connection failures.
  • Galera cluster errors like SQLSTATE[ 08S01] : Communication link failure: 1047 WSREP
  • Queries exhausting available storage and causing node crashes.
  • Inode exhaustion (too many files) indicated by IUse% > 90%.

Resolution resolution

Step 1 - Identify the directory with a space issue

Do you have a /tmp issue caused by a lack of space?

This can be indicated by a range of symptoms including:

  • /tmp mount being full
  • Site down or outage
  • Unable to SSH into a node
  • Errors like No space left on device (28)

For a list of errors resulting from /tmp being full, review /tmp mount full.

Or do you have a /data/mysql issue caused by a lack of space? Symptoms include:

  • Site outage
  • Customers unable to add products to cart
  • Database connection failure
  • Galera errors like SQLSTATE[ 08S01] : Communication link failure: 1047 WSREP

For a list of errors resulting from low MySQL disk space, refer to MySQL disk space is low on Adobe Commerce on cloud infrastructure.

If you are unsure if you have a disk space issue and you have a New Relic account, go to the New Relic Infrastructure monitoring Hosts page. From there:

  • Click on the Storage tab
  • Change the Chart Shows dropdown from 5 to 20 results
  • Look in the table for high disk use in the Disk Used % chart or table

For more detailed steps, refer to New Relic Infrastructure Monitoring > Storage tab.

If you have any of the symptoms described above, check the state of your inodes to make sure this is not being caused by a file number issue. Run in the CLI/Terminal:

code language-none
df -ih

Is IUse% > 90%?

  • YES – Too many files. Review steps to remove files safely in Safely delete files when out of disk space. Proceed to Step 2 after completing these steps. If you want to request more space, submit a support ticket.

  • NO – Check disk space usage in /tmp and /data/mysql directories. Run in the CLI/Terminal:

    code language-none
    df -h | grep mysql
    df -h | grep tmp
    

Then proceed to Step 3.

Step 2 - Check disk space

Check disk space usage?

After reducing the number of files, run the following command in the CLI/Terminal to check disk space usage in /tmp and /data/mysql.

code language-none
df -h | grep mysql
df -h | grep tmp

Is greater than 70% used for /tmp or /data/mysql?

  • YES – Proceed to Step 3.

  • NO – Queries may be exhausting available storage. This could crash the node, killing the query and removing the tmp files. Log in to the MySQL command-line client and run:

    code language-none
    SHOW PROCESSLIST;
    

Examine the output for problematic queries. If unresolved, submit a support ticket requesting more space.

Step 3 - Identify directory with high usage

Which directory has greater than 70% used?
  • /tmp – Proceed to Step 4.
  • /data/mysql – Proceed to Step 5.

Note: By default, database tmpdir writes to /tmp. To check your database configuration, log in to the MySQL command-line client and run:

code language-none
SHOW VARIABLES LIKE "TMPDIR";

If tmpdir is still writing to /tmp, you will see /tmp in the Value column.

Step 4 - Troubleshoot /tmp mount full

Troubleshoot /tmp mount full

Follow Troubleshoot /tmp mount full for Adobe Commerce. After applying solutions, run in the CLI/Terminal:

code language-none
df -h | grep mysql
df -h | grep tmp

Is usage < 70%?

Note: The solutions in Troubleshoot /tmp mount full are designed for merchants who have not changed the variables for database tmpdir, which by default writes to /tmp. If you have changed the tmpdir value, the instructions in Troubleshoot /tmp mount full will not help.

Step 5 - Check default

Check default

Your database configuration may no longer be at the original default. Find the database tmpdir config by running in the MySQL command-line client: SELECT @@DATADIR;

If /data/mysql/ is outputted, tmpdir is writing to /data/mysql/. Increase space by following the steps in the MySQL disk space is low on Adobe Commerce on cloud infrastructure. Then run in the CLI/Terminal:

code language-none
df -h | grep mysql
df -h | grep tmp

Is usage < 70%?

Best practices for modifying database tables in the Commerce Implementation Playbook.

recommendation-more-help
3d58f420-19b5-47a0-a122-5c9dab55ec7f