Troubleshooting Guide

This guide provides solutions for common issues encountered while using Liferay Docker Manager (LDM).

💾 Disk Space Issues (Elasticsearch Flood Stage)

If you see a warning about flood stage disk watermark exceeded in ldm doctor (or you receive a high disk space warning), Elasticsearch has marked your indices as read-only because your Docker environment is nearly full.

Reclaiming Space

LDM includes a proactive check in ldm doctor to warn you about dangling resources. If warned, clean up unused LDM resources and Docker volumes:

# 1. Prune LDM orphaned resources (snapshots, certs, containers)
# Use --seeds and --samples to also clear large asset caches
ldm prune --seeds --samples

# 2. Prune unused Docker system resources
docker system prune -a --volumes

If Elasticsearch is still stuck in read-only mode after clearing space, you may need to restart the global search container:

ldm infra-restart --search

Moving Docker to an External Drive (macOS / Colima)

If your internal disk is full and you have an external drive (e.g., /Volumes/SanDisk), you can move the entire Docker engine storage to it.

Option A: Symbolic Link (Recommended for Persistence)

This method "tricks" Colima into using your external disk without needing to manage environment variables.

1. Stop Colima:

   colima stop
   

2. Move the existing data (Optional: only if you want to keep your current images/containers):

   mv ~/.colima /Volumes/SanDisk/.colima
   

3. Create the Symlink: CRITICAL: Ensure ~/.colima does not exist before creating the link, or you will create a nested link.

   rm -rf ~/.colima
   ln -s /Volumes/SanDisk/.colima ~/.colima
   

4. Start Colima:

   colima start
   

Option B: Environment Variable

1. Stop and Delete the current environment:

   colima stop
   colima delete
   

2. Point Colima to the External Disk:

   export COLIMA_HOME="/Volumes/SanDisk/colima"
   mkdir -p $COLIMA_HOME
   

3. Start a new VM:

   colima start --disk 100 --memory 8 --cpu 4
   

4. Make it permanent: Add export COLIMA_HOME="/Volumes/SanDisk/colima" to your ~/.zshrc or ~/.bash_profile.

Moving LDM Configuration & Search Data (~/.ldm)

LDM stores shared infrastructure data (Elasticsearch indices, SSL certs, and project registries) in ~/.ldm. This can grow over time.

To move it to an external drive:

1. Stop all LDM projects.

2. Move the folder:

   mv ~/.ldm /Volumes/SanDisk/.ldm
   

3. Create the link:

   ln -s /Volumes/SanDisk/.ldm ~/.ldm
   

Moving Docker to an External Drive (Windows / WSL2)

Docker Desktop on Windows stores data in a WSL2 VHDX file. To move it:

1. Shutdown Docker Desktop.

2. Export the data:

   wsl --export docker-desktop-data "D:\docker-data.tar"
   wsl --unregister docker-desktop-data
   

3. Import to the new location:

   wsl --import docker-desktop-data "D:\Docker" "D:\docker-data.tar" --version 2
   

4. Restart Docker Desktop.

---

🗄️ Database Issues

"Could not synchronize Virtual Host entries to ..."

A restore rewrites Liferay's virtualhost table so the project answers on this machine's hostname rather than the one the snapshot was taken on. This warning means that rewrite did not land.

The restore itself succeeded. The data is in place and the project will start. What is wrong is the hostname Liferay believes it serves.

What you will see later. The site does not resolve at the address you expect, or resolves but behaves oddly -- OAuth2 redirect URIs for client extensions are registered against the hostname in this table, so they point at the snapshot's origin machine and authorisation round-trips fail. None of that is obviously connected to a restore that reported success, which is why the warning exists.

What to do. Re-running the restore is the supported fix, and is usually enough -- the sync runs again at the end of it.

To confirm the problem first:

ldm db query <project> -s "SELECT hostname FROM virtualhost;"

ldm db query is deliberately SELECT-only, so it will show you the wrong hostname but cannot correct it. If you need to change it by hand rather than re-running the restore, go through the container directly:

# PostgreSQL
docker exec <project>-db-1 psql -U lportal -d lportal \
  -c "UPDATE virtualhost SET hostname = '<your-host>';"

# MySQL / MariaDB
docker exec <project>-db-1 mysql -u lportal -ptest \
  -e "UPDATE lportal.virtualhost SET hostname = '<your-host>';"

Restart the project afterwards so Liferay re-reads it. The database client's own error is recorded in ~/.ldm/last-command.log.

Until LDM-#2141 this failure was silent: the UPDATE's result was discarded, and the line announcing the sync was printed before the statement ran, so a restore that left the wrong hostname looked identical to one that worked.

MySQL: Initialization Crash / "The designated data directory is unusable"

If your database container (<project>-db-1) continuously crashes with logs indicating Table 'mysql.plugin' doesn't exist or The designated data directory /var/lib/mysql/ is unusable, the initial database setup was interrupted or failed (often due to an incorrect configuration flag for that specific version).

Because the initialization aborted, the Docker volume was corrupted.

Solution: Wipe and Re-initialize

1. Stop the broken project.
2. Remove the project and explicitly delete its corrupted volumes.
3. Run the project again to trigger a clean database initialization.

ldm stop <project>
ldm rm <project> --volumes
ldm run <project>

"Could not confirm successful schema wipe. Restore may fail."

A snapshot restore empties the target schema before importing, because a dump that lacks DROP TABLE statements would otherwise land on top of the data already there. This warning means that step did not complete.

What it means for your data. The import still runs, so the result may be a database that merged two schemas rather than replaced one -- typically showing up later as duplicate-key errors, or as rows from a build you thought you had replaced. If you see this, treat the restored database as suspect rather than assuming the warning was cosmetic.

The usual cause is timing. A freshly created database container starts a temporary server to run its initialisation, then stops and restarts it, and refuses connections in between. LDM retries for about 30 seconds to ride that out. Seeing this warning means the wipe was still failing after all of them.

What to do. Check the container is healthy (ldm status), then re-run the restore -- on a container that is already up, the wipe succeeds immediately. If it persists, the error from the database client itself is recorded in ~/.ldm/last-command.log, and it is usually a credentials or disk-space problem rather than a timing one.

Until LDM-#2140 this warning could only appear for PostgreSQL. The MySQL and MariaDB wipe was issued once and its result discarded, so the same failure was silent on those engines.

---

🐳 Docker Filesystem Errors

Docker: failed to Lchown / overlayfs: no such file or directory

If your automated test or ldm run command crashes during the Pulling phase with a deep daemon error like: failed to extract layer (...) to overlayfs ... failed to Lchown ... no such file or directory

This is a low-level Docker daemon failure completely unrelated to LDM. It happens when Docker's internal cache or filesystem (overlayfs) becomes corrupted or runs out of inodes/space inside the Linux Virtual Machine.

Understanding Sparse Disks: Providers like Colima, Docker Desktop, and OrbStack use "Sparse Disks". Even if colima list shows a 100GiB disk and you have terabytes of space on an external drive, Docker pulls images strictly into this internal virtual disk file. If an image pull gets interrupted or corrupted, a broken ghost file remains in the cache.

Solution: Wipe the Corrupted Cache

Run the native Docker command (or use ldm prune) to blast away the corrupted system cache and force a clean download:

docker system prune -a --volumes

Note: This clears all unused images and volumes, resolving the corruption.

---

🌐 Network & Connectivity

Windows/WSL2: Connection Refused (Exit Code 7)

If LDM cannot connect to infrastructure services:

1. Ensure your WSL2 instance can reach the Windows host.
2. Check if a VPN or Firewall is blocking traffic on port 9200 (Search) or 443 (Proxy).
3. Run ldm infra-restart to reset the bridge.

Port conflict detected (Exit Code 4)

❌  Port conflict detected: Port 8080 is already in use on the host and is
    required by service 'liferay' in your compose configuration.
💡 Tip:  Re-run 'ldm run' -- the pre-flight check will select port 8081
         instead. Or free up port 8080 and re-run to keep it.

LDM checks host ports twice, and they behave differently on purpose:

Check	On conflict
Pre-flight, before any work	Moves to the next free port and says so
Compose validation, just before start	Fatal, exit 4

The late check is fatal because by that point the port is written into the generated docker-compose.yml; moving it means regenerating the compose file, not just picking a different number.

Seeing the fatal one usually means the pre-flight passed silently -- the port really was free when it ran -- and something took the port during the work in between, which includes a seed download that can take minutes.

Re-running is what resolves it: the pre-flight runs again and picks the free port named in the tip. You do not need to hunt down and kill the other process unless you specifically want to keep the original port.

If the tip names no alternative, every port in the 20 above the conflicting one was also busy; free one up, or set a different port for the service.

Stack bring-up timed out (Exit Code 124)

❌  Command timed out after 1800s: docker compose up -d --remove-orphans

The bring-up ran for 30 minutes without finishing. LDM stopped waiting and exited 124 rather than hanging indefinitely (LDM-#2064).

This is a liveness bound, not a performance limit. It is long enough that a cold image pull and an image build should never reach it, so hitting it almost always means something stopped making progress rather than being slow.

Against a remote node it is more likely still: with DOCKER_HOST set to ssh://, that one command carries the pull, the build and container creation over a single connection. The signature is no output at all after Starting Container Stack.

Start with the client's own process tree, not the network. This ordering is deliberate and was learned the expensive way -- see the note below.

ps -eo pid,etime,stat,wchan:20,args | grep -E 'docker|buildx|ssh'

wchan is what makes this a diagnosis rather than an observation. Without it you see a stuck process; with it you see what it is stuck on:

what you see	what it means
docker-buildx bake in futex_do_wait, long etime	the image build has wedged. The transport is irrelevant
ssh ... dial-stdio in poll_schedule_timeout	its children waiting on a reply that will not come -- a symptom of the above, not the cause
no ssh process at all, docker still running	the transport went and the client did not notice

Then, and only then, the network:

1. Is the node reachable? ldm system doctor and docker --context <node> info.
2. Was the daemon still working? On the node, journalctl -u docker -S -30min.
3. SSH keepalives. A Host <node> block in ~/.ssh/config with ServerAliveInterval 30 and ServerAliveCountMax 6 bounds a dead peer at about three minutes. Docker's connection helper reads that file, so this needs no LDM change -- but note it does nothing for a wedged build, which is the case above.

Why this order. A consumer spent four days on SSH remedies -- keepalives, MaxStartups, ControlMaster -- for a 200-minute hang whose transport was healthy throughout: ESTAB in all 199 samples, control socket present in 198, send queue always draining. The stall was docker-buildx bake blocked in futex_do_wait for over three hours, above the transport. Every SSH remedy was already in place on the run that proved it. A healthy transport does not mean a healthy build, and the process tree says which you have in one command.

Where the build output went. With no --follow the bring-up runs captured, and LDM writes the full captured stdout to ~/.ldm/last-command.log -- silently, regardless of verbosity. That file is truncated on every ldm invocation, so copy it before running anything else. On a failed command stderr is printed to the terminal verbatim.

Locally, a genuine 30-minute bring-up means a very slow pull or a Dockerfile doing heavy work. Pull the image first with docker pull, outside the run.

If your build legitimately needs longer, raise the bound rather than patching LDM:

LDM_BRINGUP_TIMEOUT=5400 ldm run my-project

0 removes the bound entirely. That restores the unbounded wait this guard exists to prevent, so LDM warns when you set it.

Why a bound beats a hang, even when you lose the run either way

The obvious benefit is time. The larger one is evidence.

A job that hangs gets cancelled, and a cancellation kills the script mid-command -- so none of its teardown runs. No diagnostics are gathered, no logs are captured, nothing is uploaded. A bounded failure exits 124 and the script carries on to its own failure path, where all of that executes.

Measured on a consumer's CI, where this defect was reported:

run	outcome	log artifact
110 minutes	cancelled by hand	166 bytes
23 minutes	failed normally	347 KB

Same pipeline, same captures configured. The difference is entirely whether the script reached its teardown. A hang does not just cost you the run -- it costs you the ability to find out why.

Observation and figures contributed by the AI Commerce Accelerator team, who reported the original hang (LDM-#2064).

📂 Permission & Mount Issues

macOS / ExFAT: "Unable to create lock manager" or "access_denied_exception"

If Liferay fails to start with one of the following errors:

• java.io.IOException: Unable to create lock manager (Equinox / OSGi State)
• access_denied_exception on write.lock (Elasticsearch / Data)

This is caused by the lack of POSIX file locking support on ExFAT filesystems or limitations in the macOS-to-Linux filesystem sharing layer (VirtioFS/SSHFS). Liferay's core components require low-level mandatory file locks that these environments cannot provide for host-mapped directories.

Solution: Use the Hybrid Mount Strategy (v2.7.2-beta.20+)

LDM automatically resolves this by using Docker Named Volumes for the most sensitive directories. These volumes live entirely inside the Docker Linux VM's native filesystem.

1. Verify your LDM version: ldm version (Should be v2.7.2-beta.20 or higher).

2. Reset the project state (if migrating from an older version):

   ldm stop <project>
   ldm rm <project> --volumes
   ldm run <project>
   

Note: Your project code (deploy/, modules/, etc.) can remain on the external drive; only the internal state and search indices are moved to Docker internal storage.

macOS 12 Monterey (Intel): Read-Only Volumes

On older macOS versions (12.x and below), Colima relies on the QEMU backend which presents a "Catch-22" for Liferay Docker Manager:

1. sshfs mounts: Appear as read-only to the liferay container user (UID 1000), preventing the project from starting and blocking snapshots.
2. 9p mounts: Allow write access but do not support POSIX file locking (flock / fcntl), causing databases like Elasticsearch to crash on boot with AccessDeniedException on node.lock.

Because LDM requires both writable bind mounts and POSIX file locking, Colima on macOS 12 (Intel) is explicitly unsupported.

Solution: Use Docker Desktop or OrbStack

To run LDM on macOS 12 Monterey, you must use a Docker provider that supports gRPC FUSE, osxfs, or a custom optimized filesystem:

1. Docker Desktop (Recommended for macOS 12)
2. OrbStack (Lightweight alternative)

Once installed, simply uninstall Colima (brew uninstall colima) and start your new provider. LDM will detect it automatically.

ldm monitor: "Fatal: OS file limit reached even with Polling"

ldm monitor watches your workspace for new build artifacts. Recursive watching needs one file descriptor per directory, so very large workspaces (deep node_modules, many modules/*/build trees) can exhaust the per-process limit and raise OSError: [Errno 24] Too many open files.

LDM degrades in two steps before giving up:

1. On Linux and Windows it starts with the native OS watcher, and on hitting the limit it automatically falls back to a polling watcher, which holds no per-directory descriptors. You will see Hit system file limit. Switching to PollingObserver... — this is a recovery, not a failure, and monitoring continues.
2. macOS starts on the polling watcher already (the native Kqueue backend hits descriptor limits far too easily), and raises its own soft limit to 4096 at startup.

If you see the fatal message, the limit was hit with polling already active — there is no cheaper watcher left, so LDM stops rather than silently leaving part of your workspace unwatched (which would drop artifact syncs without warning).

To resolve it, raise the descriptor limit in the shell running ldm monitor:

# Inspect the current soft/hard limits
ulimit -Sn; ulimit -Hn

# Raise the soft limit for this shell (must not exceed the hard limit)
ulimit -n 10240

Then narrow what is watched by keeping generated trees out of the workspace branches LDM scans (client-extensions/, modules/, fragments/). LDM already skips node_modules, build, .gradle, .git, .idea, and .vscode while polling, so a stray build output somewhere outside those names is the usual culprit.

---

🖥️ Platform Specifics

Windows/WSL: mkcert SSL "Not Secure" in Chrome

When running LDM in WSL (Linux) but accessing the site via Chrome on the Windows host, the browser may report the connection as "Not Secure". This happens because WSL and Windows use separate Certificate Authorities (CAs). mkcert in WSL generated a Linux root CA that Windows does not trust.

First: check which CA your certificates are actually signed by

Both fixes below depend on knowing this, and it is the step that is usually skipped. Setting CAROOT does not retrospectively re-sign certificates that already exist -- LDM reuses a certificate it finds on disk, so an export that looks correct can sit alongside certificates still signed by the Linux CA.

1. Ask mkcert which CA is actually in effect in WSL:

   mkcert -CAROOT
   

   If this prints a Linux path (e.g. /home/<you>/.local/share/mkcert) rather than your /mnt/c/... path, the CAROOT export has not taken in this shell. ldm doctor reports the same value.

2. Ask the certificate itself who signed it:

   openssl x509 -in ~/liferay-docker-certs/<your-host>.pem -noout -issuer
   

   The issuer names the machine and user the CA was created for. If it names your WSL distro while mkcert -CAROOT points at Windows, the certificate predates the change and must be regenerated.

3. Regenerate with the correct command:

   ldm renew-ssl
   

   This deletes the host's certificate and key from ~/liferay-docker-certs and re-issues them against the CA currently in effect. Traefik picks the new files up automatically.

Solution A: Sync WSL to use the Windows CA

Point WSL's mkcert at the Windows trust store so that everything LDM generates is trusted by your Windows browsers.

1. Ensure mkcert is installed on the Windows host and initialized (mkcert -install in PowerShell).

2. Open your WSL terminal and set the CAROOT environment variable to point to the Windows mkcert directory. Replace <YourWindowsUsername> with your actual Windows username. Note: If your Windows username contains spaces (e.g., John Smith), you must wrap the path in quotes!

   export CAROOT="/mnt/c/Users/<YourWindowsUsername>/AppData/Local/mkcert"
   # Example with spaces: export CAROOT="/mnt/c/Users/John Smith/AppData/Local/mkcert"
   

3. Re-initialize mkcert in WSL so it registers the Windows CA in the Linux trust store:

   mkcert -install
   

4. To make this permanent, add the export to your WSL profile (~/.bashrc or ~/.zshrc):

   echo 'export CAROOT="/mnt/c/Users/<YourWindowsUsername>/AppData/Local/mkcert"' >> ~/.bashrc
   

5. Regenerate the certificates so they are re-issued against the Windows CA:

   ldm renew-ssl
   

   Then re-run the checks at the top of this section. If mkcert -CAROOT still reports a Linux path, Solution A has not taken -- use Solution B.

Solution B: Trust the WSL CA from Windows instead

Contributed by a user running LDM under WSL, where Solution A did not hold: certificates continued to be issued against the Linux CA despite CAROOT being exported.

This is the same problem solved from the opposite end. Rather than making WSL use the Windows CA, make Windows trust the CA that WSL is already using. It is the more reliable option when mkcert in WSL keeps falling back to its own root, and it is a one-off -- the CA is long-lived, so this does not need repeating per project or per certificate.

1. In WSL, find the CA directory and confirm the root certificate is there:

   mkcert -CAROOT
   ls "$(mkcert -CAROOT)"        # expect rootCA.pem and rootCA-key.pem
   

2. Copy only rootCA.pem somewhere Windows can reach. rootCA-key.pem is the CA's private key -- it is not needed for trust and should not be copied around.

   cp "$(mkcert -CAROOT)/rootCA.pem" /mnt/c/Users/<YourWindowsUsername>/Downloads/wsl-rootCA.pem
   

3. Register it in the Windows Trusted Root Certification Authorities store. In an Administrator PowerShell:

   certutil -addstore -f "ROOT" "$env:USERPROFILE\Downloads\wsl-rootCA.pem"
   

   Or via the GUI: certlm.msc -> Trusted Root Certification Authorities -> Certificates -> right-click -> All Tasks -> Import, and select the file.

4. Restart Chrome completely (it caches the trust store) and reload the site.

Note

Solution B trusts a CA whose private key lives inside your WSL filesystem. That is the same trust you already extend to mkcert on any development machine, but it is worth stating: anyone who can read rootCA-key.pem in that WSL distro can issue certificates your Windows host will trust. Remove it from the store (certlm.msc, or certutil -delstore "ROOT" <thumbprint>) if you stop using that distro.

Windows: "charmap" codec can't encode character

If you see Python encoding errors (like '\u25cf' character maps to <undefined>) in your terminal when running ldm status or ldm doctor:

1. Upgrade to LDM v2.4.26-beta.37 or later. LDM now detects legacy terminal encodings and automatically switches to ASCII fallbacks (e.g., using * instead of ●).
2. Use a modern terminal. We highly recommend Windows Terminal with PowerShell 7, which supports full Unicode output.

---

🔍 Search (Elasticsearch)

"Refusing to wipe the shared search data while other LDM projects are running"

LDM stopped before deleting ~/.ldm/infra/search/data and exited 3. This is a guard, not a failure of the repair.

Why it exists. When Elasticsearch misses its readiness window, LDM runs an automatic search volume repair that deletes that directory and starts a clean node. The directory is shared: it holds the indexes of every project on the machine, not just the one you are working on. Deleting it destroys all of them, and nothing tells you — a project's healthcheck is an HTTP probe against the portal that never touches search, so every project keeps reporting healthy until somebody runs a search and gets nothing back.

What to do. Stop the projects it names, then re-run. The refusal prints the exact commands:

ldm stop <project>

If you accept the loss, re-run with --force. Each project rebuilds its index on its next boot. ldm reindex only schedules a rebuild for the next start — it does not rebuild anything at the time you run it.

Diagnosing why the repair triggered. The repair writes the failing container's logs to ~/.ldm/infra/search/search-failure-<timestamp>.log before removing the container. Read that first: a readiness timeout on a loaded machine is not corruption, and in that case the right fix is usually to re-run rather than to wipe anything.

"Global Search's disk is N% full"

LDM refused to boot a project against shared search because the disk Elasticsearch writes to is too full (LDM-#2247). It checks this on every boot:

• Past the high watermark (90% by default), Elasticsearch will not create a new index. A project whose index already exists still boots; a new project is refused, because its Liferay would wait for an index that never appears and hang for its whole healthcheck window.
• Past flood stage (95% by default), every index is read-only, so every project is refused.
• Past the low watermark (85%), LDM warns.

The thresholds are read from the cluster, so custom watermarks are respected. Free space where Docker keeps its data, then re-run:

docker system df                 # what is using it
ldm system prune -y --images     # unused images and build cache

On macOS with OrbStack or Colima, Docker's disk lives inside a VM: what Elasticsearch sees is the VM's disk limit, or the Mac's free space for a dynamically sized disk.

Elasticsearch: Failed to obtain node locks

If Elasticsearch fails to boot with AccessDeniedException: /usr/share/elasticsearch/data/node.lock:

1. Ensure LDM is up to date. LDM v2.7.2-beta.24+ includes Just-in-Time (JIT) Permission Hardening. If LDM hits a "Permission Denied" error on Linux/macOS while writing to a directory, it will automatically attempt to reclaim the folder's permissions via a temporary helper container before retrying.
2. Fedora / Linux Workstations: If Docker Desktop is not used, search indices are created as root by the Docker daemon. LDM's JIT hardening is specifically designed to resolve these ownership mismatches on native Linux filesystems.
3. Manual Fix: If the automated hardening is bypassed, ensure the host directory has open permissions:

sudo chmod -R 777 ~/.ldm/infra/search/data

Elasticsearch: Failed to parse mappings

First, check whether a plugin is missing. If the failure log (~/.ldm/infra/search/search-failure-<timestamp>.log) contains No mapper found for type [icu_collation_keyword], the data is fine: the container started without the analysis-icu plugin that Liferay's indexes use. Deleting the data does not fix that; it only destroys every project's index. LDM recognises this case and refuses to wipe, exiting 3 (LDM-#2284).

The plugins are kept in ~/.ldm/infra/search/plugins-<version> and installed there before Elasticsearch first opens the data, so a recreated container starts with them. They are downloaded on first use. If the refusal appears, check the network and re-run ldm infra-setup --search.

If Elasticsearch fails to start after a crash or version change due to some other mapping error:

LDM includes an Auto-Repair feature. If the health check fails for 5 minutes, LDM will automatically wipe the ~/.ldm/infra/search/data directory and perform a clean boot. You can trigger this manually by running:

ldm infra-down
rm -rf ~/.ldm/infra/search/data
ldm infra-setup --search

"Global Search is running, but its cluster is RED"

LDM refuses to continue when the shared Elasticsearch answers on its port but its cluster is red:

❌  Global Search is running, but its cluster is RED -- Liferay cannot start against it.
    Cluster status: red. 17 unassigned primary shard(s), 0 active.
    Elasticsearch: the node is above the high watermark cluster setting [...]

This is a refusal, not a failure to start. Before LDM-#2247 the readiness check asked only whether Elasticsearch answered GET /, which a red cluster does perfectly happily. LDM reported "Infrastructure setup complete", and Liferay then failed in a way that pointed everywhere except search: ElasticsearchSearchEngine.activate() waits for a yellow cluster, times out, and takes SearchEngineHelperImpl, IndexSearcherHelper, FacetedSearcherManager, Searcher, AssetHelper and every portlet behind them down with it — retrying every 30 seconds, so the portal never finishes starting. Three verification runs were investigated as disk pressure and then as memory exhaustion before anyone read the cluster status.

Causes worth checking, in order:

Cause	Tell	Fix
Disk watermark	a disk_threshold explanation in the refusal	free space, then ldm prune --seeds --samples
Shard data gone from under a live cluster	unassigned primaries with no watermark named	remove the stale indices on the search node; ldm reset <project> global-search clears one project's

ldm system doctor prints the full allocation report.

If LDM instead says it could not read the cluster status, that is not a pass: the check did not run, and the boot continues unverified.

Elasticsearch: index_not_found_exception / Cascading OSGi Failures

If Liferay's ElasticsearchSearchEngine fails to activate, causing dozens of cascading NullPointerException and ServiceException logs, the search index is likely missing or out-of-sync with the database. This commonly happens if a pre-warmed seed lacked the corresponding search snapshot.

Solution 1: Surgical Search Reset (Recommended)

If you want to keep your database intact but rebuild the broken search indices from scratch:

ldm stop <project>
ldm reindex <project>
ldm run <project>

To clear the indices outright rather than rebuild them, use ldm reset <project> global-search, which deletes this project's indices from the shared search node and leaves every other project's alone. That target did nothing at all — while reporting success — until LDM-#2127; if you are on an LDM older than v2.26.7, remove the indices on the search node directly instead.

Note also that a reindex rebuilds what Liferay can index; it does not purge documents whose classes no longer have an indexer (LDM-#2119). reset global-search does remove those, because it deletes the indices rather than re-populating them.

Solution 2: Total Project Reseed

If the database and search index are completely desynchronized (often resulting in NoSuchResourcePermissionException logs), wipe all data and start fresh:

ldm stop <project>
ldm reseed <project>

---

🚀 LDM Self-Upgrade

Failed to check for updates (GitHub API Rate Limiting)

If you are running LDM behind a corporate proxy, VPN, or shared public IP address, you might occasionally see the following error when checking for updates or upgrading:

=== LDM Self-Upgrade ===
❌  Failed to check for updates.
ℹ  Please check your internet connection or try again later.

This occurs when the shared IP address hits the GitHub unauthenticated REST API limit (60 requests/hour).

The Solution: LDM automatically falls back to an HTML redirect check on https://github.com/peterrichards-lr/liferay-docker-manager/releases/latest (which is not rate-limited). If the fallback also fails, verify your network connectivity or try again later.

---

🌍 Sharing & Tunnels

Issue: "Downstream Offline: Local target port 8080 is not responsive"

The tunnel reached the gateway and leased its subdomain; what it could not find was anything to forward to. The message describes the gateway's view of your machine, not a problem with the tunnel.

Almost always this means the project is not running. Since LDM-#2009 ldm share start checks that locally first and refuses with exit code 3 before contacting anything:

Project 'my-project' is not running, so there is nothing to share.
  Start it first:
      ldm start my-project

If the project is running and you still see the gateway message, the tunnel is pointed at the wrong port. Check the project's port in its meta, and pass --ports explicitly if the service you want exposed is not the one LDM would auto-discover.

---

🚦 The Global Proxy (Traefik)

Issue: docker logs liferay-proxy-global prints nothing

Almost certainly working as configured. Traefik's own default log level is ERROR, and LDM sets no --log.* flag, so the proxy is silent unless something fails.

An empty log and an unreadable one are different facts, and they are easy to tell apart: an unreadable logging driver exits non-zero with configured logging driver does not support reading. Exit 0 with no output means the driver is fine and Traefik has genuinely written nothing.

Issue: a container has correct labels and gets no route

The symptom is a container that is running, on liferay-net, with well-formed traefik.* labels, for which Traefik loads no router -- and an empty proxy log, because Traefik does not consider any of the likely causes an error.

LDM runs the proxy with --providers.docker.exposedbydefault=false, so a container is routed only if the provider sees traefik.enable=true.

Raising the level buys nothing for error-shaped causes, and everything for the rest -- ERROR is already on, so anything Traefik considers an error is already in a log you have read. Measured against traefik:v3.6.1 with LDM's production flag set:

cause	logs at	router appears?
a malformed traefik.* label	ERR	no
traefik.docker.network naming a network the container is not on	WRN	yes, defaulted
the provider filtering the container out	DBG only	no

So a container with correct labels and no route points at the last row, which is invisible at every level you have been running:

DBG Filtering disabled container container=<name>-<id>

LDM_PROXY_LOG_LEVEL=DEBUG ldm infra restart-proxy

INFO also exists, but DEBUG is the level to reach for -- reaching for INFO to chase a label problem finds exactly what ERROR already showed.

Warning

DEBUG is voluminous, and it will break a capture sized for a silent proxy. Over a long run it produces many thousands of lines, and the events worth reading are at stack bring-up, not teardown -- so docker logs --tail N takes the wrong end. Measured on a 45-minute CI run, a 120-line tail covered the final few seconds. Select on content rather than tailing -- but see the warning below about how.

Warning

The obvious grep matches nothing. Traefik's default common format writes ANSI colour escapes between key and value:

ESC[36mrouterName=ESC[0mmy-service-svc

So grep 'routerName=my-service-svc' finds nothing while grepping the bare container name works. Either strip ANSI first, or -- better for anything machine-read -- ask for structured output:

LDM_PROXY_LOG_LEVEL=DEBUG LDM_PROXY_LOG_FORMAT=json ldm infra restart-proxy

Set these only for the run you are diagnosing. The defaults are ERROR and common for everyone else.

Issue: a route exists but reaches the wrong backend

The middle row of the table above deserves its own mention, because it fails in the least helpful direction. A traefik.docker.network label naming a network the container is not attached to does not stop routing:

WRN Could not find network named "X" for container "/Y". Maybe you're missing the project's prefix in the label?
WRN Defaulting to first available network (...) for container ...

Traefik warns and routes it anyway, to a defaulted IP. At the default level both lines are invisible, so the symptom is a route that exists and serves possibly the wrong backend -- worse than an absent route, which at least fails where you can see it. Most likely after adding a second network to a project. LDM_PROXY_LOG_LEVEL=WARN is enough to surface it.

---

🛡️ EDR / SentinelOne Quarantine (lfr-tunnel)

Issue: "Failed to verify lfr-tunnel installation after download"

When using --share or ldm share start with the default --share-provider lfr-tunnel, LDM downloads a native host-side Go client executable (lfr-tunnel). Because this binary is compiled and downloaded dynamically, corporate Endpoint Detection and Response (EDR) platforms like SentinelOne or Microsoft Defender may flag it as an unsigned/untrusted executable and quarantine it.

This causes the executable to be deleted or blocked immediately after download, resulting in the error: ❌ Failed to verify lfr-tunnel installation after download.

Solution: Use the Containerized Provider (lfr-tunnel-docker)

The easiest and safest workaround is to run lfr-tunnel inside a Docker container sidecar instead. EDR tools do not inspect or quarantine binaries running inside the isolated Docker Compose network:

1. Stop the current project.
2. Run the project with the --share-provider lfr-tunnel-docker parameter:

ldm run <project> --share --share-provider lfr-tunnel-docker --share-subdomain <subdomain>

Alternatively, if you are sharing a running project using ldm share start, explicitly request the Docker-based provider:

ldm share start <project> --provider lfr-tunnel-docker --subdomain <subdomain>

---

🔒 Project Concurrency Locks

Issue: "Concurrency Violation: Another instance of LDM is running on this project"

If a previous ldm process was hard-killed (e.g. via kill -9 or a VM reboot/OOM event) or crashed, the project concurrency lock file might remain on disk. While LDM includes stale-lock auto-recovery (automatically clearing the lock if the process is no longer running), in rare network/virtualized filesystem configurations the lock check can block subsequent commands.

The Solution: You can force-clear the project lock using the rescue command:

ldm rescue --clear-lock [project]

Alternatively, you can manually delete the lock file located in your project work directory:

rm -f <project-path>/.liferay-docker/.ldm_lock

📂 Project Resolution

Issue: "Project '' not found"

LDM could not resolve the name you passed to a command. The message deliberately stays short, because the list of places LDM looked is diagnostic detail rather than something you can act on.

The Solution:

Start with the list of projects LDM actually knows about:

ldm list

If the project you expected is missing from that list, it is not registered and not sitting in a searched directory. If it is in the list but the command still cannot find it, the registry entry is stale -- the directory has been moved or deleted since it was registered. Running ldm list again prunes entries whose path no longer exists.

To see every location that was searched, including the global registry, re-run with --verbose (or --info):

ldm --verbose info <name>

❌  Project 'my-project' not found.
Details:  Looked in:
  - /Users/you/ldm
  - /Users/you
  - /Volumes/SanDisk/ldm
  - the current folder
  - the project registry (/Users/you/.ldm/registry.json)

Projects are discovered from the current folder, its parent, ~/ldm, the LDM installation directory, and the global registry at ~/.ldm/registry.json. Setting LDM_WORKSPACE overrides all of those and searches that directory exclusively.

---

Last Updated: 2026-10-23 | Last Reviewed: 2026-10-23