Remote Node Architecture: Target Resolution¶

Status: Approved design — Phase 1 (TargetContext + resolve_target_context()) implemented. See Migration plan for remaining phases.

Scope: How LDM resolves and uses --node/--target compute targets across the codebase.

1. The core problem¶

Remote-node support was added as a parameter threaded through existing code paths, not as its own concept. There was no single place that decided "what target does this operation use, and how do I reach it" — that decision was re-derived independently at every call site, using whichever of three overlapping signals (explicit CLI flag, project metadata, persisted default) that particular call site happened to remember to check, in whatever order its author happened to pick.

The identical bug appeared independently in three different files, because there was no one function doing this resolution:

File	Symptom	Root cause
`handlers/base.py` (`_pre_flight_checks`)	Brand-new project's `ldm run --node X` ran the local-only port check anyway	`meta["target"]` is never populated during `run` itself, and `self.target` wasn't checked first
`handlers/composer.py` (`write_docker_compose`)	Bind-mount paths stayed local even with a persisted default target set	`get_active_target()` was only called inside `if target_name:` — skipped entirely when nothing explicit was set, so the persisted-default fallback never ran
`diagnostics/info.py` (`run_list`/`run_status`/`run_info`)	`ldm list` displayed `Target: local` while the project was actually running on the persisted default `node-b`	`meta.get("target", "local")` — passing the literal string `"local"` into `get_active_target()` makes it treat that as an explicit choice, which short-circuits its own persisted-default check

Three fixes, three files, one underlying cause. Without a shared abstraction, there will be a fourth place with the same bug the next time someone touches this.

2. Full inventory of the problem's actual size¶

A dedicated audit (covering every docker/docker compose invocation in ldm_core/ outside tests) found:

~18 call sites using get_compose_cmd() (ldm_core/utils.py), which is not context-aware at all — always returns ["docker", "compose"] regardless of any active target.
A second, distinct bug class: call sites that do go through the context-aware DockerService.* methods, but simply never pass the target_name parameter even though it was sitting in scope (usually from an already-read meta dict two lines above).
InfraService.target (handlers/infra.py:18) is declared but never assigned anywhere in the codebase — dead plumbing.
Bind-mount path remapping for remote targets required a hand-rolled, duplicated dict (mount_paths) built inline inside composer.py's write_docker_compose(), mirroring setup_paths()'s structure by hand rather than through any shared abstraction.

High-impact clusters still using the old pattern (full list preserved on issue #1133):

runtime/orchestration.py: cmd_stop, cmd_restart, cmd_down, cmd_reset all hardcode local, directly inconsistent with cmd_start in the same class, which already resolves the target correctly.
handlers/infra.py: setup_global_database/setup_global_search — every Docker call hardcoded local, even when called from an already target-aware caller. A project correctly targeting a remote node still provisions its shared DB/search locally.
dashboard/server.py: api_projects/api_logs — reads and displays meta.get("target") in the same function that queries the wrong (local) daemon for status.
runtime/readiness.py: cmd_wait/_wait_for_ready — the core post-run health-polling loop, every Docker call raw/local throughout.
A long tail of medium-impact call sites in runtime/fragments.py, runtime/search.py, handlers/share.py, snapshot/*.py, handlers/mcp.py, diagnostics/doctor.py.

Separately, categories that are ambiguous by design, not just unfixed — these need a product decision, not a code fix: diagnostics/prune.py, handlers/system.py's cmd_nuke/cmd_rescue, handlers/database.py's shared-DB cmd_start/cmd_stop. None of these currently have any target concept at all, and it's not obvious they should — "prune the orchestrating host's own Docker Desktop" is arguably correct as a host-local operation regardless of any project's target.

3. Additional UX gaps found via live use¶

Silent remote provisioning. When a persisted default target is set via ldm target use, ldm run on a plain project (no explicit --node) silently provisions against that remote target with no upfront indication — the first sign is --verbose output showing docker --context node-b ... mid-run. A user who forgot they'd set a default, or inherited a machine with one already set, has no warning before real infrastructure gets touched.
Redundant double confirmation. Re-running ldm run against an existing, currently-running project shows two separate confirmations in sequence: an interruptible_pause ("already exists, CTRL+C to cancel") from ProjectInitializationStage, followed by a blocking UI.confirm ("already running, reconfigure and restart? [Y/n]") from RuntimeValidationStage. These fire independently because the two stages don't share what they each already know. The CTRL+C style pause is preferred where one is sufficient, since it doesn't block indefinitely on input.

Neither of these is a "remote node" bug specifically, but both stem from the same underlying issue: state (is this remote? is this already running? has the user already been warned once this command?) isn't tracked as a single resolved thing and passed along — it's recomputed piecemeal at each point that happens to need it.

3.1 What is now fixed¶

Two of the gaps above have been closed:

Announcement before blocking (LDM-#1341). list, status, start, stop, restart and down now emit a single UI.info line before any remote context is dialled, naming the project/node pairs involved:

ℹ  1 project targets a remote compute node (e2e-test-env -> node-a); this may
   take longer than usual, and will wait if the node is asleep.

It is emitted once per command rather than per project, and names the pairs so a project unexpectedly pointed at a node is itself visible. UI.info is deliberate: UI.detail is gated behind --info/--verbose (LDM-#1036) and would be invisible exactly when it is needed. It is suppressed under --json, which is a machine-readable contract (LDM-#1093).

Remoteness is decided by DockerService.get_docker_cmd_prefix() rather than re-derived, so there is one definition of "is this remote" -- including that all of 127.0.0.0/8 counts as local.

Diagnosis instead of a raw blob (LDM-#1345). A docker --context <node> command that cannot reach its node used to print the entire underlying failure: a URL-encoded Compose label filter, a docker.example.com placeholder host that looks alarming and is not real, and -- in the last eight words -- the one fact that mattered. It now reports:

❌  Cannot reach compute node 'node-a' over SSH (ec2-user@203.0.113.20:22 timed out).
💡 Tip:  Nothing answered within the connect timeout. That is usually a firewall
         or security group dropping the packets, or a node that is stopped and
         whose public IP has changed since it was registered. Check with
         'ldm target status node-a', then re-register with 'ldm target add node-a
         --host <ip> --user ec2-user --key <key>'.

The raw stderr is retained behind --verbose/--info. Unrecognised failures still print stderr verbatim, because for most commands -- a Compose config error, a failed build -- the stderr is the useful output. This narrows the cases LDM can name; it does not suppress detail generally.

Each reason carries its own tip (LDM-#1863). Originally every reason shared one, pointing at a stale stored host because that is the usual cause (LDM-#1346). It is not the usual cause on a node that has just been powered on, and that is precisely when the message is read: an EC2 instance ~15 seconds into a cold boot accepts a TCP connection on 22 well before sshd will authenticate, so a correct setup was reported as "the node may be stopped, or its public IP may have changed". The message already named the cause correctly -- only the advice was generic, and the advice is the half a caller acts on.

A refused connection now says sshd may not be up yet; refused credentials says the node is reachable and this is authentication rather than connectivity, and deliberately does not suggest re-registering, because connectivity is already proven. Host-key failures name the ssh-keygen -R entry to purge. _SSH_FAILURE_TIPS in ldm_core/utils.py holds the table, and a test asserts every reason LDM can name has advice of its own -- a reason without one is a half-finished diagnosis.

Waiting for a node that is waking (LDM-#1863). A transport failure is retried for up to LDM_SSH_READY_TIMEOUT seconds (default 30, 0 disables) before it is reported. See Remote Node Readiness for the knob; the reasoning belongs here.

Retrying an arbitrary failed command would be wrong -- it might have already done half its work. This is safe because of where it sits: the branch is reached only for error during connect, which means SSH never carried the command at all, so the remote side provably did not run it. The safety comes from the failure being a transport failure, not from the command being idempotent, and it does not generalise to the ordinary failure path below it.

Only causes that waiting can fix are retried -- refused connection, refused credentials, connect timeout. A changed host key, an unresolvable hostname and an absent route are excluded deliberately: they do not heal, so waiting on them converts a correct immediate error into a slow one.

When the budget is spent the original exit code, message and tip are emitted unchanged, so nothing downstream sees a different failure than before.

3a. Deploying a single artifact to a node¶

ldm deploy <project> <file...> used to refuse outright when the project ran on a node, because copying the artifact into the local project directory would have updated something the remote container never reads — silently. The guard was right; the capability behind it was missing, and the only guidance was ldm run for a full resync, which restarts the stack. That is a fair escape hatch for a developer and useless to a CI run that has already waited out a boot (LDM-#1894).

The command is the same whatever the target. The node comes from the project's own meta, so a caller never names it, and the local end state does not depend on where the project happens to run: the artifact is placed locally exactly as before — including expanding a client-extension zip into client-extensions/ for image builds — and a project on a node additionally has it shipped there.

It needs no container operation and no restart. osgi/modules and osgi/client-extensions are bind-mounts from the project directory, so a file landing in the node's copy is immediately visible to the running container. That is the property that makes this useful mid-run, and it is the reason the fix is a file placement rather than a docker cp.

Two details that are easy to get wrong, and are handled:

The upload is staged, then moved. Liferay watches these directories, so a partially transferred artifact is not untidy — it is a deploy of a truncated file. The helper copies to <name>.ldm-partial and mvs it into place, which is atomic within one filesystem, and removes the staged file if the move fails.
Ownership is settled afterwards. A file arriving over SSH belongs to the SSH user while Liferay runs as liferay, and Liferay ignores what it cannot read without saying so. LDM resets it through docker --context <node> exec, so callers do not need their own docker exec -u 0 … chown afterwards — which, done with bare docker, would hit the caller's own daemon where the container does not exist.

Doing the local placement regardless also keeps the two ends consistent: a later ldm run rsyncs the whole project, and would otherwise push a directory missing the expanded client-extension.

4. Resolution precedence¶

Three overlapping signals decide "what target should this operation use," ranked most to least specific:

Explicit --node/--target CLI flag — a per-invocation override, for a user who wants to target a specific node "just this once."
The project's own persisted meta["target"] — set via ldm target set, or pinned automatically the first time an unpinned project resolves a target (see Pinning below). This lives in the project's own metadata, not ~/.ldmrc, because a project's source/config always lives locally regardless of where it executes — where a specific project runs is a property of that project, not of the machine running LDM.
The persisted global default (ldm target use, i.e. TargetNode.is_default) — the "set and forget" fallback for a user who mostly works against one non-local node.
local — the ultimate fallback when nothing else is configured.

This is precisely get_active_target()'s own existing logic — the design here isn't a new algorithm, it's making sure every caller reaches it the same way (passing None through when nothing is known, never a hardcoded "local" string) and stops re-implementing the priority chain by hand.

Two metadata locations, by design, not a bug. ~/.ldmrc is the catalog of known nodes and which one is the global default. A project's own metadata file is where that specific project's target assignment lives. These aren't competing sources of truth — they're different questions ("what nodes exist, and which is the default?" vs. "where does this project run?").

Conflict¶

If an explicit --node disagrees with a project that's already pinned to a different target, LDM warns and gives the user a CTRL+C window before proceeding with the override for that run only. The project's synced files, named volumes, or other state may only exist on the pinned node, so silently switching would produce confusing "it's not there" failures. The override is not written back as a new pin — a one-off --node flag shouldn't silently and permanently reassign a project. Use ldm target set/ldm target migrate for that.

Pinning¶

If a project has no pinned target yet, whatever it resolves to — whether from an explicit --node or by falling through to the global default — is written back into the project's metadata. This stops a project's effective target from silently drifting if the global default changes later: without pinning, a project that only ever inherited the ambient default would appear to move to a different node the moment someone runs ldm target use for something unrelated, even though its actual files and volumes never moved. This is exactly the failure mode observed live: a project provisioned on node-b (the default at creation time) was later displayed as a local node project once other commands re-derived the target independently instead of reading a pinned value.

Open question: sync strategy for `deploy`/`monitor`¶

Commands like ldm deploy/ldm monitor may need continuous/incremental remote sync (not just the one-time sync_project_to_target() used at provisioning) to push files to a remote node before container deployment. This is explicitly not resolved by this design — it's deferred to whenever those specific commands are migrated (Phase 5). Existing SSH/rsync infrastructure (sync_project_to_target(), resolve_remote_home()) may be reusable as-is or may need an incremental mode.

Open question: scaffolding the routes tree on a remote node¶

The routes tree (see The Routes Tree in .agents/skills/ldm-architecture/SKILL.md) is a set of host directories that Liferay, every client extension and every custom service bind-mount subtrees of. Those directories must exist before the mount, because Docker creates a missing bind-mount source itself, as an empty root-owned directory — which is the same failure mode this document's mount_paths remapping exists to avoid.

_scaffold_routes_tree() (handlers/composer.py) creates them from the generated compose, but deliberately skips remote targets: it compares mount_paths["root"] against paths["root"] and returns when they differ, because creating a remote target's directory tree on this machine would be worse than not creating it at all.

So on a remote node the subtrees are currently left to Docker. Fixing it properly means creating them over the connection — the same unresolved question as the sync strategy above, and deferred with it. Anyone adding remote provisioning should create the routes subtrees as part of it.

5. One resolver function, every command calls it¶

resolve_target_context() (ldm_core/config.py) is the single function every command calls to find out what compute target it's running against, implementing the precedence chain, conflict warning, and pinning write described above. It returns a TargetContext:

@dataclass
class TargetContext:
    target: TargetNode  # the resolved node (local/remote)
    is_remote: bool
    docker_prefix: list[str]  # ["docker"] or ["docker", "--context", name]
    compose_prefix: list[str]  # docker_prefix + ["compose"]
    conflict_overridden: bool = False
    newly_pinned: bool = False
    local_root: Path | None = None
    remote_root: str | None = None

    def map_path(self, local_path: Path) -> Path | PurePosixPath:
        """Local targets: identity. Remote targets: rewrite onto the
        already-synced remote project root, replacing composer.py's
        hand-rolled mount_paths dict."""

What this fixes structurally, not just symptomatically:

The "hardcoded local before checking" bug class becomes structurally impossible — there's one function that resolves the target, and everything else just reads the result. Keeping this to one choke point also makes it far easier to debug and fix if new issues surface, rather than chasing the same class of bug across a dozen independently-written call sites.
composer.py's hand-rolled mount_paths dict is replaced by context.map_path(), removing duplicated path-construction logic and making the remote path mapping testable in one place instead of embedded in compose generation.
ldm list/status/info's display and the actual Docker command construction are guaranteed to agree, because they'd read the same resolved context instead of two independently-computed values that happened to diverge.
New code gets this by default (read from context) rather than by remembering to call get_active_target() correctly — the failure mode shifts from "silent wrong behavior" to "doesn't compile/obviously missing a parameter."

Where it's called from:

For the run/init pipeline: resolved once in ProjectInitializationStage (where project_meta first becomes available) and threaded through PipelineContext, the same way paths/project_meta already are. Every later stage (ComposerStage, RuntimeValidationStage, ExecutionStage) reads it from context instead of recomputing it.
For ad-hoc handler methods (orchestration.py, diagnostics/info.py, handlers/infra.py, etc.): called once per command entry point, with the resulting context threaded through rather than re-resolved.

6. Migration plan¶

Phased, each phase unit-tested and live-verified against a real remote node before merging — Docker command construction is exactly the kind of change that can be unit-test-green and still wrong:

Introduce the abstraction. TargetContext + resolve_target_context(), with full unit coverage. No behavior change — exists alongside current code, unused.
Migrate the run/init pipeline. Highest-traffic path; replaces composer.py's mount_paths hack directly.
Migrate orchestration.py (cmd_stop/restart/down/reset) — the highest-confidence cluster from the #1133 audit, already inconsistent with its own sibling cmd_start.
Migrate diagnostics/info.py and dashboard/server.py — closes the display-vs-reality gap permanently.
Sweep the remaining medium-impact call sites (fragments.py, search.py, share.py, snapshot/*.py, mcp.py, doctor.py) as a broader pass once the pattern is established and proven.
Revisit the category-(b) items (prune, nuke, rescue, shared DB) as their own product conversation, once the mechanism exists to make them target-aware if that's the decision.

Separable, smaller UX fixes not blocked on the full migration:

Collapsing ldm run's redundant double confirmation for already-running projects down to one CTRL+C-style pause.
Surfacing the effective resolved target upfront, before provisioning begins, when it's not local.

See also: Multi-Node Orchestration & Remote Node Setup for the user-facing ldm target guide, and Architecture Overview for how this fits into LDM's broader layering.

7. Pinning the container MAC (licence activation)¶

Liferay's licence binds to a MAC address. On a remote node the container takes a bridge-assigned address and validation fails -- the portal boots healthy, logs License registered, and then serves the DXP Activation page instead of the Sign In form.

Measured on node-a with liferay/dxp:2026.q3.0, the container recreated identically except for --mac-address:

	Before	After
Container MAC	`02:42:ac:12:00:03`	`06:d0:95:e5:26:a7` (the host's `ens5`)
Validation	failed -- "MAC address matching failed"	passed
Portal serves	DXP Activation page	Sign In form

Same image, mounts, environment and licence. Only the MAC. On success, LiferayTrialActivation_DXPNonProduction_production.li is written into /opt/liferay/data/license.

The licence's product version is not a factor: a 2026.Q1 key activated dxp-2026.q3.0 in the run above.

Do you need this?¶

Only if your licence is machine-bound. The two kinds differ:

developer key      … license-type, license-version, start-date,
                     expiration-date, max-http-sessions, key
                     ^ no mac-address, no host, no IP -- activates anywhere

trial key          … max-servers, mac-addresses, mac-address
                     ^ bound to one NIC

A developer key carries no machine binding at all, which is why local runs activate today without any of this. Only MAC-bound trial keys need a pin. Check your key for a mac-address field before configuring anything.

Configure it per node¶

ldm target add node-a --host <node-ip> --user ec2-user \
  --mac-address 06:d0:95:e5:26:a7

ldm target set is a different command -- it assigns a target to the active project, not node properties -- so re-run ldm target add with the same name to record or change the MAC. Re-running it merges: flags you do not pass keep the values already stored, so changing one setting cannot lose another (see section 8).

LDM does not infer the value. ip a on a typical node shows ens5 alongside docker0 and br-*: all plausible, only one licensed. A wrong guess produces exactly the failure above -- a healthy container that will not let you sign in -- so the configured value is the contract.

target add does read the node's interfaces and warns when the value it was given is none of them, naming each so the right one can be picked. It warns rather than refuses, because Liferay validates the container's MAC against the licence and does not care what the host's interfaces are: a licence bound to a MAC that is not a current NIC is legitimate, if unusual, and refusing would block a working configuration to catch a likely typo. A node that cannot be reached is silent -- unable to ask is not the same as wrong (LDM-#1780).

LDM verifies that the pin actually took¶

Writing mac_address into the compose file is a request, not a guarantee. LDM therefore re-reads the MAC from the running container and refuses when it does not match, with both values named. That one check covers three failures that are otherwise indistinguishable:

a Docker/Compose version that ignores the form LDM wrote
a compose key the running Docker does not honour

Measured caveat: on Docker 25.0.16 the check cannot be made to fire through the CLI at all. ldm run regenerates docker-compose.yml, and Compose recreates the container whenever the MAC changes -- so LDM never reaches the "stale container" state the check was partly written for. Verified on a real node with the container ids captured either side:

before : 7d32972afe3d  02:42:54:0d:a3:56   (configured: docker0)
         # reconfigure the node to ens5, `ldm run` again, no `ldm rm`
after  : 0fcdfec4a18e  06:ff:c5:f9:cf:a5   (configured: ens5)
exit   : 0

A different container, and the MAC followed the configuration. The check stayed silent because it had nothing to report. It remains a correct safety net for a toolchain that accepts mac_address and ignores it -- which is what it was written for, and which this combination does not exhibit (LDM-#1789).

What it cannot catch is a wrong configured value, and this is worth stating plainly because the check reads as though it covers it. It compares the container's MAC against the configured one, so a typo satisfies it perfectly: LDM pins the wrong address faithfully, both values agree, the check passes, and Liferay then refuses the licence. The operator gets the original symptom -- healthy container, License registered, Activation page -- roughly twenty minutes after making the mistake. That half is covered at target add instead, by the interface warning above, where the typo is cheapest to fix (LDM-#1780).

The MAC can only be set when a container is created. docker network connect --mac-address does not exist in Docker 25.0.14, so an existing container cannot be corrected in place -- it has to be recreated. In practice ldm run performs that recreate itself, as the measurement above shows.

An older client drops the pin, and says nothing¶

Compose is generated client-side, so the client's version is the whole compatibility surface. An ldm older than v2.23.0 has no --mac-address at all; run one against a node configured for a pin and it renders a compose file without it:

node configured : mac_address = 06:ff:c5:f9:cf:a5
$ ldm-2.22.0 -y run myproject --node node-a
  exit          : 0
  container MAC : 02:42:ac:12:00:03     <- bridge address

Exit 0, "Project started", and no mention of the MAC anywhere. That is the original failure at the top of this page, reachable purely by using an older binary against a correctly configured node.

The fix cannot live in the old client, so ldm doctor detects it after the fact: it reads the MAC the container actually has, compares it with the node's configured value, and names the recreate that repairs it. Re-running a current ldm run restores the pin (LDM-#1789).

The duplicate-MAC caveat, stated precisely¶

Pinning the container to the host's NIC address means two interfaces share a MAC. This is contained on a bridge network and only there. The container sits on a NAT'd bridge (172.18.0.0/16), which is a different L2 segment from the VPC interface, so the duplicate never reaches the wire.

On macvlan or host networking that containment disappears and the duplicate address is on the physical segment, where it will cause the problems duplicate MACs normally cause. The same option is reachable in those setups, so the safety here is a property of bridge networking -- not of the option.

7a. One SSH connection per run (`LDM_DOCKER_TUNNEL`)¶

Every remote Docker command is a separate docker --context <node> process, and Docker's SSH connection helper opens a fresh SSH connection for each one. With ~86 call sites that is hundreds of connections in a single run.

Measured on an affected node: 478 Accepted publickey in 30 hours, against sshd's default MaxStartups 10:30:100, producing 714 log lines of

sshd: error: beginning MaxStartups throttling
sshd: drop connection #10 from [...] past MaxStartups

A shed connection surfaces as Docker not accessible, naming neither SSH nor the node -- which is why it cost the reporting deployment four runs and a wrong diagnosis before the mechanism was found.

Setting LDM_DOCKER_TUNNEL opens one SSH port-forward to the node's Docker socket for the life of the run, and every command addresses it through docker --host tcp://127.0.0.1:<port>:

LDM_DOCKER_TUNNEL=1 ldm run my-project

Off by default. An environment variable rather than a flag or an ~/.ldmrc key, because the consumers who need it drive LDM from CI, where a variable costs nothing and a config file has to be written into an ephemeral runner.

--host rather than a process-wide DOCKER_HOST deliberately: it is explicit per command and stays correct if a run touches two nodes, which one environment variable would silently get wrong.

What it does not fix¶

A keepalive was the original proposal and it was wrong. The premise was that DOCKER_HOST=ssh:// holds one persistent connection going idle; there is no such connection, and keepalives applied independently changed nothing -- exactly what that premise predicts. The irony is load-bearing rather than decorative: a keepalive is the wrong fix for connection-per-command and the right companion to the tunnel, because a single long-lived forward is the first thing LDM has ever held that can genuinely go idle. Hence ServerAliveInterval on the tunnel's own ssh invocation.

When the tunnel dies mid-run¶

An established tunnel that closes would otherwise surface as a connection refused against a loopback port -- a symptom naming the wrong subject. The next Docker command reports it as itself instead, distinguishes "established then died" from "never established", and falls back to per-command connections rather than failing the run: a slower run beats no run.

The report says detected at, not died at. Nothing watches the ssh process; its death is noticed the next time a command needs the tunnel, which may be well after the fact.

Verifying it¶

Absence of Docker not accessible proves nothing on a node whose MaxStartups has been raised -- that node cannot shed connections regardless. The measurement that means something is the accept count:

ssh <node> 'journalctl -u sshd --since -5m | grep -c "Accepted publickey"'

Hundreds per run means connection-per-command. Roughly one means the tunnel is carrying the run.

8. A target's SSH user lives in two places¶

A remote target is described twice, and only one of the two is LDM's:

Fact	`~/.ldmrc`	Docker context
host	`host`	`ssh://<user>@<host>`
SSH user	`user`	`ssh://<user>@<host>`

ldm target add writes both, so they agree at the moment it runs. Nothing afterwards keeps them in step. A docker context update, a hand-edited ~/.ldmrc, or a context rebuilt by anything other than LDM separates them silently -- and the result is the least legible failure in the product, measured on node-a while preparing v2.23.0-pre.5:

$ docker context inspect node-a --format '{{.Endpoints.docker.Host}}'
ssh://ldm-automation@203.0.113.10

$ ldm target ls    # ~/.ldmrc
node-a  host=203.0.113.10  user=ec2-user

$ docker --context node-a version
ldm-automation@203.0.113.10: Permission denied (publickey,...).

ldm target ls showed a correct configuration, the node was genuinely reachable, and every command that went through the context failed naming a user nobody had configured.

ldm doctor now reports it, the same way it reports a dropped MAC pin -- configured state compared against reality rather than trusted:

[node-a] SSH user     DRIFTED (ldm-automation)

The host half of the same divergence is reported by ldm target status (LDM-#1346). Both are silent when the two agree, when the node is local, when no user is configured, and when the context cannot be read at all -- a node may legitimately be registered before its context exists.

`target add` merges; it used to replace¶

The repair is to re-register the node with the right user. That used to be the trap rather than the fix. save_target_node does raw_targets[target.name] = target.to_dict() -- a replace -- and target add built a fresh node from only the flags it was given, so:

ldm target add node-a --host 203.0.113.10 --user ec2-user   # no --mac-address

reported "registered successfully" and destroyed the node's MAC pin. The next ldm run then booted a container with a bridge address, Liferay refused the licence, and the operator got the section 7 symptom -- healthy container, License registered in the log, Activation page instead of Sign In. Section 7 is where LDM learned to detect a dropped pin; this dropped one using the documented command, during routine maintenance, with no warning (LDM-#1797).

target add on an existing node now keeps every setting it was not passed, which is how the command already reads -- "add or update this node". Clearing is still expressible, because an absent flag and a flag passed empty are different inputs:

ldm target add node-a --user ec2-user        # keeps host, key and MAC pin
ldm target add node-a --mac-address ""       # removes the pin, deliberately

--default is the one exception. A store_true flag has no empty spelling, so target add can only set it; moving the default to another node is what ldm target use <name> is for.

An update is never silent. It names what changed and what it kept, on UI.info rather than UI.detail, because detail is gated behind --info/--verbose (LDM-#1036) and would be invisible in exactly the invocation that matters:

✅  Target node 'node-a' (203.0.113.10) updated.
ℹ  Changed SSH user: ldm-automation -> ec2-user
ℹ  Docker context 'node-a' rebuilt to dial ssh://ec2-user@203.0.113.10.
ℹ  Kept host 203.0.113.10, MAC pin 06:ff:c5:f9:cf:a5 (not passed).
ℹ  Pass a flag with an empty value to clear one, e.g. --mac-address "".

An update that moves nothing in ~/.ldmrc reports stored settings unchanged rather than "nothing changed" -- because the Docker context is rebuilt on every remote add, and in the drift repair above ~/.ldmrc was right all along:

✅  Target node 'node-a' (203.0.113.10) already matched; stored settings unchanged.
ℹ  Docker context 'node-a' rebuilt to dial ssh://ec2-user@203.0.113.10.

Last Updated: 2026-10-23 | Last Reviewed: 2026-10-23