Skip to content
devsess
Esc
↑↓navigate↵open⌘Jpreview

Recover a failed run

Recover a failed run

Read this when a service exits unexpectedly, a command returns nonzero, or a run cannot be stopped.

tail and attach use the service’s completion status: normal exit returns its exit code; signal termination returns 128 + signal when the exit event includes a signal. Tailing an already exited service within an active run replays its retained output and then reports its saved exit status. A persistence failure terminates that service and completes its tail with exit code 1. If a daemon restart finds a process gone before an exit status was saved, exit code 1 with no signal means the status is unknown. In follow mode with several services, tail waits for all and returns the first nonzero status in completion order, or 0 if all succeed. Without follow, it exits after replay and reports any service already completed with failure. Zero exit and explicit detach return 0; daemon, connection, selection, and live-output-overflow errors exit 1. Cancelling a picker exits 130. devsess daemon status uses 3 when the daemon is unreachable.

Services run in Unix process groups. The daemon saves a PID, process-group ID (PGID), and birth fingerprint and verifies ownership before signaling a saved process. If a leader is gone, a surviving numeric PGID might have been reused and is not signaled automatically. A recovered run can be stopped normally only if its saved leader still matches; otherwise the service is marked orphaned. A normal stop that cannot verify a group names the service, run, and PGID and suggests both devsess stop --force and kill -TERM -<pgid>. --force explicitly allows termination of a saved group with an unverified leader, but refuses invalid PGIDs and can fail if the group does not terminate. Successful termination reconciles the record so the preset can start again. Termination waits up to five seconds after SIGTERM and then up to five seconds after SIGKILL.

If the daemon crashes after spawning a PTY but before saving its identity, that process may leak. On macOS, ps lstart birth fingerprints have one-second precision. After a daemon restart, the environment originally captured for a run is unavailable for restart; stop and start the run again before restarting individual services. See devsess docs read running for readiness failures and devsess docs read logs for retained output.

Was this page helpful?