Docker Exit Code 137: Confirm OOM Before Adding RAM

Imagine a batch worker stops halfway through an import. Docker reports exit code 137, and the tempting next step is to buy more RAM. Before you resize anything, capture the container state and the events around its last exit. 137 is consistent with SIGKILL; it does not, by itself, prove an out-of-memory kill. A stop timeout can end with the same signal.

On Linux, SIGKILL is signal 9, and Bash documents the familiar 128 + signal exit-status convention: 128 + 9 = 137. That explains the number, not who sent the signal. A process can also explicitly return a numeric status, so treat the code as a clue rather than a complete diagnosis. See the Linux signal manual and Bash exit-status documentation.

Capture the exit before changing the container

Start with the affected container's identity, state and configured memory limit. Docker container inspect supports formatted output; Moby's inspection implementation exposes the state fields used here.

docker container inspect --format \
'container={{.Id}} status={{.State.Status}} exit={{.State.ExitCode}} oom={{.State.OOMKilled}} finished={{.State.FinishedAt}} memory_limit_bytes={{.HostConfig.Memory}}' \
worker

Save the output with the deployment version and the workload that was running. OOMKilled=true is useful evidence for an OOM investigation. false doesn't identify the alternative cause. Inspection describes the container state available now; if it has restarted or been replaced, do not assume you're looking at an immutable record of the earlier failure. Keep the container ID so a reused service name doesn't confuse your timeline.

Next, ask Docker for the incident window. The following times are an example in Asia/Kolkata, not a reported incident. Use a window that includes the failure and the preceding stop or deployment.

docker events \
  --since '2026-11-05T05:00:00+05:30' \
  --until '2026-11-05T06:00:00+05:30' \
  --filter container=worker

Look for oom, kill, die, stop and start, then line up their timestamps with the release history. Docker's events reference documents these events and time filters. Its history is limited to the last 256 events, so an empty result is not proof that no OOM occurred. For recurring failures, retain an event stream outside that short history.

A deployment can produce the same ending

Suppose the hypothetical worker only reports 137 when a release replaces it. Before changing memory, inspect how it shuts down. Docker stop sends the configured stop signal—SIGTERM by default—and sends SIGKILL if the process hasn't exited before the grace period expires. Docker kill also sends SIGKILL by default.

That gives you a different investigation: did the application receive the stop signal, stop accepting work and finish or safely hand off its current job? Check the entrypoint and application signal handling. Compare the configured grace period with measured job shutdown time. Extending a timeout can be reasonable for a bounded cleanup task, but it won't repair a worker that never responds to shutdown.

A kill event alone still needs context. Correlate the signal, exit time and operator or deployment activity. A nearby deployment is a hypothesis to check, not permission to declare memory innocent.

When OOM evidence is present, find the boundary

There are two memory boundaries to investigate. One is the container's control-group limit; the other is the available memory of the Linux instance shared by Docker and everything else running there. Docker documents hard and soft memory constraints. Without resource constraints, a container is not automatically given a safe share of the host.

A hard container ceiling can be reached while the instance still has spare memory. Conversely, several containers and other processes can exhaust the instance together. The kernel's cgroup v2 documentation distinguishes the memory.max boundary from OOM event counters. Its oom_kill counter counts processes killed by an OOM killer; it isn't a timestamped explanation of which job caused the pressure. Paths, hierarchy and available controls vary, so don't paste a guessed cgroup path into a runbook. Check swap configuration too; memory-plus-swap settings affect the available allowance, and swap is not a substitute for measuring the workload.

On a systemd-based host, inspect the kernel journal around the failure as another source of evidence. This relative window is only a starting point; extend it if the incident is older.

TZ=Asia/Kolkata journalctl -k --since '-30min' --no-pager -o short-iso

The journalctl manual documents kernel filtering and time windows. The command displays timestamps in Asia/Kolkata; -k selects the current boot by default, so investigate the relevant retained boot separately after a reboot. Access depends on your existing privileges, and retention may be incomplete. Match any OOM record to the process, cgroup and time; an unrelated OOM on the same machine does not diagnose this container.

For a worker that is running again, a quick current reading is useful:

docker stats --no-stream worker

Docker stats reports running-container usage. A quiet reading after recovery cannot recover the missing peak from the failed import. The Linux CLI also subtracts cache from its displayed memory usage, so label what you measured when comparing it with other monitoring data.

Turn the evidence into one change

• OOM evidence tied to this container, with pressure near its hard ceiling: investigate the job's memory demand and limit. Reduce batch size or concurrency where appropriate; profile growth before choosing a larger allowance.
• Kernel evidence of shared instance pressure: budget all simultaneous workloads, including services outside Docker. Increasing one container's ceiling can make this case worse.
• Exit aligned with a stop timeout, supported by the shutdown timeline: fix signal handling and bounded shutdown, then verify the grace period.
• No retained evidence: keep the cause unresolved. Capture events and memory history for the next representative run instead of labeling every 137 as OOM.

For the hypothetical import, consider this deliberately simple budget: a 4 GiB instance reserves 1 GiB for the operating system, Docker and other services, leaving 3 GiB for workers. Two workers allowed 2 GiB each could together ask for 4 GiB. Each limit might look plausible alone, yet the concurrent allowances exceed the budget. These numbers are illustrative arithmetic, not a Cube sizing recommendation or a measured memory requirement.

The practical choice might be one worker at a time, smaller batches, a corrected application allocation pattern, or more instance memory after measurement. A container ceiling does not reserve that much physical memory, and a restart policy does not create additional capacity. Docker explicitly advises against circumventing OOM safeguards; don't disable the OOM killer to make the symptom disappear.

Verify the workload that actually failed

After a planned change, inspect the effective container configuration again. Replay representative work in a separate test environment or an appropriate maintenance window, record concurrency and input size, and watch both container usage and instance headroom. Then check that the job completed correctly and that shutdown still works. An absence of new 137 exits during an idle period isn't much evidence.

Keep a short incident note: container ID, exit time, supporting events, kernel evidence if available, input size, simultaneous jobs, the one change made and the next run's result. That note makes the next sizing decision easier to defend. If the separate symptom is growing log storage, use the Docker log rotation guide; disk retention needs its own diagnosis.

Creation context: this AI-assisted guide was assembled from the linked primary documentation. The scenario and memory budget are illustrative, the commands were not run for this article, and the cover is a generated conceptual illustration.