IPC stability bugs

Bug 101

Failed IPC socket setup is ignored: client crashes, listener silently missing

Status
Open
Symptom
Crash; worst effect seen: crash
Where seen
Bitcoin Core unit tests
First reported
Reported privately in an independent security review, 2026-09-27
Fixed
Not yet
In released binaries
Yes (v32.0)
Defect ever on master
Yes

Summary

When libmultiprocess sets up a new IPC connection or listener, it hands the socket to Cap'n Proto inside a function that runs on the event loop thread. If that setup throws, the event loop logs the error and the calling code carries on as if it had worked. A client then crashes on a missing connection, and a listener reports success without listening. It was reported privately in an independent security review, using an injected allocation failure. The node and GUI already treat allocation failure as fatal, but the same setup can also fail when the operating system refuses to register the socket for event notification, which is rare. It has not been seen outside tests. No fix is merged yet.

What happened

Nothing has been seen in a deployment. An independent security review reported on 2026-09-27 that errors while wrapping IPC sockets are lost. Its unit tests make a testing hook throw an allocation failure just before the socket is wrapped. The report says that without a fix the client test process crashed with SIGSEGV and the listener test returned normally with no listener. Those runs have not been repeated independently.

Root cause

ConnectStream() and ListenConnections() in include/mp/proxy-io.h run their setup through EventLoop::sync(), which runs a function on the event loop thread and waits for it. The function wraps the socket descriptor in a kj stream or listener object and then creates the Connection or Listener. Since mp#260 (2026), the event loop catches an exception from such a function, logs "EventLoop: m_post_fn threw" and lets sync() return normally; its own comment warns that the caller gets no indication of failure. Neither caller checks.

The report's trigger is a failed memory allocation for the wrapper object. Bitcoin Core's node and GUI terminate on allocation failure anyway. But kj also registers the socket with the operating system's event notification while creating the wrapper (epoll_ctl on Linux, kevent on macOS), and that system call can fail, for example when the kernel is short of memory or the user has reached the per-user epoll watch limit. That failure takes the same path. On libmultiprocess master the client-side wrapping has moved into MakeStream(), which returns an empty stream in the same situation, and ConnectStream() then uses it.

The report also found that if the allocation itself fails, the socket descriptor leaks, because kj allocates the wrapper before the part that owns the descriptor exists. That part needs an allocation failure, so it is treated as hardening and is not part of this bug.

The same dropped exception causes a client to wait forever when building a request throws.

Fix

No fix merged yet. The fix proposed with the report adds a SyncWithException() helper that catches the exception inside the function run on the event loop and rethrows it on the calling thread, and uses it in ConnectStream() and ListenConnections(). It also keeps a local owner for the descriptor until kj has taken it, which fixes the leak. An alternative is to make EventLoop::sync() itself rethrow, which would cover every caller at once. An open change, mp#376, already moves the exception onto the calling thread but still only logs it there.

Next steps

Timeline

Lessons

What would have prevented this bug, found it sooner, or made it easier to debug. The patterns page groups these lessons across all bugs.

Pattern Error paths that skip cleanup or completion: Swallowed setup error

  • Prevented by a different API shape
  • Earliest detector a unit test (did not exist then)
  • Diagnosis easy
  • Fix design easy
  • Debugging aids logging
  • Rust partly

Would Rust have prevented it? Partly. If the closure run on the event loop returned a Result, the compiler would force callers of sync() to handle it, so ConnectStream could not silently continue; a panic inside the closure would still need explicit handling, and nothing in Rust stops a helper from discarding an error on purpose. More on Rust

Could it have been prevented?

Yes, with an API that cannot lose an error: a function that runs a closure on another thread and waits for it should return the closure's result or rethrow its exception. EventLoop::sync() returns nothing, and since mp#260 its event loop logs and drops exceptions to stay alive. mp#260's own comment predicted that callers "will return without any indication of failure, which will likely cause other bugs". ConnectStream() and ListenConnections() are two such callers: they assume the setup finished because sync() returned.

Could it have been found sooner?

By a test that makes socket setup fail and checks the caller sees an error. No test injected failures into kj socket wrapping; the only failures exercised were disconnects after setup. The independent security review that found it injected an allocation failure with a testing hook. A system-call failure (epoll_ctl or kevent refusing to register the socket) reaches the same path but is hard to produce on purpose, so a wrapper around the I/O provider is the practical way to test it.

What made it hard to debug?

The failure shows up far from its cause. The only record is one "EventLoop: m_post_fn threw" log line; the crash comes later, in a constructor that dereferences a null connection, and a missing listener shows up only as clients being refused. Propagating the original exception would have reported the real error at the call site.

Lesson

When a helper runs work on another thread and waits for it, it must hand back the work's failure, not just its completion: logging an error in the worker and returning normally turns one clear failure into a later crash or a silent outage elsewhere.

Classification

The survey's classification of this bug. The explorer charts these fields across all bugs; the method page explains them.

Field Value
Symptom crash
Worst effect observed crash
Seen in unit-test
Scope production
Trigger other (other)
Phase startup
Side both
Component proxy-io
Root cause kind unhandled-exception
Fix kind none
Detection review
Needs a peer that is not libmultiprocess C++ no
Needed a sanitizer to see no
Defect ever on master yes
Released binary
Confidence low