IPC stability bugs

Bug 88: IPC listener stops for good after one temporary accept failure

2026-10-15T00:00:00Z

If accepting an incoming IPC connection fails once with a temporary resource error, for example because the process has run out of file descriptors (EMFILE), the libmultiprocess listener gives up for good: it closes its listening socket, and every later client gets "connection refused" until the process restarts. The error is temporary, so the listener should wait and try again. It was reported privately in an independent security review and reproduced with tests; it has not been seen in a running node. Bitcoin Core now counts IPC connections in its file descriptor budget (bitcoin#35037), so running out needs a descriptor leak or other open files, but the listener should survive it. No fix is merged yet.

Bug 103: An IPC client's notification handler returning an error aborts the node

2026-10-06T00:00:00Z

A program that subscribes to chain notifications over the Chain IPC interface could crash bitcoin-node by returning an error from a notification handler such as updatedBlockTip. The error comes back to the node as an ipc::Exception on the thread that delivers validation notifications, nothing catches it there, and the process terminates. A disconnect while a notification is being delivered has the same effect. Only the open PR bitcoin#29409, which exposes Chain over IPC, makes this reachable, so master and releases are not affected. The fix is in bitcoin#36520, not yet merged.

Bug 102: Process aborts when two thread proxies post to one thread at once

2026-09-28T00:00:00Z

Each thread that runs IPC calls has a Waiter that holds one pending call. Since mp#240, each thread proxy (ProxyServer) waits for its previous call to start before posting the next, and then asserts that the post succeeded. But one thread can have several thread proxies, one per connection it makes calls over, so two of them can post before the thread picks up the first, and the assertion aborts the process. It was reported privately in an independent security review. The libmultiprocess C++ client cannot trigger it, and no Bitcoin Core call sequence that does is known; a client that overlaps calls across proxies could. No fix is merged yet.

Bug 101: Failed IPC socket setup is ignored: client crashes, listener silently missing

2026-09-27T00:00:00Z

When libmultiprocess sets up a new IPC connection or listener, it hands the socket to Cap'n Proto inside a function that runs on the event loop thread. If that setup throws, the event loop logs the error and the calling code carries on as if it had worked. A client then crashes on a missing connection, and a listener reports success without listening. It was reported privately in an independent security review, using an injected allocation failure. The node and GUI already treat allocation failure as fatal, but the same setup can also fail when the operating system refuses to register the socket for event notification, which is rare. It has not been seen outside tests. No fix is merged yet.

Bug 100: IPC process aborts if its cleanup thread cannot be started

2026-09-27T00:00:00Z

When a client releases a server object without calling destroy, for example by disconnecting with objects still alive, libmultiprocess hands the object's cleanup to a background thread so the event loop is not blocked. That thread is created the first time it is needed, from inside the ProxyServer destructor. If creating it fails, for example because the process has reached its thread limit, the destructor cannot report the error and C++ terminates the whole process. It was reported privately in an independent security review, with a test that injects a failure at that step; it has not been seen in a running process, and the trigger needs a thread limit to be hit at one particular moment. No fix is merged yet.

Bug 99: One mining interrupt can leave a second wait blocked

2026-09-27T00:00:00Z

A mining client can cancel its long waits with Mining.interrupt() (for createNewBlock and waitTipChanged) or BlockTemplate.interruptWait() (for waitNext). Each object keeps a single flag for this, and the first wait to wake clears it. If a client has two waits running on the same object from two threads, one interrupt releases only one of them; the other stays blocked until the next block, its timeout, or another interrupt. The node is not affected, but the client can stall. It was reported privately in an independent security review, with a functional test. No fix is merged yet.

Bug 98: Process aborts instead of reporting an error when the IPC event loop fails to start

2026-09-27T00:00:00Z

Bitcoin Core starts its IPC event loop on a background thread the first time a process listens for or opens an IPC connection. If creating the loop fails, for example because the process has run out of file descriptors, the error escapes that thread and C++ terminates the whole process with an abort. The caller expects an exception it can report: bitcoin-node would otherwise exit with "Unable to bind to IPC address". It was reported privately in an independent security review, with a test that injects the failure; it has not been seen in a running process, and in most cases the process would have stopped anyway, just with a clear message instead of an abort. No fix is merged yet.

Bug 97: Failed IPC startup leaks the socket descriptor and the spawned child

2026-09-27T00:00:00Z

When Bitcoin Core's IPC setup fails partway, it can leave resources behind. If starting the IPC event loop fails after a socket has been opened, the socket's file descriptor is never closed, and if it fails after a child process has been spawned, the child keeps running and is never waited for. It was reported privately in an independent security review, with tests that inject the failures. On master the effect is small: the callers that can hit it either exit after the error or are a test-only RPC, so this is recorded as a low-confidence bug. No fix is merged yet.

Bug 96: Failed startup does not exit while a mining client is waiting

2026-09-27T00:00:00Z

If bitcoind or bitcoin-node fails to start while an IPC mining client is waiting inside the node, for example in makeMining for the chain to load, the node logs "Shutdown done" and then stays running until it is killed. With -daemon, the stuck process also keeps the data directory locked, so a corrected restart fails. The cause is that the startup-failure path never raises the shutdown signal that every mining wait checks. It was reported privately in an independent security review, with a functional test. No fix is merged yet.

Bug 95: Concurrent first IPC use can abort the process

2026-09-27T00:00:00Z

Bitcoin Core starts its IPC event loop on a background thread the first time a process uses IPC. The check for "is the loop already running?" has no lock, so if two threads make their first IPC call at the same time, both can start a loop thread, and replacing the first, still-running thread makes C++ terminate the process. It was reported privately in an independent security review with a test that forces the timing. On master the only way to make two first calls at once is the hidden, test-only echoipc RPC on a bitcoin-node started without -ipcbind. No fix is merged yet.

Bug 94: checkBlock before genesis is connected aborts the node

2026-09-26T00:00:00Z

If a mining client calls Mining.checkBlock while bitcoin-node is starting on an empty data directory or reindexing, after the mining interface is available but before the genesis block is connected, the node aborts on an assertion that the chain has a tip. The other mining calls wait for a tip or report that there is none; checkBlock goes straight to a check written for callers that run after startup. It was reported privately in an independent security review, with a functional test; it has not been seen in a running deployment. No fix is merged yet.

Bug 93: IPC processes abort when the monotonic clock steps backwards, with Cap'n Proto older than 1.2

2026-09-25T00:00:00Z

A program using libmultiprocess, such as bitcoin-node with IPC or the ipctests unit tests, aborts with kj/timer.c++:120: failed: expected newTime >= time ... can't advance backwards in time if the Linux CLOCKMONOTONIC clock steps back, even by microseconds, while the event loop is waiting. The check that throws is in Cap'n Proto's kj library before version 1.2.0. It was seen once in Bitcoin Core's ASan CI job, which uses Ubuntu's Cap'n Proto 1.0.1 package. Release binaries are built with Cap'n Proto 1.2.0 or later from depends and are not affected; self-built binaries linked against a distribution's Cap'n Proto (every Ubuntu release through 26.04 ships 1.1 or older) are. A workaround in libmultiprocess, mp#369, is in review.

Bug 84: Negative or NaN wait timeout from a mining client aborts debug builds of the node

2026-09-18T00:00:00Z

An IPC client that calls Mining.waitTipChanged (wait until the chain tip changes, or until a timeout) with a negative or NaN timeout makes bitcoin-node abort on Assume(timeout >= 0ms) in builds with ABORTONFAILEDASSUME (the Debug build type and all fuzz builds). In release builds the Assume does nothing and a negative timeout is clamped to zero, but NaN passes both clamps and reaches std::conditionvariable::waituntil, which is undefined behavior. It was reported privately in an independent security review, with a functional test that aborts a checked build, and a separate test gives the same abort over IPC; the release-build NaN effect comes from reading the code. Released binaries from v30.0 on accept the value; their NaN case is the only one that can misbehave there. No fix yet.

Bug 92: Chain.getBlockHash with a height above the tip aborts the node

2026-09-09T00:00:00Z

Calling getBlockHash on the Chain IPC interface with a height that is not in the active chain, for example one above the current tip, makes bitcoin-node abort on an Assert. The method was written for the wallet running in the same process, which only asks for heights it knows exist; over IPC the height comes from another program. Only the open PR bitcoin#29409, which exposes Chain over IPC, makes this reachable, so master and releases are not affected. The fix is in bitcoin#36520, not yet merged.

Bug 83: Reading a block template while submitting a solution for it is a data race

2026-09-06T00:00:00Z

A mining client gets a BlockTemplate from the node, reads it with getBlock, getBlockHeader and getCoinbaseMerklePath, and hands in a solved block with BlockTemplate.submitSolution. submitSolution writes the coinbase, header fields, merkle root and cached check flags into the template's stored block in place, with no lock, while the getters read the same block. An IPC client that uses one template from two threads (each client thread gets its own worker thread on the server) can make these run at the same time: a data race in bitcoin-node. Most fields would only be read torn, but the coinbase std::sharedptr (vtx[0]) is copied on one thread while it is reassigned on another, which can corrupt its reference count. It was reported privately in an independent security review with an in-process test that ThreadSanitizer flags, and a separate test gives the same ThreadSanitizer reports; it was later reported publicly in bitcoin#36515, found with the fuzzamoto fuzzer under ThreadSanitizer over real IPC. It has not been seen without the sanitizer. Released binaries from v30.0 on are affected. A fix is in review in bitcoin#36525.

Bug 78: Listener at its connection limit stops accepting after a local disconnect

2026-08-20T00:00:00Z

A ListenConnections listener that had reached its maxconnections limit stopped accepting new connections permanently if the server then closed one of its connections locally, because the active-connection counter was only decremented on a remote disconnect. New clients would wait forever. It was found in code review and shown with a regression test; it has never been observed in practice. Bitcoin Core cannot trigger it today, since it only closes IPC connections locally at shutdown, and no release contains the connection limit. A fix is in mp#361, which fixes several disconnect bugs found in review, not yet merged.

Bug 77: Disconnect handler can still run after its connection was destroyed locally

2026-08-20T00:00:00Z

A Connection's onDisconnect handler could run after the Connection had been destroyed locally, if the peer disconnected and the local side destroyed the connection in adjacent event-loop turns. With the handlers in use this deletes the Connection a second time or uses a dangling pointer. It was found in code review and shown with a constructed mptest case; it has never been observed in practice. Bitcoin Core releases v22 through v31.1 contain the code, and it could possibly be reached when a client disconnects in the same loop turn that Ipc::disconnectIncoming() destroys its connection at shutdown. A fix is in mp#361, which fixes several disconnect bugs found in review, not yet merged.

Bug 79: Node can crash when a client dies while a call's reply is being built

2026-08-19T00:00:00Z

When an IPC client died in the middle of a call, the server's worker thread could read Cap'n Proto connection state in its first getResults() call while the event-loop thread was tearing that state down. TSan reports it as a data race, and one interleaving dereferences a null pointer, so a client that disconnects mid-call could take down bitcoin-node. It was found by Antithesis running a TSan build; no crash was observed. Released binaries v30.0 through v31.1 contain it. Fixed in mp#349; not yet in Bitcoin Core's subtree.

Bug 75: Node spawned by the GUI never exits after shutdown because disconnected callbacks stay alive

2026-08-14T00:00:00Z

When bitcoin-gui spawned bitcoin-node and was told to stop, bitcoin-node finished its shutdown and then never exited, and bitcoin-gui waited for it forever. The GUI registers callbacks with the node over IPC, and the node wraps each one in an object that keeps the node's IPC event loop running while it exists. Those wrappers stayed alive after the GUI disconnected them, because btcsignals (Bitcoin Core's replacement for boost::signals2) did not destroy a disconnected callback. The btcsignals behavior is on master, but only the multiprocess GUI in the unmerged bitcoin#10102 registers signal callbacks over IPC, so no released binary is affected. The proposed fix is bitcoin#36011, not yet merged.

Bug 82: Destroying an object during a running call frees it under the call

2026-08-13T00:00:00Z

If an IPC client calls destroy on an object from one thread while a call on the same object is still running on another server thread, the server frees the object's implementation under the running call. For BlockTemplate, destroying a template while its waitNext is outstanding makes bitcoin-node read the freed template for the rest of the wait, a use-after-free that can crash the node. The libmultiprocess C++ client cannot do this, so only a multi-threaded client written with another Cap'n Proto library (pycapnp, capnp-rust) can trigger it. It was reported privately in an independent security review. A test that forces the interleaving reproduces the use-after-free; it has not been seen in practice. The code has been this way since the library's first commit; released binaries from v30.0 on, which accept external IPC clients with -ipcbind, are affected. No fix yet.

Bug 81: Node crashes when a client calls a method on an object it already destroyed

2026-08-13T00:00:00Z

An IPC client that calls destroy on an object and then calls any other method on the same object makes the server call a virtual function through a null pointer, and bitcoin-node crashes with SIGSEGV. For BlockTemplate, one destroy followed by getBlock, waitNext or interruptWait is enough. The libmultiprocess C++ client cannot do this, so only a client written with another Cap'n Proto library (pycapnp, capnp-rust), or a buggy one, can trigger it. It was reported privately in an independent security review. A test that sends the calls directly reproduces the crash every time; it has not been seen in practice. The code has been this way since the library's first commit; released binaries from v30.0 on, which accept external IPC clients with -ipcbind, are affected. No fix yet.

Bug 74: Connect test fails on slow machines by expecting one of two valid errors

2026-08-12T00:00:00Z

The mptest case "ConnectStream defers disconnect failure to the first IPC request for interfaces without construct()" failed intermittently on slow machines because it expected one of two error messages that the library can correctly produce after a disconnect, depending on timing. The defect was in test code and did not affect users. It was introduced with the test in mp#298, which reworked error handling when creating clients, and fixed two days after that merged, in mp#343, by accepting both messages.

Bug 91: Signal during spawned-process cleanup can terminate the parent

2026-08-07T00:00:00Z

When an IPC parent waits for a spawned child process to exit, libmultiprocess's WaitProcess calls waitpid once and treats every failure as final, including EINTR, which only means a signal handler interrupted the call. In Bitcoin Core that wait runs as a cleanup step inside a destructor that cannot throw, so the error terminates the parent process; other callers that catch the error are left with a child that is never reaped. Bitcoin Core's own SIGTERM, SIGINT and SIGHUP handlers can cause the interruption, but the signal has to be handled on the waiting thread, which is rare, and on Bitcoin Core master only a test-only RPC spawns processes. Not seen in a running node. No fix merged yet.

Bug 87: Spawned processes can inherit one stray file descriptor

2026-08-07T00:00:00Z

Before exec'ing a spawned process, SpawnProcess closes inherited file descriptors in a loop that stops one short of the highest descriptor, and it never visits descriptors above a lowered RLIMITNOFILE. A descriptor without FDCLOEXEC in one of those slots leaks into the spawned process. It was reported privately in an independent security review, and a test reproduces both cases. It has not been seen in practice: the top slot is normally free, Bitcoin Core never lowers the limit, and on master SpawnProcess only runs for the test-only echoipc RPC. Released binaries since v30.0 contain the code. No fix yet.

Bug 86: Client thread hangs forever if building a request throws

2026-08-07T00:00:00Z

If anything throws while an IPC client is building a request, the calling thread waits forever in clientInvoke, the library function that sends a call and waits for its reply. The exception escapes into EventLoop::loop(), which logs and discards it, so sync() returns normally, no request is sent, and nothing ever marks the call done. It was reported privately in an independent security review. A test with a throwing argument builder reproduces the hang every time, but it has not been seen in practice: no argument builder in the library or in Bitcoin Core throws in normal use. The unguarded code is in all releases since v30.0 (v30 and v31 in an older form that hangs in post() instead). No fix yet.

Bug 76: Windows IPC processes corrupt the heap at thread exit with older mingw-w64

2026-07-22T00:00:00Z

On Windows builds made with mingw-w64 older than 13.0.0, bitcoin-node and bitcoin-cli could abort with STATUSHEAPCORRUPTION (0xC0000374) when a thread that had made IPC calls exited, because the mingw-w64 runtime could free threadlocal storage before running libmultiprocess's ~ThreadContext destructor on it. The defect is in the mingw-w64 runtime, reached through libmultiprocess's threadlocal ThreadContext. It was seen only in the Windows cross-compiled CI jobs of the Windows IPC port (bitcoin#32387); no released binary was affected, since Bitcoin Core does not build IPC for Windows and its release toolchain already used mingw-w64 13. It went away when bitcoin#35877 moved the CI jobs to a mingw-w64 13 toolchain; the libmultiprocess workaround, mp#318, was closed unmerged.

Bug 70: Example program crashes at startup when its event loop exits too early

2026-07-22T00:00:00Z

The mpexample sample program crashed intermittently at startup because its main() held no reference to the EventLoop, so the loop could exit between MakeStream and ConnectStream when no Connection yet kept it alive. The defect was in example code and did not affect users or Bitcoin Core, which already holds its own loop reference; it matters because the example is the code new users copy. It was introduced when mp#274 split connection setup into two calls, and fixed in mp#315.

Bug 69: Spawned process's socket could leak into another child started at the same moment

2026-07-20T00:00:00Z

SpawnProcess left the spawned child's socket inheritable across exec for a window in the parent, so a fork+exec on another thread at the same moment (for example a -blocknotify shell command) could leak a duplicate of the socket into an unrelated process. A leaked duplicate keeps the parent from seeing EOF when the spawned process exits, so the connection is never cleaned up. It was found by review and shown with a demonstration test; it was never seen in CI or by a user. Released binaries (v30.0 through v31.1) have an older, wider form of the same window, reachable only through the test-only echoipc RPC. Fixed in mp#311 except for a residual window noted below. Bitcoin Core master is still fully affected: its subtree predates both mp#274 and mp#311, so its SpawnProcess sets no FDCLOEXEC at all, and any fork+exec on another thread can inherit the socket. A test reproduces that. Bitcoin Core gets the fix only with a subtree update.

Bug 80: macOS IPC listener stops accepting connections after a queued client disconnects

2026-07-17T00:00:00Z

On macOS, a bitcoin-node -ipcbind socket stopped accepting IPC connections for good if a client connected and then closed its socket before the node accepted it. The node kept running, but every later connection attempt was refused. The defect is in Cap'n Proto's kj library, not in libmultiprocess or Bitcoin Core: macOS reports EINVAL when kj sets the TCPNODELAY option on such a dead socket, kj treated that error as fatal, and the exception ended libmultiprocess's accept loop. Released macOS binaries v30.0 through v31.1 are affected. It was fixed by bitcoin#35796, which patches Cap'n Proto in depends.

Bug 73: Event loop never exits if a client's setup call fails while it is created

2026-07-13T00:00:00Z

Some interfaces, such as Bitcoin Core's Init interface, declare a construct() method that the client calls while its ProxyClient is being created. If that call threw, the client constructor exited by exception, its destructor never ran, and the Connection and the EventLoopRef it holds were leaked, so EventLoop::loop() never returned and any thread joining the event loop hung. It was found when a new mptest case hung the whole suite. Released Bitcoin Core binaries (v30.0 through v31.1) contain the code, but there the leak needs construct() to throw on a live connection, which Bitcoin Core's Init.construct does not do short of running out of memory; no user-facing hang is known. Fixed in mp#298, which reworked error handling when creating clients; not yet in Bitcoin Core's subtree.

Bug 72: Client can use freed memory if the server closes the socket right away

2026-07-10T00:00:00Z

A client connecting over IPC could abort or read freed memory if the peer closed the socket immediately: ConnectStream registered the handler that deletes the Connection on disconnect before the ProxyClient that owns the connection existed, so an early EOF could free the Connection while the client constructor was still reading it. It was found by a new test that failed on every NetBSD CI run. Released Bitcoin Core binaries (v30.0 through v31.1) contain the defect, and bitcoin-cli -ipcconnect connecting to a peer that dies at once can reach it. Fixed in mp#298, which reworked error handling when creating clients; not yet in Bitcoin Core's subtree.

Bug 66: Sending an empty byte string over IPC is undefined behavior

2026-07-01T00:00:00Z

Serializing an empty byte vector into a Cap'n Proto Data field (the schema type for byte strings such as scripts) called memcpy with a null source pointer, which is undefined behaviour even though nothing is copied, and UBSan reports it. It was found by the new IPC fuzz target in Bitcoin Core's fuzzer,address,undefined,integer CI job. There is no known observable effect on any real platform. Released binaries v30.0 through v31.1 contain it (and v29 source builds with IPC enabled). Fixed in mp#305 and shipped to Bitcoin Core master by the subtree update bitcoin#35720.

Bug 64: Data race on a proxy's connection pointer when a peer disconnects

2026-06-03T00:00:00Z

clientDestroy(), which runs when a ProxyClient is destroyed, read the proxy's mcontext.connection pointer without a lock, while the disconnect cleanup on the event-loop thread could set it to null. The only thing the read decided was which log line to write, so the practical effect was a TSan data-race report, first seen in libmultiprocess sanitizer CI on master. Released Bitcoin Core binaries (v30.0 through v31.1) contain the code; it has only been triggered with a server-owned callback proxy in mptest. Fixed in mp#286 by dropping the check.

Bug 90: Chain IPC calls with an unset transaction would crash the node

2026-05-20T00:00:00Z

If a client of the Chain IPC interface called broadcastTransaction or checkChainLimits without setting the transaction argument, bitcoin-node would dereference a null pointer and crash. Cap'n Proto lets a client leave any field unset, and libmultiprocess turns an unset transaction into a null pointer, which the node code never expected because the in-process wallet always passes a transaction. It was found by reading the code and never triggered. Only the open PR bitcoin#29409, which exposes Chain over IPC, makes it reachable, so master and releases are not affected. The fix is in bitcoin#36520, not yet merged.

Bug 89: On Windows, a thread exiting after IPC calls can deadlock the process

2026-04-20T00:00:00Z

On Windows, a program using libmultiprocess can hang when one of its threads that made IPC calls exits. The thread's per-thread IPC state is cleaned up by a threadlocal destructor, which Windows runs while holding its global loader lock, and that cleanup waits for the event loop. If the event loop is at that moment waiting for a worker thread to start or exit, the worker needs the loader lock, and all three threads wait on each other forever. It was found with the Windows port of the Stratum v2 template provider (sv2-tp) and reproduced with native MSVC and MinGW builds. libmultiprocess does not support Windows on master yet, so no release is affected. The fix is mp#368, a draft.

Bug 62: Spawn test fails on slow CI runners that miss a one-second exit deadline

2026-04-01T00:00:00Z

The mptest case "SpawnProcess does not run callback in child" failed once in Bitcoin Core CI because it gave the spawned child process only one second to exit, which a loaded CI runner did not meet. The defect was in test code and did not affect users: SpawnProcess itself behaved correctly. Fixed by raising the deadline to 30 seconds in mp#266, shipped to Bitcoin Core in the subtree update bitcoin#34977.

Bug 61: CI jobs killed when too many slow IPC tests ran at once

2026-03-25T00:00:00Z

The CI jobs in the libmultiprocess repository that build Bitcoin Core with the library change and run its tests were killed with SIGTERM (exit code 143) a few minutes into running Bitcoin Core's IPC functional tests, on every PR and on master, for about a day. They ran up to ten times as many parallel test processes as CPUs, and once mp#263 raised the test timeout factor from 4 to 40, dozens of slow interfaceipcmining.py runs piled up at once. This was a CI configuration problem and did not affect users. Fixed in mp#264 by lowering the parallelism and run counts.

Bug 63: Thread waits forever when a function it posted to the event loop throws

2026-03-19T00:00:00Z

EventLoop::post() is how another thread runs a function on the event loop thread and waits for it to finish. If a function passed to it threw an exception, the thread that called post() blocked forever, because the event loop skipped the code that marks the call finished and wakes the caller. No caller in libmultiprocess or Bitcoin Core lets a posted function throw, so this was never observed in practice; it was found by code reading and shown with a purpose-built reproducer. Fixed in mp#260 by catching and logging the exception. The code was present in v30.0 through v31.1 with no known way to trigger it.

Bug 85: Disconnect test fails when the server thread starts late

2026-03-12T00:00:00Z

The interfaceipc.py test "disconnect during BlockTemplate.waitNext" disconnects a fixed 0.1 s after the server has queued the waitNext request and then expects the server to log "canceled while executing". When the worker thread starts late, which is common under sanitizers on a loaded CI runner, the server instead cancels the request before it runs and logs "canceled before it could be executed". The test then fails waiting for a line that will never appear. The defect is in test code and does not affect users; the server behaves correctly. No fix yet.

Bug 59: Mining IPC test times out on slow CI because its waits ignore the timeout factor

2026-03-12T00:00:00Z

interfaceipcmining.py failed intermittently on slow CI runners because its waitNext and waitTipChanged timeouts were hard-coded millisecond values that ignored the test framework's --timeout-factor, so a 1-second wait for a new block template could expire before the test's fee-bumping transaction arrived. The defect was in test code and did not affect users. Fixed by commit ad75b147b5c3 ("test: scale IPC mining wait timeouts by timeoutfactor") in bitcoin#34727.

Bug 6: Cancel callback writes to a finished call's stack after a disconnect

2026-03-10T00:00:00Z

An IPC server could write to freed stack memory when a client disconnected just as an asynchronous request was finishing. The request cancellation added in mp#240 lets the server stop work on a call whose client has gone away, and its callback could still run after the worker's stack frame it referenced had gone away. AddressSanitizer reported it as a stack-use-after-return in mptest in Bitcoin Core CI four times in two weeks, and Antithesis saw it too; without ASan it is a silent write that can corrupt whatever reuses that stack. Fixed in mp#249 ("fixes for race conditions on disconnects"), before any release contained the bug.

Bug 5: Server reads freed request parameters when a client disconnects right after calling

2026-03-09T00:00:00Z

An IPC server could crash with a use-after-free when a client disconnected right after sending an asynchronous request: the worker thread read the request's parameters after the disconnect had cancelled the request and Cap'n Proto had freed them. It was seen in Antithesis runs, in libmultiprocess sanitizer CI, and as a plain segfault with a Rust mining client against a Bitcoin Core master build. It was introduced by the request cancellation added in mp#240, which lets the server stop work on a call whose client has gone away. Fixed in mp#249 ("fixes for race conditions on disconnects") and shipped before any release contained the bug.

Bug 7: Server crashes when a client drops a new worker thread right away

2026-03-02T00:00:00Z

An IPC server could crash with a null-pointer dereference when a client created a server thread with ThreadMap.makeThread and dropped it immediately: the new worker thread's Waiter could be destroyed before the thread had locked it. Bitcoin Core's UBSan CI job hit it in interfaceipc.py, and Antithesis hit it as a segfault. The race has existed since makeThread was written. Fixed in mp#249 ("fixes for race conditions on disconnects") by taking the lock earlier; v30.0 through v30.3 binaries have the bug.

Bug 4: Cancellation flag is written unlocked because its lock was a temporary

2026-02-24T00:00:00Z

The request cancellation added in mp#240 lets the server stop work on a call whose client has disconnected. Its cancellation callback took its mutex as an unnamed temporary, so the lock was released immediately and the requestcanceled = true write that followed ran unlocked. In theory a worker thread could then miss a cancel and write results into structs Cap'n Proto was about to free. It was found in code review, was never observed in practice, and was fixed in mp#245 five days after mp#240 merged, before it reached Bitcoin Core or any release.

Bug 56: Mining RPCs report an internal bug, or abort, if the node shuts down mid-call

2026-01-12T00:00:00Z

The getblocktemplate and generateblock RPCs (and the generateBlocks helper behind generatetoaddress) fail a CHECKNONFATAL(blocktemplate) check when the node starts shutting down while they are running. Since 2025, Mining::createNewBlock() returns null when the node shuts down while it waits, so that IPC mining clients never receive a template built on a missing chain tip. These three RPC call sites still assert that it cannot return null. In release builds the caller gets "Internal bug detected: blocktemplate" instead of "Shutting down"; in builds that abort on failed assumptions (such as the Antithesis runs that found it) the node aborts. Released binaries v30.0 through v31.1 have the bug. The proposed fix is bitcoin#36047, not yet merged.

Bug 1: Process spawning leaks a socket if closing the child's end fails

2026-01-09T00:00:00Z

If SpawnProcess() failed to close the child's end of its socketpair in the parent process, it threw without closing the parent's end, leaking a file descriptor. close() on a freshly created socket essentially never fails, so this was never observed in practice. Found in code review and fixed in mp#237 ("Made SpawnProcess() behavior safe post fork()"). In Bitcoin Core releases through v30.x the only caller of SpawnProcess is the hidden echoipc test RPC.

Bug 67: Code generator's child process could deadlock if starting the compiler failed

2026-01-08T00:00:00Z

The mpgen code generator ran the capnp compiler by calling fork() and then, in the child, allocating memory and (if execvp failed) calling perror, iostreams and std::filesystem. In the child of a multithreaded process that can deadlock on a lock another thread held at fork time. It was found in code review and never observed: mpgen is a single-threaded build-time tool, so the precondition does not arise in practice, and nothing that runs in a node or wallet process used this path. Removed in mp#274, which prepared the library for non-Unix platforms, by switching to posixspawnp.

Bug 2: Spawned child process can deadlock before it starts running

2026-01-01T00:00:00Z

A child process spawned with SpawnProcess() could deadlock between fork() and exec(), because the child ran code that allocates memory, and if another thread of the parent held an allocator lock at the moment of fork(), the child inherited it locked. In Bitcoin Core this showed up as an intermittent rpcmisc.py timeout in the echoipc RPC, which spawns a bitcoin-node child. Fixed in mp#237 ("Made SpawnProcess() behavior safe post fork()") by building the argument list before forking. v30.x binaries have the bug, but in releases SpawnProcess is reached only through the hidden echoipc test RPC.

Bug 65: Unit test hangs on a missed wakeup when three calls share one thread

2025-12-05T00:00:00Z

mptest occasionally hung forever in its "Make simultaneous IPC calls on single remote thread" case because the test's response handlers notified the Waiter condition variable without holding its mutex, so the main thread could miss the last wakeup. The defect was in test code and did not affect users. It showed up as 2400 s ctest timeouts on slow Bitcoin Core CI runners (Alpine musl, arm32). Fixed in mp#295, which also adds thread-safety annotations so an unlocked notify no longer compiles under clang's analysis.

Bug 58: Mining client connecting during node startup crashes the node

2025-12-02T00:00:00Z

bitcoin-node aborted at startup with chainman: Assertion mnode.chainman' failed when an IPC mining client connected and called mining methods before the node had loaded its chainstate (the block index and UTXO set). The node starts listening on -ipcbind sockets before it creates the ChainstateManager, so a Stratum v2 client restarted together with the node, or started before it, could crash it. Released v30.0 through v30.3 bitcoin-node binaries are affected, as are v29.0 source builds with IPC enabled; fixed by bitcoin#34661 ("ipc mining: Prevent Assertion mnode.chainman' failed errors on early startup"), which is in v31.0.

Bug 55: Node memory grows while Stratum v2 mining clients are connected

2025-11-24T00:00:00Z

bitcoin-node v30.0 memory grew steadily while Stratum v2 mining clients used the IPC mining interface on mainnet, until a node on a 2 GB machine was killed by the kernel's out-of-memory killer after about a day. The cause is not known, and it is still undetermined whether the growth is a Bitcoin Core bug, a consequence of how many block templates clients keep alive, or a client-side problem. Seen with a released v30.0 binary. No fix yet; tracked in bitcoin#33940 ("Memory leak when using IPC mining interface").

Bug 54: IPC functional test checks a second node before the block reaches it

2025-11-15T00:00:00Z

The interfaceipc.py functional test failed intermittently in CI at its "Block should propagate" check, because it asserted the second node's tip height right after submitSolution returned on the first node, before the block had been relayed. The defect was in test code and did not affect users. Fixed by bitcoin#33880 ("test: Fix race condition in IPC interface block progation test").

Bug 53: Node accepts its own invalid block when a mining client omits the witness commitment

2025-10-30T00:00:00Z

An IPC mining client that called BlockTemplate.submitSolution with a coinbase missing its witness commitment made bitcoin-node accept the invalid block as its own chain tip, while other nodes rejected it, leaving the miner's node stuck on an invalid chain. The node reused the template's CBlock after swapping in the client's coinbase without clearing the cached "witness commitment already checked" flag. This is not a crash or hang; it is a wrong validation result reachable only through IPC. Released v30.0 through v30.3 bitcoin-node binaries are affected, as are v29.0 source builds with IPC enabled; fixed on master by bitcoin#33745 ("mining: check witness commitment in submitBlock").

Bug 3: Node crashes when a disconnected client's call returns a new object

2025-10-06T00:00:00Z

bitcoin-node crashed when an IPC client disconnected while one of its calls was still running on the server and that call then returned an interface pointer, such as a BlockTemplate from waitNext. Nothing cancelled a server request when its connection went away. So when the call finished, the worker thread built a ProxyServer for the returned object from the already-deleted Connection, aborting on Assertion 'mloop' failed or segfaulting. Rust Stratum v2 clients hit it repeatedly against v30.0rc3 and v30.2. Fixed in mp#240 ("Avoid errors from asynchronous (non-c++) clients"), which adds request cancellation, and shipped in Bitcoin Core v31; v30.0 through v30.3 have the bug.

Bug 48: 32-bit x86 IPC server crashes while logging an exception thrown by a server method

2025-10-03T00:00:00Z

On 32-bit x86 builds, an IPC server process crashed with a segfault whenever a server method threw an exception and the server tried to format it for a log message with kj::str(e). The crash is inside kj's out-of-line kj::operator(kj::::Stringifier, const kj::Exception&) formatting function, which receives a reference that points to garbage instead of the exception. It was seen in libmultiprocess's gnu32 CI job and in a Debian i386 sanitizer build of bitcoin-node. The cause is not known. There are no official 32-bit x86 release binaries, so only people who build IPC for that architecture themselves are affected. There is no fix. Of the two places that format an exception this way, one now logs the exception differently and avoids the crash; the other still crashes.

Bug 51: Thread-busy test checks an error message no standard library produces

2025-10-02T00:00:00Z

The mptest test "Make simultaneous IPC callbacks with same requestthread and callbackthread", added in mp#214 (which fixed a crash on simultaneous IPC calls to one thread), compared std::futureerror::what() against the literal "Future already retrieved". Neither libstdc++ nor libc++ produces that text, so the check failed. The failure was printed but did not fail the test, because the check ran off the main test thread. The defect was in test code and did not affect users. Fixed in mp#222 ("test, ci: Fix threadsanitizer errors in mptest") by comparing the error code instead.

Bug 50: Thread-busy test copies capabilities off the event loop thread

2025-10-02T00:00:00Z

The mptest test "Make simultaneous IPC callbacks with same requestthread and callbackthread", added in mp#214 (which fixed a crash on simultaneous IPC calls to one thread), copied Thread::Client capabilities on the main test thread. That raced the event loop thread on Cap'n Proto's non-atomic reference counts. Bitcoin Core's TSan CI job reported it, and it blocked the libmultiprocess subtree updates for master and 30.x until fixed. The defect was in test code and did not affect users. Fixed in mp#222 ("test, ci: Fix threadsanitizer errors in mptest").

Bug 49: Test log callback writes to std::cout from several threads unlocked

2025-10-02T00:00:00Z

The mptest log callback wrote to std::cout from several threads at once without a lock, which ThreadSanitizer reported as a data race inside libc++'s stream formatting once libc++ itself was built with TSan. The defect was in test code and did not affect users. Fixed in mp#222 ("test, ci: Fix threadsanitizer errors in mptest") by logging through KJLOG instead.

Bug 8: Node aborts when freeing a callback object from a client that disconnected

2025-09-29T00:00:00Z

When a client passes a callback object to the node, such as a handler for chain notifications, the node holds a ProxyClient for it and calls the callback's destroy method when it frees it. bitcoin-node aborted with std::terminate when a server-owned ProxyClient for a callback object was destroyed after the peer had disconnected: the proxy's cleanup made a remote destroy() call on the dead connection, which threw inside a noexcept destructor. It was seen with a Rust wallet client against the branch of bitcoin#29409, which adds an IPC wrapper for the node's Chain interface so a wallet can run as a separate process. Released binaries (v30.0 through v31.1) contain the code but cannot reach it, because no interface on master has a server-owned callback proxy with a destroy method. Fixed in mp#273.

Bug 46: Python IPC test aborts when libmultiprocess is also installed system-wide

2025-09-17T00:00:00Z

The interfaceipc.py functional test aborted at startup with mp/proxy.capnp:0: failed: Duplicate ID @0xcc316e3f71a040fb on machines where libmultiprocess was installed system-wide in the same prefix as Cap'n Proto, because the test's schema import path listed the system include directory before the in-tree one. The test then loaded two copies of the same schema file. The defect was in test code: only the Python test process aborted, no node was involved, and users were not affected. Fixed by bitcoin#33420 ("test: Avoid interfaceipc.py Duplicate ID errors").

Bug 44: Node aborts at shutdown while an IPC mining call is still running

2025-09-14T00:00:00Z

bitcoin-node can abort on Assert(mnode.chainman) when it shuts down while an IPC mining client has a call in progress. During shutdown the node disconnects its IPC clients and then frees the ChainstateManager. Disconnecting does not stop a server method that is already running on a libmultiprocess worker thread, so that method can call MinerImpl::chainman() after the pointer has been cleared. It was seen with a Stratum v2 client (sv2-tp) when the node was stopped with Ctrl-C, and in an Antithesis run. Released binaries v30.0 through v31.1 have the bug. The fix is in two open PRs, bitcoin#35932 and mp#335.

Bug 57: Node terminates when a call on a worker thread throws, e.g. on bad input

2025-09-08T00:00:00Z

bitcoin-node terminated when an IPC method running on a server worker thread threw an exception, for example while deserializing a malformed argument such as an empty coinbase passed to BlockTemplate.submitSolution. The exception escaped the worker thread and std::terminate ended the process, instead of an error being returned to the client. Any client able to send malformed arguments (in practice Python or Rust clients) could crash the node. Fixed in mp#240 ("Avoid errors from asynchronous (non-c++) clients") and shipped in Bitcoin Core v31; v30.0 through v30.3 have the bug.

Bug 47: Node aborts when a client sends overlapping calls to one server thread

2025-09-05T00:00:00Z

An IPC server aborted on Assertion '!mfn' failed in Waiter::post when a client sent a second request for a server thread that already had a request waiting. The libmultiprocess C++ client never does this, but Python and Rust clients that reused one Context.thread for overlapping asynchronous calls could crash bitcoin-node. It was seen in Bitcoin Core's first Python IPC functional test and twice with a Rust Stratum v2 pool. Fixed in mp#214, which returns a "thread busy" error instead. v30.0 has the bug; the fix reached the 30.x branch afterwards.

Bug 71: IPC code built on NetBSD with GCC before 14 has exception handling compiled out

2025-08-23T00:00:00Z

On NetBSD with GCC older than 14, Cap'n Proto's kj/common.h mistakes a stub hasfeature macro in for "exceptions disabled", so code built there has its exception handling compiled out. With the pkgsrc capnproto package this made bitcoin-node terminate whenever an IPC server method threw (the client sees "Peer disconnected" instead of an error reply), made mptest abort, and, with mixed compilers, made mptest fail to link. The defect is a build-time misdetection in an upstream header; nothing in libmultiprocess or Bitcoin Core code is wrong, and it affects only people building IPC code on NetBSD with an older GCC. libmultiprocess CI works around it in its NetBSD jobs (mp#196, which added them, and mp#321, which moved them to NetBSD 11.0); the upstream fix (capnproto#2756) is not in a capnproto release, and compile-time checks are proposed in mp#364 and bitcoin#36060.

Bug 42: Mining client that leaves block options unset crashes the node

2025-08-19T00:00:00Z

bitcoin-node aborted on an Assert when an IPC client called Mining.createNewBlock with BlockCreateOptions fields out of range, including a client that left the fields at their Cap'n Proto zero defaults. Any non-C++ client (Python, Rust) that did not set every field could crash the node. Fixed by bitcoin#33222 ("miner: clamp options instead of asserting"), which clamps the values instead, before v30.0 shipped bitcoin-node.

Bug 43: IPC build hangs forever with Cap'n Proto 0.8 in C++20 mode

2025-08-12T00:00:00Z

Building Bitcoin Core with IPC enabled against Cap'n Proto 0.8 (for example Ubuntu 22.04's libcapnp-dev) never finished, because the mpgen code generator hung in an infinite loop. The cause was a const char/StringPtr comparison operator in Cap'n Proto 0.8's kj/string.h that calls itself when compiled as C++20. This was a build-time problem only; programs that did get built were not affected. Worked around in mp#194, which also added a CI job building against old dependencies, and shipped to Bitcoin Core before v30.0.

Bug 37: Double free when a thread exits while its connection is being destroyed

2025-07-10T00:00:00Z

When a thread that had made IPC calls exited at the same moment its Connection was destroyed on the event-loop thread, both sides could destroy the same ProxyClient entry (the handle to the server thread that runs this thread's calls) in the thread's requestthreads / callbackthreads maps, giving a heap-use-after-free and a glibc "double free or corruption" abort. This is the thread-exit path of the race family described in mp#189; the in-call path was fixed separately in mp#201 (bug page). It was seen as an intermittent CI crash in a new mptest case. Bitcoin Core v29 through v31.1 contain the code, and the fix commit says it is reachable on master through ConnectStream connections when the peer dies while such a thread exits. A fix is in mp#361, which fixes several disconnect bugs found in review, not yet merged.

Bug 45: Server deadlocks when a client disconnects just as a call returns

2025-07-01T00:00:00Z

An IPC server could deadlock (or, with different timing, crash) when a client disconnected just as a server method was returning, because a worker thread and the event loop thread both tried to destroy the same per-connection ProxyClient, the server's handle for the client thread that made the call. It made the mptest "disconnecting and blocking" test hang intermittently in Bitcoin Core CI, and the same lock cycle stopped a v30.0 bitcoin-node from answering IPC calls under a Rust Stratum v2 client with no disconnect involved. Fixed in libmultiprocess by mp#201 and shipped to Bitcoin Core master and the 30.x branch.

Bug 35: Disconnect test destroys a promise its server thread still needs

2025-06-28T00:00:00Z

The mptest test "Calling IPC method, disconnecting and blocking during the call" could destroy a std::promise that its server thread was still about to use, which MemorySanitizer reported as a read of destroyed memory. The defect was in test code and did not affect users. It was seen once in Bitcoin Core's MSan CI job and fixed in mp#186, a PR that fixed several mptest failures seen in Bitcoin Core CI, by reordering two declarations.

Bug 34: Unlocked cleanup state races when a client disconnects as a call returns

2025-06-27T00:00:00Z

When a client disconnected while a server worker thread was finishing an IPC call, the event loop thread, destroying the Connection, read and wrote state shared with the worker without holding the locks the worker used. ThreadSanitizer reported three data races in the mptest test "Calling IPC method, disconnecting and blocking during the call". The unlocked list walk and pointer accesses could in principle crash, but no crash was attributed to them. Fixed in mp#186, a PR that fixed several mptest failures seen in Bitcoin Core CI, by taking the missing locks. The code was in Bitcoin Core source releases v22 through v29, which did not ship IPC binaries.

Bug 33: Calling a method after disconnect binds a reference to a null connection

2025-06-27T00:00:00Z

When a C++ IPC client called a method after its connection was closed, clientInvoke (the library function that sends a call and waits for the reply) bound a Connection& reference to a null pointer before checking for the disconnect. The reference was never used on that path, so the only effect was an UndefinedBehaviorSanitizer report in the mptest test "Call IPC method after client connection is closed". The bad ordering existed on libmultiprocess master for twelve days (2025-06-19 to 2025-07-01) and never reached Bitcoin Core's subtree or any release. Fixed in mp#186, a PR that fixed several mptest failures seen in Bitcoin Core CI.

Bug 32: Destroying a client proxy races with the event loop closing its connection

2025-06-27T00:00:00Z

When a ProxyClient object was destroyed while its connection was being torn down, its cleanup handler called Connection::removeSyncCleanup on the destroying thread. Nothing synchronized that call with the event loop thread, which could be destroying the same Connection at that moment. ThreadSanitizer reported it as a data race in the mptest "disconnecting and blocking during the call" test. With different timing it could have crashed, but no crash was attributed to it. Fixed in mp#186 by making the call on the event loop thread. The code was in Bitcoin Core source releases v22 through v29, which did not ship IPC binaries.

Bug 36: Test fixture starts its thread before the members that thread writes

2025-06-26T00:00:00Z

The mptest fixture TestSetup declared its std::thread member before the members that thread writes, so the thread could start and write those members while the constructor was still default-initializing them, possibly overwriting the thread's values. ThreadSanitizer reported it in every test that used the fixture. The defect was in test code and did not affect users. Fixed in mp#186, a PR that fixed several mptest failures seen in Bitcoin Core CI, by moving the thread to the end of the member list.

Bug 31: Reading IPC fields calls a function through a dangling reference, without visible effect

2025-06-19T00:00:00Z

ReadDestEmplace, a helper libmultiprocess uses when reading fields of incoming messages, stored a reference to the function object passed to it. ReadDestTemp passed it a temporary lambda, so every field read through ReadDestTemp called that lambda through a dangling reference. This is undefined behavior in the standard's terms, but the lambda had no state and its call never touched the object, so no misbehavior was ever seen; only clang-tidy's static analyzer flagged it. Fixed in mp#184 by storing the function object by value. The code was in Bitcoin Core source releases v25 through v29, which did not ship IPC binaries.

Bug 28: Server can hang at shutdown if a client connects as the last one leaves

2025-06-10T00:00:00Z

An IPC server could hang at shutdown if a client connected just as the last one was going away. The event loop's async cleanup thread destroys server objects left behind by unclean disconnects. It exited for good as soon as it saw zero clients. Cleanup work queued by a connection that arrived afterwards was never run, so the event loop never finished and the process never exited. It was found through @darosior's connect/disconnect stress test with a Rust wallet client. No log of the hang was posted, and the fix's commit message is the only description of it. Source builds of v22 through v29 with multiprocess enabled have the defect. Fixed in mp#160, shipped in v30.0.

Bug 68: Windows child processes hang at shutdown because closing a socket dropped queued messages

2025-05-02T00:00:00Z

On Windows, when a process destroyed a Connection over a socketpair, the Cap'n Proto Release messages still queued for the peer were discarded. Cap'n Proto sends a Release message when a process drops its last reference to a remote object; without it, the peer never freed its ProxyServer objects, and child processes spawned over IPC hung at shutdown. It was found while porting libmultiprocess to Windows and never affected a released or supported platform. Fixed in mp#274, which prepared the library for non-Unix platforms, by flushing pending writes before closing the stream; not yet in Bitcoin Core's subtree.

Bug 52: Ctrl-C cannot stop the node while a mining client waits for a new tip

2025-05-01T00:00:00Z

bitcoin-node could not be stopped with Ctrl-C or SIGTERM while an IPC client had a Mining.waitTipChanged call (wait until the chain tip changes) outstanding: shutdown printed "Shutdown: done" and the process then hung until killed. The signal path never woke the condition variable the call was waiting on, so a client that used a long timeout kept the IPC event loop alive forever. Affected v30.0 and v30.1; fixed on master by bitcoin#33511 ("init: Fix Ctrl-C shutdown hangs during wait calls") and backported to the 30.x branch in bitcoin#34192 (released in v30.2).

Bug 26: Node started with -disablewallet aborts when a GUI with wallets enabled connects

2025-03-27T00:00:00Z

A bitcoin-node started with -disablewallet aborts on an Assert in NodeImpl::walletLoader when a bitcoin-gui started without -disablewallet connects to it over IPC, and the GUI hangs. The GUI asks the node for its wallet loader, and the node asserts that it has one. That was safe when the GUI and node ran in one process with one set of options, but over IPC the two processes can be configured differently. The trigger needs bitcoin-gui -ipcconnect, which only exists in the unmerged bitcoin#19461, so no release is affected, but the asserting line is still on master. No fix yet.

Bug 27: Mining client calls during node startup or early shutdown crash the node

2025-02-03T00:00:00Z

An IPC mining client that called Mining.waitTipChanged or Mining.createNewBlock during node startup, before a chain tip was connected, or during an early shutdown, could crash bitcoin-node: waitTipChanged dereferenced a null Tip() and createNewBlock hit assert(pindexPrev != nullptr). The IPC server starts listening before init waits for the tip, so an early client could reach this window. It was found by reading the code and never seen at runtime. v29.0 source builds with IPC enabled have the bug; fixed by bitcoin#31785 ("Have createNewBlock() wait for tip, make rpc handle shutdown during long poll and wait methods") before v30.0 shipped bitcoin-node.

Bug 25: IPC unit tests abort on 32-bit CI when GCC and clang code are mixed at -O0

2025-01-31T00:00:00Z

With depends and the main build both at -O0, ipctests aborted deterministically on the i686 (32-bit x86) multiprocess CI job with a kj "Out-of-bounds ArrayPtr::slice()" exception thrown from the EventLoop constructor. kj is the support library that ships with Cap'n Proto. The job built Cap'n Proto with GCC and Bitcoin Core with clang, and on i686 the two compilers pass kj's empty PlacementNew argument differently, so a GCC-built kj::ctor called a clang-built operator new and got back a wrong pointer. This was a CI toolchain problem in third-party headers, not a defect in libmultiprocess or Bitcoin Core code, and it did not affect users. Fixed by bitcoin#31998 ("depends: patch around PlacementNew issue in capnp"), which patches Cap'n Proto in depends.

Bug 41: Node never finishes shutting down while an IPC client stays connected

2024-12-16T00:00:00Z

bitcoin-node could not finish shutting down while a client connected through -ipcbind (the option that lets outside processes connect over a Unix socket) stayed connected. It logged Shutdown: done and then waited for every IPC client to disconnect on its own, which a client that ignored the node going away never did. Any external IPC client, such as a Stratum v2 mining client, could trigger it. v29.0 source builds with IPC enabled have the bug; fixed by bitcoin#32345 ("ipc: Handle unclean shutdowns better"), which disconnects incoming IPC clients during shutdown, before v30.0 shipped bitcoin-node.

Bug 40: Node crashes at shutdown when a client left a notifications handler behind

2024-12-16T00:00:00Z

bitcoin-node segfaulted at shutdown when an IPC client had registered for chain notifications and dropped the returned Handler without calling destroy (the IPC message that tells the server to free an object). libmultiprocess deferred destroying the node-side NotificationsHandlerImpl until the connection was torn down, and by then Shutdown() had already freed the ValidationSignals object its destructor unregisters from. It needs Chain.handleNotifications over IPC, which only the unmerged wallet-over-IPC branch bitcoin#29409 ("multiprocess: Add capnp wrapper for Chain interface") exposes, so no release could reach it. Fixed by bitcoin#32345 ("ipc: Handle unclean shutdowns better") together with mp#160 ("refactor: EventLoop locking cleanups + client disconnect exception").

Bug 29: Client aborts if the connection closes just as it starts a call

2024-12-03T00:00:00Z

An IPC client could abort with memory.h:258: failed: expected ptr != nullptr; null Own<> dereference if the connection closed just as it started a call. clientInvoke, the library function that sends a call, checked on the calling thread that the connection was still open, and only then handed the request to the event loop thread. If the disconnect was handled in between, the request was built on a capability that had already been cleared. It was hit by bitcoin-node on Ctrl-C when running with a bitcoin-wallet subprocess on the wallet-over-IPC branch, and by bitcoin-cli -ipcconnect connecting to a socket that was not a Cap'n Proto server. On master the only C++ IPC client whose server can disconnect this way is the node's echoipc connection to its own child, so no released code path is known to hit it. Source builds of v22 through v29 with multiprocess enabled have the defect. Fixed in mp#160, shipped in v30.0.

Bug 23: Scripts passed to the node over IPC are decoded in the wrong format

2024-12-02T00:00:00Z

A CScript passed to bitcoin-node over IPC, for example the coinbase script given to Mining::createNewBlock, was decoded wrongly. libmultiprocess converts each C++ type to and from a Cap'n Proto field with a pair of overloaded functions, one to build the field and one to read it. For scripts the sender wrote raw bytes, but there was no matching read function, so the receiver parsed the bytes with Bitcoin serialization, which expects a length prefix first. @Sjors reported it as a crash while testing a Stratum v2 template provider, and @maflcko reproduced it under valgrind as a memory error in SpanReader::read. The read is bounds-checked, so a release build most likely threw an exception from the server method or decoded a wrong script rather than segfaulting; the exact failure in a normal build was not recorded. No release was affected: v28 had no mining IPC interface and v29 has the fix. Fixed in mp#135 by adding a read function that matches the raw-bytes build function, shipped in bitcoin#31740.

Bug 21: Client proxy code casts to a derived object before it exists or after it is gone

2024-11-22T00:00:00Z

Every time a ProxyClient was constructed or destroyed, the base class ProxyClientBase did a staticcast from itself to the derived ProxyClient type inside its own constructor and destructor. At those points the derived object does not exist yet, or no longer exists. That is undefined behavior, and UBSan reported it as a dynamic-type-mismatch downcast in Bitcoin Core's ipctests. It had no practical effect, because the generated ProxyClient classes add no state of their own. Source builds of v22 through v28 with multiprocess enabled have it. Fixed in two steps, mp#121 (constructor) and mp#127 (destructor).

Bug 22: Spawned node process aborts at shutdown with a "Broken pipe" error

2024-10-24T00:00:00Z

A bitcoin-node child process spawned over IPC could abort at shutdown with terminate called after throwing an instance of 'kj::ExceptionImpl' what(): mp/proxy.cpp:242: disconnected: write(mpostfd, &buffer, 1): Broken pipe. Other threads wake the event loop by writing a byte to a pipe, and the loop closes that pipe when it exits. EventLoop::removeClient sometimes wrote an extra wake-up byte before the loop was really done, so the loop could exit and close its pipe just before another thread's final wake-up write. That write then failed with EPIPE and threw an exception nothing caught. It made rpcmisc.py fail intermittently in Bitcoin Core CI. Source builds of v22 through v28 with multiprocess enabled have it. Fixed in mp#129.

Bug 30: Server crashes using a freed connection after a client disconnects mid-call

2024-10-16T00:00:00Z

When an IPC client disconnected while a server method it had called was still running, the server's Connection object was destroyed before the ProxyServer object handling the call. Code that later found the event loop through the ProxyServer's mcontext.connection pointer read the freed Connection, and bitcoin-node crashed. It was seen three times over eight months: with a mining client killed during a long wait, with a Rust wallet client connecting and disconnecting in a loop, and with bitcoin-cli -ipcconnect run repeatedly against a node. A workaround in mp#118 covered only the first of three code paths, at the cost of a memory leak. The fix came in mp#160, shipped to Bitcoin Core in bitcoin#32345. Source builds with multiprocess enabled before v30.0 have the bug.

Bug 20: Server crashes when a client disconnects during a long-running call

2024-10-08T00:00:00Z

If an IPC client disconnected while a server method it had called was still running, the server crashed when that method returned. While a call runs, the server's worker thread keeps an entry in its requestthreads map that records which client thread to send callbacks to. The worker thread erased that entry after the call through an iterator saved before the call, but the disconnect had already removed the entry on the event loop thread. @tdb3 hit it every time by pressing Ctrl-C on a mining client blocked in a long wait and then mining a block. Source builds of v22 through v28 with multiprocess enabled have the bug, which became reachable on master once the mining interface had methods that wait a long time. Fixed in mp#118 by erasing the entry by key instead.

Bug 19: Closing a connection writes to freed memory and leaks server objects

2024-09-26T00:00:00Z

When a libmultiprocess Connection was destroyed while the other side still held references to objects this side was serving, the cleanup of those objects wrote into memory that had already been freed. This is undefined behavior, and the cleanups queued that way never ran, so the wrapped objects leaked. The cause was the order in which ~Connection destroyed its members: the lists of cleanup functions went before the Cap'n Proto RpcSystem, and it is the RpcSystem's teardown that disposes of the remaining ProxyServer objects and queues their cleanups. It showed up only as a MemorySanitizer report in Bitcoin Core CI. Every connection teardown with undestroyed server objects hit it, in source builds of v22 through v28 with multiprocess enabled. Fixed in mp#116.

Bug 18: Client crashes after reconnecting because it reuses a remote thread from the old connection

2024-08-07T00:00:00Z

A client thread that made IPC calls over one connection, disconnected, and then connected again could segfault. Each client thread remembers which worker thread on the server runs its calls, per connection, in a map keyed by the raw Connection pointer. libmultiprocess never removed the entry when the connection went away. If the new Connection was allocated at the old one's address, the next call went to a thread belonging to the dead connection. It was found by a new Bitcoin Core unit test that disconnects and reconnects, before Bitcoin Core accepted IPC connections from outside programs. Source builds of v22 through v28 with multiprocess enabled have the defect, but nothing in them reconnects this way outside that test. Fixed in mp#106.

Bug 17: Wallet process hangs and keeps its wallets locked after the node is killed

2024-05-24T00:00:00Z

When the connection between bitcoin-node and a spawned bitcoin-wallet process broke (for example because the node was killed), the wallet process hung forever and kept its wallet directory locked. When a connection breaks, libmultiprocess destroys the objects the process was serving to its peer. It destroyed them in the order they were created, so the wallet process destroyed its WalletLoader (the object the node uses to load and unload wallets) before the Chain::Notifications objects that still held references to the wallets. The WalletLoader destructor waited for those references forever. The multiprocess wallet exists only on the unmerged bitcoin#10102 branch, so no release could hit it. Fixed in mp#101 by running these cleanups in reverse creation order.

Bug 16: Wallet process hangs holding its wallet lock after the node is killed

2024-05-24T00:00:00Z

On the unmerged multiprocess branch where bitcoin-node spawns a separate bitcoin-wallet process (bitcoin#10102, "Multiprocess bitcoin"), killing the node could leave the wallet process hung forever, still holding its wallet directory lock, so the wallet could not be opened again. The wallet process runs a scheduler thread for periodic wallet tasks, and some of those tasks make IPC calls to the node. When the connection broke, the wallet's WalletLoader server object stopped that scheduler from its destructor, which runs on the IPC event loop thread, while the scheduler thread could be waiting on that same event loop for an IPC call to finish. Each thread waited for the other. The code only exists on the bitcoin#10102 branch (and on bitcoin#29409, "multiprocess: Add capnp wrapper for Chain interface", which it builds on); it was never on master or in a release. The fix is in that branch too, so it will only be merged along with the PR.

Bug 24: IPC shutdown locks a mutex after its event loop was destroyed

2024-04-16T00:00:00Z

At IPC shutdown, EventLoop::post and EventLoop::removeClient could lock EventLoop::mmutex again after EventLoop::loop() had returned. Bitcoin Core destroys the EventLoop as soon as loop() returns, so the lock was on a destroyed mutex. Depending on the platform this threw std::systemerror: mutex lock failed: Invalid argument (macOS, FreeBSD, NetBSD), tripped a glibc assertion, or hung (Linux). It made ipctests and rpcmisc.py fail intermittently in Bitcoin Core CI, and in rpcmisc.py the parent bitcoin-node exited with an uncaught exception after "Shutdown: done". Source builds of v22 through v28 with multiprocess enabled have it. Fixed in mp#159, shipped in v29.0.

Bug 15: Node aborts when a separate GUI process opens confirmed transaction details

2022-09-12T00:00:00Z

In the multiprocess GUI, where bitcoin-gui runs in its own process and talks to bitcoin-node over IPC, opening the details of a confirmed transaction could kill the node process. Node::getUnspentOutput left its Coin& output parameter in the spent state when the coin was not found, and the IPC server serialized that parameter anyway, hitting assert(!IsSpent()) in Coin::Serialize. The interface contract dates from 2018 and was in releases v22 through v26, but no released binary could reach it, because the in-process GUI checks the return value; only the out-of-process GUI of the unmerged bitcoin#10102 ("Multiprocess bitcoin") branch crashed. Fixed on master in bitcoin#28721 ("multiprocess compatibility updates").

Bug 14: Separate GUI process crashes when broadcasting a PSBT

2021-11-12T00:00:00Z

In the multiprocess GUI, where bitcoin-gui runs in its own process and talks to the node over IPC, clicking "Broadcast" in the PSBT operations dialog crashed bitcoin-gui with a null pointer dereference. The dialog called a node-internal function through node().context(), which is null when the node runs in another process. Released binaries (v22.0 included) contain the call but cannot reach it, because their GUI runs in the same process as the node; only the out-of-process bitcoin-gui of the unmerged multiprocess branches crashed. Fixed on master by bitcoin#23499 ("multiprocess: Add interfaces::Node::broadCastTransaction method") ahead of that feature.

Bug 13: GUI address book test aborts after two separately merged PRs combine

2021-09-16T00:00:00Z

testbitcoin-qt aborted deterministically in AddressBookTests on an Assert of a null WalletClient after bitcoin#22219 ("multiprocess: Start using init makeNode, makeChain, etc methods"), a refactor for the multiprocess split, merged. The defect was in test code (the address book test fixture never created a wallet client) and did not affect users. Fixed the same day by bitcoin#22992 ("Fix Qt test broken by #22219").

Bug 12: GUI aborts at startup after a refactor of node argument setup

2021-06-11T00:00:00Z

For about 15 hours on 2021-06-11, bitcoin-qt built from master aborted at startup with Assertion '"node.args" && check' failed in WalletInit::Construct. A refactor that prepared argument setup for the multiprocess split, bitcoin#22216, moved the node.args = &gArgs assignment out of SetupServerArgs into the bitcoind and bitcoin-node startup code, and the GUI's startup function GuiMain never got one. This was not a bug in the IPC mechanism, only in a preparatory refactor, and no release was affected. Fixed the same day by a one-line change in gui#361.

Bug 11: Multiprocess CI job runs out of memory linking the debug GUI binary

2021-05-24T00:00:00Z

The Cirrus "[multiprocess, DEBUG] [focal]" CI task, the only CI job that built Bitcoin Core with IPC support at the time, started failing reliably in May 2021 because the container's memory limit killed the linker while it linked bitcoin-gui. This was a CI resource problem, not a code defect, and it did not affect users. Fixed by bitcoin#22061 ("ci: Bump multiprocess memory"), which gave the task more memory and CPUs.

Bug 39: Ctrl-C kills the spawned wallet process before the node can shut it down

2020-03-05T00:00:00Z

On the unmerged multiprocess branch where bitcoin-node spawns a bitcoin-wallet process, pressing Ctrl-C in the terminal also sent SIGINT to the spawned wallet, which had no handler for it and died at once. That broke the node's orderly shutdown, in which the node tells the wallet to flush and stop, and made bitcoin-node abort. The missing handler was on master from 2021 and in releases v22 through v29, but no release was affected in practice: the only process spawned on master is the short-lived echoipc child. Fixed by bitcoin#32345 ("ipc: Handle unclean shutdowns better"), which makes spawned processes ignore Ctrl-C and wait for the parent.

Bug 38: Node aborts during shutdown if the wallet process has already exited

2020-03-05T00:00:00Z

On the unmerged multiprocess branch where bitcoin-node spawns a bitcoin-wallet process, bitcoin-node aborted during shutdown with an uncaught ipc::Exception (or, in older versions, std::logicerror) if the wallet process had already gone away. Shutdown() made IPC calls to the wallet and had nothing to catch the error a broken connection raises. The unguarded call was on master from 2018 and in releases v22 through v29, but released binaries could not reach it, because their chain clients (the wallet objects the node notifies and stops) run in-process; only the spawned wallet of the unmerged bitcoin#10102 ("Multiprocess bitcoin") branch triggered it. Fixed by bitcoin#32345 ("ipc: Handle unclean shutdowns better").

Bug 10: Connection deleted twice when its disconnect handler ran after destruction

2019-08-06T00:00:00Z

A Connection's disconnect handler could run after the Connection had already been destroyed and delete it a second time. It showed up at shutdown on the bitcoin#10102 branch, which splits Bitcoin Core into node, wallet and GUI processes, as soon as destroying the last client proxy started destroying the Connection directly on the event-loop thread. This was in libmultiprocess's first weeks, before Bitcoin Core used the library, so no release was affected. Fixed in mp#7, the same PR that added the destroyconnection option which exposed it; a narrower window of the same bug remained (bug page) and is addressed in mp#361, not yet merged.

Bug 9: Unbounded IPC debug logging made the test harness run out of memory

2019-07-09T00:00:00Z

IPC debug logging wrote the full escaped text of every request and response to debug.log with no size limit, so on the multiprocess branch the log grew large enough that the functional test harness died with a Python MemoryError reading it. The node itself did not fail. This was in libmultiprocess's first weeks, before Bitcoin Core used the library, so no release was affected. Fixed in mp#3 by truncating logged messages to about 1000 characters.