Corrections: the fog of war

The explanation written when a bug is fixed is often partly wrong. A commit message is written by someone who has just found a way to make a failure stop, usually under time pressure, often from one stack trace, and before anyone has read the code with the answer in hand. A review comment is written from what the reviewer can see in the diff that day. Later, with the fix merged, more reports collected, sanitizer output and debugger traces to compare, and sometimes a second bug that shows how the first one really worked, the picture gets clearer.

This page collects the places where the survey's later reading of the evidence differs from what was said at the time. None of these are mistakes in the ordinary sense. They are what debugging looks like: people act on the best explanation available, and the explanation improves afterwards. In most entries below the fix itself was right; what changed is the account of what it fixed, how, and since when.

Each entry says what was said, where and when, what the evidence shows, and links to the bug page that has the detail. Where the survey is inferring rather than reading it from the code or a log, the entry says "most likely". Several entries correct the maintainer's own commits and comments. Corrections to these corrections are welcome.

What caused the failure

Linker out of memory on the multiprocess CI job (2021). A reviewer comment on 2021-05-25 suggested that bitcoin#21869 had pushed the job over its memory limit. Later analysis found that the suspected commit, fa9249aaccc3, only adds a libc++ debug flag, which has no effect on a job that builds against libstdc++. What tipped the link over the limit that week is not known. Bug page

A node that would not exit after Ctrl-C with a waiting client (2025). When pinheadmz reported that a node printed "Shutdown: done" and never exited while bitcoin-cli waited in waitforblock over IPC, the reply on 2025-05-01 put it down to known issues that bitcoin#32345 would fix. Later analysis found that the worker serving waitforblock sat in a wait that Ctrl-C never woke, which is the bug fixed separately in bitcoin#33511; closing the connections at shutdown would not have released it. Bug page

A crash report attributed to the startup fix (2026). For the crash in bitcoin#34664, a comment on 2026-02-24 hoped that bitcoin#34661, the fix for a client connecting during startup, would cover it. Later analysis of the report's log found the sequence of a different bug: the Mining server destroyed, the socket disconnected, then the abort right after the next createNewBlock(). Most likely it was the abandoned-call crash, fixed by request cancellation.

A CI timeout bump expected to stop a test failure (2026). mp#263 (2026-03-25) raised the CI timeout factor to 40, saying it "should prevent false positives like" a failure seen in mp#256. Later analysis found that failure was a test race: the test waits for a log line that is never written when the worker thread starts after the disconnect, so a longer timeout only waits longer. The failures stopped after mp#264 cut the number of parallel test runs, most likely because it lowered the load. Bug page · related CI bug

Which Windows builds were affected by a thread-exit crash (2026). The code comment in the unmerged MinGW workaround in mp#318 (2026-07-30) said the crash affected msvcrt builds "but not x86_64-w64-mingw32ucrt builds". A revert test reported on 2026-08-05 showed the ucrt job failing too. The bug does not depend on the C runtime. Bug page

How the failure happened

"Segfault" that was an assert: spent coin over IPC (2023). The fix commit 156f49d682ef (2023-10-20) describes a "multiprocess segfault". Later analysis of the code at the time found an assertion failure, assert(!IsSpent()) in Coin::Serialize, when the node serialized a spent coin for the GUI. Either way the node process died. Bug page

"Gui segfault" that was an assert (2021). The fix commit d7f3b1af21d (2021-06-11) is titled "Fix gui segfault". The stack trace and bitcoin#22227 show an assertion failure, not a segmentation fault. Bug page

"Null pointer dereference" that was an assert (2021). The fix PR bitcoin#22992 (2021-09-16) calls the test failure a "null WalletClient pointer dereference". The CI log shows an Assert catching the null pointer and aborting before any dereference. Bug page

"Null pointer dereference" that was a use-after-free (2024). The workaround commit 196e6fcdbc1f (2024-10-16) calls the crash a null pointer dereference. Later analysis found the connection pointer was not null: it pointed to a freed Connection, and the null event loop value most likely came from reading that freed memory. AddressSanitizer later reported the same bug as a heap-use-after-free. Bug page

What the old waitTipChanged did before the node had a tip (2025). The fix commit 64a2795fd4fe (2025-03-13) says the old code "would return the last known tip during shutdown". That holds once a tip exists. Before one is connected, the old code dereferenced a null tip, as pointed out in review. Bug page

When a cancel can land (2026). The fix commit e0f1cd762190 (2026-02-24) describes the event loop cancelling "the request before the execution thread runs", after which the worker touches "deleted parameters". Later analysis found the window narrower and later: the cancel callback is only registered once the worker is running the request, and what the worker touches afterwards is the results. Bug page

Why a test promise needed to outlive the server thread (2025). The code comment added by 84b292fcc4db (2025-06-28) says signalling "may not wake up the thread right away". Later analysis found that a thread already waiting in get() was never at risk, because the future keeps the shared state alive. The only window was the one in the reported trace: the server thread had not yet called get_future(). The reordering in the fix closes it. Bug page

How long a descriptor leak window lasted (2026). The fix commit 1e0c7ff9a51a (2026-07-20) places the window between the fcntl call and fork(). Later analysis found it lasted until the parent closed its copy of the child's socket after fork(). Bug page

When the build tool's forked child could deadlock (2026). The fix commit 1c6ef7a26c0d (2026-07-13) says the child "could deadlock if exec() failed". Later analysis found a slightly wider window: the child also allocated memory before execvp on every call, not only on failure. Bug page

Whether exceptions in posted callbacks were already handled (2026). The report mp#259 (2026-03-19) says existing callers are safe because clientInvoke captures exceptions. Later analysis found that only its response handler does. The request-building half has never caught exceptions, and is safe only because no current argument builder throws; one that did would leave the calling thread waiting forever. Bug page · related open bug

What an Assume does to an IPC caller (2025). A summary of bitcoin#31785 on 2025-02-19 describes the change as adding "an Assume check to disallow passing negative timeout values". That holds for callers in the same process. Over IPC the same function is the entry point for client input, and an Assume does not disallow anything: in checked builds it aborts the node, and in release builds it is skipped. Bug page

How serious a failed CHECK_NONFATAL at shutdown is (2026). A comment on 2026-08-31 called the proposed fix "basically a small style cleanup". For release builds that is close: the node keeps running and the caller gets a misleading error. In builds that abort on a failed CHECK_NONFATAL, such as the Antithesis runs that found it, the node aborts. Bug page

A string check that failed on every platform (2025). The fix commit c332774409ad (2025-10-02) says the test's error-message check "does not work on all platforms". Later analysis found the expected text matched neither libstdc++'s message nor libc++'s, so most likely it worked on no platform the tests run on. The failed check did not fail the test, which is why nobody noticed. Bug page

What a fix did

A "refactor" that fixed a crash (2026). Commit c4762c7b513a (2026-01-22) says it "does not change behavior other than changing log messages". Later analysis of the diff found that it moves the method call inside the code that catches exceptions, and that move is what fixes the crash on a malformed request. Bug page

A cancellation fix that covered one more case than it said (2026). Commit 0174450ca2e9 (2026-01-22) describes the case of a call cancelled while running. Its "canceled before it executes" branch also fixes a variant reported later in bitcoin#35545, as explained on the PR. Bug page

An exceptions cleanup that fixed a race (2025). Commit 56fff76f940b (2025-06-11), "Improve IPC client disconnected exceptions", presents itself as making the errors for a closed connection consistent. Later analysis found it also fixes a race: it moves the connection check from the calling thread onto the event loop thread, where it cannot interleave with disconnect handling. Bug page

A mechanical refactor that changed behavior (2025). Commit 315ff537fb65 (2025-06-11) is titled and described as a refactor that gives each proxy its own event loop pointer. It changes behavior in exactly the case of a server object that outlives its connection, and the later revert of the old workaround, c6f7fdf17350 (2025-06-30), names it as what made the workaround unnecessary. Bug page

A shutdown fix that carried a second fix (2025). Commit 7f65aac78b95 (2025-08-04) describes only the shutdown hang it fixes. The June 2025 change that fixed a segfault at shutdown, by closing IPC connections before validation state is torn down, was squashed into it. Bug page

A patch that grew after its description was written (2026). The description of bitcoin#35796 (opened 2026-07-25) describes a patch "based on the EINVAL handling". After review asked why the rest of the upstream change was left out, the patch was replaced with the full upstream Cap'n Proto commit, which also handles a second macOS case. The description was not updated. Bug page

A disconnect handler that could still run after its connection was gone (2019). Commit c685fa9bedaf (2019-08-06) says "onDisconnect events are now cancelled as Connection objects are destroyed". That holds until the disconnect fires. A handler already queued to run on the next turn of the event loop is not cancelled, so a connection destroyed locally in that turn can still be deleted twice. The remaining window was found in 2026 and is fixed in the open mp#361. Bug page · the 2026 bug

Which change introduced a bug

When Ctrl-C stopped waking mining waiters (2025). The fix commit c25a5e670b27 (2025-10-01) dates the behavior from bitcoin#30409 and says bitcoin#30967 moved it "without changing behavior". Later analysis of the code found that the bitcoin#30409 version woke waiters on every shutdown path, including Ctrl-C, and that bitcoin#30967 (5ca28ef28bcc) moved the wake-up to a path that the signal handler bypasses. That change introduced the hang. Bug page

A refactor that made block templates shared mutable state (2025). Commit e6c2f4ce7a84 (2025-05-13) is titled "interfaces: refactor: move submitSolution implementation to miner". Later analysis found it was not behavior-neutral: the old code modified a copy of the template's block, and the new code modifies the stored block. That created a race between submitSolution and the template's getters, and made a stale-check bug reachable. Bug page · the stale-check bug

A bug described as always present (2026). The fix commit bb473690c97c (2026-08-04) lists a leak on a failed client setup among "two bugs that have always been present". Later analysis found the leak itself is old, but the hang it caused in the new test existed only on the pull request's branch, because of how that branch ordered setup and disconnect handling. Bug page