IPC stability bugs

Bug 71

IPC code built on NetBSD with GCC before 14 has exception handling compiled out

Status
Worked around
Symptom
Crash; worst effect seen: crash
Where seen
Bitcoin Core functional tests, libmultiprocess tests (mptest), a CI build
Trigger
Normal use; no special event is needed
First reported
2025-08-23 (first report)
Fixed
2026-08-03
Fix
mp#196, mp#321

Summary

On NetBSD with GCC older than 14, Cap'n Proto's kj/common.h mistakes a stub __has_feature macro in <sys/cdefs.h> for "exceptions disabled", so code built there has its exception handling compiled out. With the pkgsrc capnproto package this made bitcoin-node terminate whenever an IPC server method threw (the client sees "Peer disconnected" instead of an error reply), made mptest abort, and, with mixed compilers, made mptest fail to link. The defect is a build-time misdetection in an upstream header; nothing in libmultiprocess or Bitcoin Core code is wrong, and it affects only people building IPC code on NetBSD with an older GCC. libmultiprocess CI works around it in its NetBSD jobs (mp#196, which added them, and mp#321, which moved them to NetBSD 11.0); the upstream fix (capnproto#2756) is not in a capnproto release, and compile-time checks are proposed in mp#364 and bitcoin#36060.

What happened

Root cause

kj/common.h tests #if defined(__has_feature) first and, when the macro exists, sets KJ_NO_EXCEPTIONS if !__has_feature(cxx_exceptions); only a compiler without the macro falls through to the __GNUC__/__EXCEPTIONS test (capnproto 1.2.0 and 1.4.0). NetBSD's <sys/cdefs.h> defines a dummy __has_feature(x) 0 for compilers that lack it natively, which is GCC older than 14. So every translation unit that GCC 10 or 12 compiles with kj/common.h on NetBSD is built as if exceptions were disabled.

Fix

No fix in libmultiprocess or Bitcoin Core code; the upstream fix, capnproto#2756, is not yet in a capnproto release or in NetBSD's pkgsrc package. libmultiprocess CI works around it:

Open proposals: mp#364 (ryanofsky) adds an #error so a build with KJ_NO_EXCEPTIONS set fails at compile time instead of crashing at runtime, and bitcoin#36060 (@151henry151, draft) adds a configure-time check.

Timeline

Lessons

What would have prevented this bug, found it sooner, or made it easier to debug. The patterns page groups these lessons across all bugs.

Pattern CI, toolchain and platform: A feature macro detected wrongly

  • Prevented by nothing practical
  • Earliest detector an assertion (did not exist then)
  • Diagnosis hard
  • Fix design easy
  • Debugging aids logging, CI artifacts
  • Rust n/a

Would Rust have prevented it? Not applicable. A feature macro misdetected by a header on NetBSD is a build and toolchain problem in which no libmultiprocess code is wrong. More on Rust

Could it have been prevented?

Not by this project. The defect is in Cap'n Proto's kj/common.h, which trusts a __has_feature(cxx_exceptions) answer of zero without checking that the macro is the compiler's own; NetBSD's <sys/cdefs.h> stubs __has_feature to 0 for GCC older than 14, so every unit built there has exception handling compiled out. Nothing in libmultiprocess or Bitcoin Core code is wrong, and the upstream fix (capnproto#2756) is not in a release. What the project can do is refuse to build in that state: libmultiprocess depends on serverInvoke rethrowing into the RPC layer and on kj::runCatchingExceptions actually catching, so an #error when KJ_NO_EXCEPTIONS is set, which mp#364 proposes, turns a runtime abort on every error path into a build failure with a message. That assumption ("the library is built with exceptions") was never written down because nobody imagined a build where it was false.

Could it have been found sooner?

The link-error form was found the first day anyone built on NetBSD (mp#196, 2025-08-23) and ryanofsky named KJ_NO_EXCEPTIONS the same day, so detection was not the problem; the runtime form was hidden for eleven months by the workaround itself, since pinning GCC 14 for libmultiprocess and building capnproto with -DKJ_NO_EXCEPTIONS=0 removed the mismatch from the only NetBSD job. The runtime form surfaced twice on the morning of 2026-07-27, in hebasto's nightly Bitcoin Core run against the pkgsrc package (bitcoin#35823) and in the mp#321 job once it dropped the pin. A compile-time check would have caught both forms at configure time in 2025; a CI job that builds the library the way a distribution package does (default compiler, prebuilt capnproto) is what actually did.

What made it hard to debug?

The two forms looked unrelated: undefined references to kj::getCaughtExceptionAsKj() in 2025, then "Peer disconnected" where a remote-exception reply was expected and a terminate called after throwing abort in 2026, and they were only tied together on 2026-08-01. hebasto's December diagnosis of the header logic was correct but its premise was questioned as late as September 2026, because pkgsrc passes no -fno-exceptions and nothing visible in the build explained why exceptions would be off; the <sys/cdefs.h> stub that completes the story was found by reading NetBSD's system header. The node log in bitcoin#36058, showing mp::serverInvoke catch and rethrow followed by libstdc++'s terminate message, was what proved the exception escaped above the library rather than being mishandled in it. A preprocessor-level diagnostic (#error or a configure check printing the detected exception mode) would have replaced all of that with one line.

Lesson

When a library's correctness depends on a build-configuration assumption such as "exceptions are enabled", assert it at compile time; a feature-detection macro can be wrong on a platform you have not built on, and the runtime symptom of a silently disabled catch block is a crash far from the cause.

Notes

Classification

The survey's classification of this bug. The explorer charts these fields across all bugs; the method page explains them.

Field Value
Symptom crash
Worst effect observed crash
Seen in functional-test, mptest, ci-build
Scope build
Trigger Normal use (ordinary-path)
Phase any
Side both
Component capnproto
Root cause kind unhandled-exception
Fix kind config
Detection ci-deterministic
Needs a peer that is not libmultiprocess C++ no
Needed a sanitizer to see no
Confidence medium