build the bundled onetbb as a shared library on windows by kevinushey · Pull Request #274 · RcppCore/RcppParallel

kevinushey · 2026-07-28T07:43:40Z

Makes Windows use the bundled oneTBB, built as a shared library and linked the
same way as on every other platform. Net -310 lines.

Why

Windows was the only platform where TBB wasn't a shared library loaded at
runtime, and nearly every Windows-specific wart traced back to that. Rtools
ships static libraries only, so adopting its TBB meant linking it into
RcppParallel.dll. Two consequences followed:

The TBB version, and ABI, became a property of the user's toolchain -- Intel
TBB 2017 on Rtools42, oneTBB later. That mismatch is what broke rstan.
Downstream packages had no TBB library to link against.

The workarounds accumulated: re-exporting the whole tbbmalloc archive so the
allocator could be resolved (#262), a tbb.dll stub so packages linking
-ltbb could load (#249, #250), building that stub differently for legacy
toolchains (#258), offering it from RcppParallelLibs() so rstan could link at
all (#269), and a library lookup that searched for archives never installed with
the package (#270).

And the stub was not a thin shim. It linked the TBB archives itself, so it was a
second complete copy of the oneTBB scheduler. Measured on a released install,
both it and RcppParallel.dll export the same 81 tbb::detail::r1:: entry
points; both are loaded in every session; which one a downstream package binds to
is up to the linker. Two schedulers with separate thread pools and separate
global state is a correctness problem, not a linking inconvenience.

What this removes

the Rtools TBB detection in configure.R, and with it the TBB_NAME
derivation from archive names and the tbb12 special case
the --whole-archive tbbmalloc re-export, which existed only because there
was no DLL to link
BUILD_SHARED_LIBS=0 for Windows
src/tbb-compat/ and buildTbbStub() entirely, along with the
#ifndef _WIN32 guard in observer_proxy.cpp and the
__TBB_LEGACY_TASK_SCHEDULER_OBSERVER_PROVIDED macro that let the stub tell
which headers it was compiling against
the Windows branches in .install.libs(), loadTbbLibrary(), tbbLdFlags()
and tbbLibraryPath(), and the Windows-first load order in .onLoad()

The parts that aren't just deletions

Library naming. mingw CMake would name a shared build libtbb.dll. Patch
PREFIX to "" so the runtime is tbb.dll -- the name binaries built
against 5.1.11 and earlier import. The import library keeps its lib prefix,
so -ltbb still resolves it. The upstream binary-version suffix is dropped
for the same reason.
The legacy ABI symbol. task_scheduler_observer_v3::observe now applies
on Windows too. mingw has no .def file, so exports come from dllexport
directives: the declaration gains TBB_EXPORT, which expands to nothing when
consuming the headers. Verified on macOS that the symbol is still exported
from libtbb.dylib.
Load order. .onLoad() loads tbb before RcppParallel on every platform
now, since RcppParallel.dll imports from tbb.dll and the package library
directory is not on the DLL search path.
-lRcppParallel stays on Windows. isProcessForkedChild is compiled into
RcppParallel.dll, and Windows has no lazy binding to resolve it at load
time. So RcppParallelLibs() emits -lRcppParallel -ltbb -ltbbmalloc there.

Downstream-visible

RcppParallelLibs() gains -ltbb -ltbbmalloc on Windows. Packages that
resolved TBB symbols out of RcppParallel.dll, or the tbbmalloc API via
-lRcppParallel alone, should rebuild against the current flags.

Verification

The CI additions from #272 are what validate this, and the downstream check now
asserts the property the whole change is for: the downstream library's TBB
symbols come from exactly one module, that module is tbb.dll, and
RcppParallel.dll imports from it rather than carrying its own copy.

Locally (macOS, where behaviour should be unchanged): full install, all 12 test
files pass, CxxFlags() and RcppParallelLibs() byte-identical to master, the
same three libtbb*.dylib installed, and the legacy observe symbol still
exported. All five oneTBB patches reverse-apply against the vendored tree;
patches/legacy_tbb_abi.diff and patches/unversioned_libraries.diff were
regenerated from a reconstructed pristine upstream.

What needs CI

Everything Windows, none of which is testable locally:

Does oneTBB build shared under mingw? Only static has been tried here.
It is upstream's supported configuration -- static emits "highly discouraged
... not supported" -- so I expect this to be the easier direction, but it is
unproven.
Does PREFIX "" actually yield tbb.dll + libtbb.dll.a, and does
-ltbb resolve against the import library?
Does windows-11-arm have cmake in its Rtools? If not it falls back to
tinythread, which would be a regression there and would need addressing
before merge. (Rtools42 does ship cmake -- confirmed from a CI cache dump:
C:/rtools42/x86_64-w64-mingw32.static.posix/bin/cmake.exe.)
Does RcppParallel.dll link against tbb.dll cleanly, and does the load
order in .onLoad() work?

Windows was the only platform where TBB was not a shared library loaded at runtime, and nearly every Windows-specific wart traced back to that. Use the bundled oneTBB there too, built shared, shipped as tbb.dll / tbbmalloc.dll and linked exactly as on other platforms. Rtools ships static libraries only, so adopting its TBB meant linking it into RcppParallel.dll. That made the TBB version -- and ABI -- a property of the user's toolchain (Intel TBB 2017 on Rtools42, oneTBB later), and left downstream packages with no TBB library to link against. The workarounds accumulated: re-exporting the whole tbbmalloc archive so the allocator could be resolved (#262), a tbb.dll stub so packages linking -ltbb could load (#249, #250), building that stub differently for legacy toolchains (#258), offering it from RcppParallelLibs() so rstan could link at all (#269), and a library lookup that searched for archives never installed with the package (#270). Worse, the stub linked the TBB archives itself, so it was a second complete copy of the oneTBB scheduler. Both it and RcppParallel.dll export the same 81 tbb::detail::r1:: entry points, both are loaded in every session, and which one a downstream package binds to is up to the linker. Two schedulers with separate thread pools and separate global state is a correctness problem, not a linking inconvenience. This removes: - the Rtools TBB detection in configure.R, and with it the TBB_NAME derivation from archive names and the tbb12 special case - the --whole-archive tbbmalloc re-export, which only existed because there was no DLL to link - BUILD_SHARED_LIBS=0 for Windows - src/tbb-compat/ and buildTbbStub() entirely, along with the #ifndef _WIN32 guard in observer_proxy.cpp and the __TBB_LEGACY_..._PROVIDED macro that let the stub tell which headers it was compiling against - the Windows branches in .install.libs(), loadTbbLibrary(), tbbLdFlags() and tbbLibraryPath(), and the Windows-first load order in .onLoad() Notes on the pieces that are not just deletions: - mingw CMake would name a shared build libtbb.dll; patch PREFIX to "" so the runtime is tbb.dll, the name binaries built against 5.1.11 and earlier import. The import library keeps its lib prefix, so -ltbb still resolves. - the legacy task_scheduler_observer_v3::observe definition now applies on Windows too. mingw has no .def file, so its export comes from a dllexport directive: the declaration gains TBB_EXPORT, which is empty when consuming the headers. - .onLoad loads tbb before RcppParallel on every platform now, since RcppParallel.dll imports from tbb.dll and the package library directory is not on the DLL search path. - tbbLdFlags() keeps -lRcppParallel on Windows: isProcessForkedChild is compiled into RcppParallel.dll, and Windows has no lazy binding to resolve it at load time. The downstream CI check now asserts the property this is all for: the downstream library's TBB symbols come from exactly one module, that module is tbb.dll, and RcppParallel.dll imports from it rather than carrying its own copy.

Two failures from the first CI run, both only reachable now that TBB is linked as a shared library on Windows -- a static build has no link step. Rtools42 (gcc 10): the TBB link failed with undefined references to __stack_chk_fail, __strncpy_chk and __strncat_chk. TBB compiles with -fstack-protector-strong and _FORTIFY_SOURCE, and on mingw those helpers live in libssp rather than libgcc; newer toolchains pull it in implicitly, Rtools42 does not. Pass -lssp via CMAKE_SHARED_LINKER_FLAGS, mirroring the -lssp that configure.R already adds for RcppParallel's own link. aarch64 Windows: 'lld: error: unknown argument: -z'. Rtools45-aarch64 drives clang with lld, and Clang.cmake adds -Wl,-z,relro,-z,now for any non-Apple target. GNU.cmake already guards this with NOT MINGW; give Clang.cmake the equivalent guard. windows-latest (release) passed on the first run, so the shared mingw build, the PREFIX "" naming and the reordered load in .onLoad() all work.

TBB compiles with -fstack-protector-strong and _FORTIFY_SOURCE=2, and on mingw the helpers those need (__stack_chk_fail, __stack_chk_guard, __strncpy_chk, ...) live in libssp rather than libgcc. Rtools42 (gcc 10) does not pull it in implicitly, so the link fails with undefined references to them. Passing -lssp through CMAKE_SHARED_LINKER_FLAGS was not enough: those flags are emitted before the objects, and GNU ld only resolves symbols that are already undefined when it reaches a library, so it was discarded. tbb and tbbmalloc happened to survive that because tbb_handle_ipo gives them LTO; tbbmalloc_proxy has no such call and failed to link. Putting ssp in TBB_COMMON_LINK_LIBS routes it through target_link_libraries instead, which lands after the objects, and covers all three targets.

Two reasons it was never actually running, both visible in the CI log even though the job passed: - the path was built with paste0("libs", .Platform$r_arch), giving 'libsx64' rather than 'libs/x64'. system.file() returned "" for that, so objdump was handed '/RcppParallel.dll' and reported "No such file". Use archSystemFile(), which the package already has for this, and fail loudly if the library still can't be found. - on aarch64 the runner has an x86_64 objdump from C:/mingw64 ahead of Rtools' own on the PATH, and it cannot read an aarch64 PE, so both halves of the check skipped there. Try the objdump sitting beside the compiler first, and fall back through the remaining candidates.

The previous attempt looked beside Sys.which("gcc"/"clang"), but on the aarch64 runner those resolve to C:/mingw64 -- the same x86_64 toolchain that leads the PATH -- so both candidates were the one objdump that cannot read an aarch64 PE, and the check still skipped there. They were also not deduplicated, differing only in slash direction. 'R CMD config CC' names the compiler that produced the library, so its objdump can read it by construction. Try that first, and normalize the candidates so one objdump is not tried under two spellings.

The Windows branch of tbbLdFlags() named the TBB libraries unconditionally, dropping the file.exists() guard the previous stub-based code had. On a build that fell back to tinythread -- configure.R sets TBB_ENABLED = FALSE only there, when cmake is missing and TBB_LIB is unset -- RcppParallelLibs() then emitted -L<libs> -lRcppParallel -L<lib> -ltbb -ltbbmalloc for libraries that were never built or shipped, so a downstream package would fail to link with "cannot find -ltbb". CxxFlags() correctly reports -DRCPP_PARALLEL_USE_TBB=0 in that configuration, so the package is not even using TBB. Name them only when TBB is enabled and the library actually resolves. Also drop an unverified aside from NEWS: Rtools42 and Rtools45 are confirmed to ship cmake, Rtools40 was not checked.

kevinushey added 6 commits July 28, 2026 00:43

kevinushey merged commit fdd181e into master Jul 28, 2026
9 checks passed

kevinushey mentioned this pull request Jul 28, 2026

require cmake on all platforms; tbb is always enabled #275

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

build the bundled onetbb as a shared library on windows - #274

build the bundled onetbb as a shared library on windows#274
kevinushey merged 6 commits into
masterfrom
feature/windows-shared-tbb

kevinushey commented Jul 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

kevinushey commented Jul 28, 2026

Why

What this removes

The parts that aren't just deletions

Downstream-visible

Verification

What needs CI

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant