address review findings on the windows onetbb build - #272
Merged
Conversation
- configure: derive TBB_NAME / TBB_MALLOC_NAME whenever an Rtools tree is adopted, not only when it is oneTBB. gating both on 'oneapi' broke an explicitly configured legacy TBB_LIB, which linked -ltbb against a directory holding libtbb_static.a - configure: don't let the cmake probe throw. it now runs on Windows, where an unrunnable CMAKE or unparseable --version output has to degrade to tinythread rather than fail the configure - configure: correct the required cmake version in the error message (3.5, matching the check and DESCRIPTION), and rename useBundledTbb to buildBundledTbb so it no longer collides with the install.libs.R function - install.libs.R: report the Windows static build explicitly instead of logging 'no tbb runtime libraries found', which reads like a failure - tbb.R: note that the stub carries its own oneTBB runtime, so resolving against it means a second scheduler rather than a free fallback - downstream check: assert the built library imports nothing from the tbb.dll stub, which is the property that comment now relies on - GNU.cmake: probe with an empty file rather than the null device, which drops the WIN32 special case and removes the unverified 'gcc -c NUL' - add windows-11-arm to the downstream matrix, restore the trailing newline in observer_proxy.cpp, pin rstan.yaml against DESCRIPTION, checkout@v4 - regenerate patches/legacy_tbb_abi.diff and patches/mingw_gnu_cmake.diff
CI disproved the premise of the import assertion added in the previous commit. On both windows-latest configurations -- the Rtools45 oneTBB and the bundled build -- the downstream library takes its entire TBB surface (13 tbb::detail::r1:: entry points, including isolate_within_arena and observe) from tbb.dll, and imports nothing at all from RcppParallel.dll. R CMD SHLIB links RcppParallel.dll against an export list generated from the package's own objects, so the TBB entry points pulled out of the static library are never re-exported; '-ltbb' is therefore load-bearing, not a fallback. So assert the invariant that does hold and does matter: the TBB surface must come from exactly one module. A library split across RcppParallel.dll and the stub would register observers with one scheduler while its tasks ran in the other, which is silent, unlike the link error this replaced. Two further defects in the check itself: - an objdump that cannot read the file exited non-zero having printed no import table, which parsed as 'no imports' and passed vacuously. This is what made the windows-11-arm job green: Sys.which() finds the runner's x86_64 objdump, which cannot read an aarch64 PE. - the export table follows the import tables and lists the library's own inlined tbb::detail::d1:: instantiations. Left in, it was absorbed into the last 'DLL Name:' block and credited that library with TBB imports it never had -- harmless only because tbb.dll happened to be listed last. Also correct the tbbLdFlags() comment, which described RcppParallel.dll as re-exporting an incidental part of the runtime.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #269, which was merged before these review findings were applied.
Two are behaviour fixes; the rest tighten things #269 introduced.
Fixes
configure.R: derive the TBB library names whenever an Rtools tree is adopted.#269 gated the whole Windows block on
oneapi/being present, but that blockdoes two jobs: deciding whether to adopt the tree, and deriving
TBB_NAME/TBB_MALLOC_NAMEfrom the archive names. Gating both broke an explicitlyconfigured legacy
TBB_LIB: the names fell back totbb/tbbmalloc, soPKG_LIBSemitted-ltbbagainst a directory holdinglibtbb_static.a. The_staticalternation in the pattern exists precisely for this, and both thelegacy
pkgLibsbranch andbuildTbbStub's "re-export the static tbb librarywholesale" branch still contemplate a non-oneTBB Windows TBB. An explicitly
configured TBB is now honoured whatever its vintage; auto-discovery next to
gcc still requires oneTBB.
configure.R: don't let the cmake probe throw. The probe now runs onWindows, where the whole point is degrading to tinythread -- but
system(..., intern = TRUE)[[1L]]raises a subscript error ifCMAKEpointsat nothing runnable, and
numeric_version()raises on output it can't parse.Either turned the intended fallback into a hard configure failure. Any probe
failure is now treated as "no usable cmake": non-fatal on Windows, still a
hard error off it.
R/tbb.R+ downstream check: the-ltbbfallback isn't free. The orderingrationale is right, but the comment presented the fallback as pure upside. The
stub links
-ltbb12itself, so it embeds its own copy of the oneTBB runtime; apackage resolving some symbols from
RcppParallel.dlland the rest from thestub would straddle two independent schedulers, and an observer registered with
one would never fire for arenas owned by the other. In practice most of the
archive is pulled into
RcppParallel.dlltransitively (arena.cppreferencesobserver_list), but the failure mode changed from a link error to a silentcorrectness bug. The comment now says so, and
tbb-downstream-check.Rassertsvia
objdump -pthat nothing lands on the stub -- which also answers one of#269's own open CI questions.
GNU.cmake: probe with an empty file, not the null device.gcc -c NULwas plausible but unverified and depends on gcc accepting a device name as an
input file. A
file(WRITE ...)probe works identically everywhere and dropsthe
WIN32branch entirely.install.libs.R: don't log the Windows static build as a failure..install.libs'sshlibPatternstill matches^tbb.*\.dll$on Windows, so abundled Windows build logged
no tbb runtime libraries found in 'tbb/build/lib_release'. Benign -- the archive is linked intoRcppParallel.dll-- but it reads like a failure in exactly the logs someoneconsults when this goes wrong.
Smaller items
windows-11-armadded to thedownstreammatrix:RcppParallelLibs()changed for every Windows platform, and arm64 has both its own vendored
patch and
itt_notifycompiled out.>= 3.5, matchingthe check and
SystemRequirements).useBundledTbbtobuildBundledTbbinconfigure.R, so it nolonger collides with the
useBundledTbb()function ininstall.libs.R.observer_proxy.cpp.rstan.yamlpins againstDESCRIPTIONrather than a.9000suffix, so itsurvives development version bumps.
actions/checkout@v3->v4, and consistent${R_ARCH_BIN}in thedownstream Makevars.
patches/legacy_tbb_abi.diffandpatches/mingw_gnu_cmake.diffto match the amended sources.
Verification
configure.Rinto a mocked environment. Rtools43+ output is byte-identicalto build the bundled onetbb on windows when rtools has no onetbb #269, so its compatibility claim still holds. Confirmed against the
pre-fix code that both bugs were real: the explicit legacy
TBB_LIBcaseproduced
TBB_NAME: tbb, and an unparseable cmake version aborted configurewith
invalid version specification.R CMD INSTALLplus all 9 test files; the downstream checkpasses end-to-end (
CXX_STDconfirmed honoured)."no input files", and the unparseable-version fallback prevents the
math(EXPR)failure.objdumpoutput,including case-insensitivity and the last-block boundary.
What still needs CI
Unchanged from #269: none of the Windows behaviour is testable locally. Two
additions here can legitimately turn CI red and are the ones to watch --
the
windows-11-armdownstream job, and the import assertion, which fails ifanything genuinely does resolve from the stub. Both surface information rather
than assume it.
Note
patches/windows_arm64.diffdoes not reverse-apply (it referencessrc/tbb/build/, which is.Rbuildignored). That predates #269 and is leftalone here.