SERVER-130777: fix(index-builds): persist side-write multikey state for resumable index builds - #1821
Open
Abuhaithem wants to merge 1 commit into
Conversation
…dex builds A hybrid index build records the multikey information generated by concurrent writes only in memory in the IndexBuildInterceptor; the rows in the side writes table store bare index keys. The resume state persisted on clean shutdown captures multikey information only from the bulk builder (i.e. what the collection scan observed), so multikey state recorded by the interceptor is silently dropped across a restart. Re-draining the side writes table cannot reconstruct it: each row is applied with an empty set of multikey paths. The resumed build then commits an index that contains multikey entries but is not flagged multikey, so the query planner builds bounds and covered plans that are only legal for non-multikey indexes, silently returning incorrect results, and validate reports the collection as invalid. Fold the interceptor's multikey paths into the multikey state that is already persisted in IndexStateInfo. On resume the merged state is restored into the bulk builder, and the existing commit path marks the index multikey. The change can only widen the multikey state, which is the always-safe direction, and requires no on-disk format change.
AlexTalks
self-requested a review
July 14, 2026 21:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: SERVER-130777
Fixes a resumable index build defect that can commit an index which contains multikey entries
but is not flagged multikey, causing the query planner to silently return incorrect results.
A hybrid index build tracks the multikey information generated by concurrent (side) writes only in
memory, in
IndexBuildInterceptor::_multikeyPaths; the side writes table rows store bare indexkeys. The resume state persisted on clean shutdown captures multikey state only from the bulk
builder — i.e. only what the collection scan observed:
After a restart the interceptor is reconstructed empty, and re-draining the side writes table
cannot recover the information —
applyIndexBuildSideWrite()applies one bare key per row with anempty
MultikeyPaths{}, soshouldMarkIndexAsMultikey(1, {}, {})never fires. At commit,setMultikey()is never called for those writes.Repro shape: scalar-only collection → build passes the collection scan → client inserts
{a: [3, 4]}(captured only by the interceptor) → clean shutdown → resume → commit. The committedindex reports
isMultiKey: false,find({a: {$gte: 4, $lte: 3}})intersects bounds to an emptyinterval and misses the array document, and
validate({full: true})fails.Existing resumable tests miss this because the harness inserts its "side writes" before the
collection scan and waits for them to majority-commit, so the scan also indexes them and the bulk
builder's multikey state masks the gap. The adjacent fix SERVER-127943 restored the bulk builder's
wildcard metadata flag on resume but did not touch the interceptor path.
Changes
multi_index_block.cpp—_buildIndexStateInfo()now folds the interceptor's recordedmultikey paths into the persisted
isMultikey/multikeyPaths. On resume the merged state isrestored into the bulk builder by the existing resume constructor, and the existing commit path
marks the index multikey.
index_build_block.h— adds a constgetIndexBuildInterceptor()accessor (the blockalready owns the interceptor).
multi_index_block_test.cpp— new regression testResumeStateIncludesMultikeyFromSideWrites;findPersistedResumeState()gains an optional outparameter returning the parsed
ResumeIndexInfo.jstests/noPassthrough/index_builds/resumable_index_build_multikey_from_side_writes.js—new end-to-end regression test covering shutdown/resume through a real
mongodrestart.Why it's correct
BulkBuilderImpl's resume constructor already restoresisMultikey/multikeyPathsfromIndexStateInfo, andMultiIndexBlock::commit()already callssetMultikey()whenever thebulk builder reports multikey — including for drain-writes-phase resumes. No new restore code.
(the planner is merely less aggressive); the pre-fix behavior is incorrect.
isMultikeycan enable the resume-derived_hasMultiKeyMetadataKeys, which only gates deduplication of adjacent metadata keys — a no-opwhen there are none.
(SERVER-50899), so downgraded binaries still parse the state document.
Testing
bazel test //src/mongo/db/index_builds:multi_index_block_testResumeStateIncludesMultikeyFromSideWrites— asserts the persisted resume state carriesisMultikey: trueand path component0for a side-written array document; fails without thefix.
python buildscripts/resmoke.py run --suites=no_passthrough jstests/noPassthrough/index_builds/resumable_index_build_multikey_from_side_writes.jsisMultiKey: trueinexplain, correct results for{a: {$gte: 4, $lte: 3}}, and aclean
validate({full: true})after resume; fails all three without the fix.resumable_index_build_*suites continue to cover the bulk-builder multikeyround-trip, including wildcard variants.
Risk
Low. One function on the shutdown-persist path plus a const accessor; failure direction is
strictly conservative; no format changes.