Skip to content

SERVER-130777: fix(index-builds): persist side-write multikey state for resumable index builds - #1821

Open
Abuhaithem wants to merge 1 commit into
mongodb:masterfrom
Abuhaithem:fix/index-builds-resumable-side-writes-multikey
Open

SERVER-130777: fix(index-builds): persist side-write multikey state for resumable index builds#1821
Abuhaithem wants to merge 1 commit into
mongodb:masterfrom
Abuhaithem:fix/index-builds-resumable-side-writes-multikey

Conversation

@Abuhaithem

Copy link
Copy Markdown

Fixes: SERVER-130777

Fixes a resumable index build defect that can commit an index which contains multikey entries
but is not flagged multikey
, causing the query planner to silently return incorrect results.

A hybrid index build tracks the multikey information generated by concurrent (side) writes only in
memory, in IndexBuildInterceptor::_multikeyPaths; the side writes table rows store bare index
keys. The resume state persisted on clean shutdown captures multikey state only from the bulk
builder
— i.e. only what the collection scan observed:

indexStateInfo.setIsMultikey(index.bulk->isMultikey());
for (const auto& multikeyPath : index.bulk->getMultikeyPaths()) { ... }

After a restart the interceptor is reconstructed empty, and re-draining the side writes table
cannot recover the information — applyIndexBuildSideWrite() applies one bare key per row with an
empty MultikeyPaths{}, so shouldMarkIndexAsMultikey(1, {}, {}) never fires. At commit,
setMultikey() is never called for those writes.

Repro shape: scalar-only collection → build passes the collection scan → client inserts
{a: [3, 4]} (captured only by the interceptor) → clean shutdown → resume → commit. The committed
index reports isMultiKey: false, find({a: {$gte: 4, $lte: 3}}) intersects bounds to an empty
interval and misses the array document, and validate({full: true}) fails.

Existing resumable tests miss this because the harness inserts its "side writes" before the
collection scan and waits for them to majority-commit, so the scan also indexes them and the bulk
builder's multikey state masks the gap. The adjacent fix SERVER-127943 restored the bulk builder's
wildcard metadata flag on resume but did not touch the interceptor path.

Changes

  • multi_index_block.cpp_buildIndexStateInfo() now folds the interceptor's recorded
    multikey paths into the persisted isMultikey/multikeyPaths. On resume the merged state is
    restored into the bulk builder by the existing resume constructor, and the existing commit path
    marks the index multikey.
  • index_build_block.h — adds a const getIndexBuildInterceptor() accessor (the block
    already owns the interceptor).
  • multi_index_block_test.cpp — new regression test
    ResumeStateIncludesMultikeyFromSideWrites; findPersistedResumeState() gains an optional out
    parameter returning the parsed ResumeIndexInfo.
  • jstests/noPassthrough/index_builds/resumable_index_build_multikey_from_side_writes.js
    new end-to-end regression test covering shutdown/resume through a real mongod restart.

Why it's correct

  • BulkBuilderImpl's resume constructor already restores isMultikey/multikeyPaths from
    IndexStateInfo, and MultiIndexBlock::commit() already calls setMultikey() whenever the
    bulk builder reports multikey — including for drain-writes-phase resumes. No new restore code.
  • The change can only widen multikey state. A conservatively-multikey index is always correct
    (the planner is merely less aggressive); the pre-fix behavior is incorrect.
  • Wildcard interaction is safe: side-write-driven isMultikey can enable the resume-derived
    _hasMultiKeyMetadataKeys, which only gates deduplication of adjacent metadata keys — a no-op
    when there are none.
  • No IDL, wire-protocol, or on-disk format change; the persisted fields already exist
    (SERVER-50899), so downgraded binaries still parse the state document.

Testing

  • bazel test //src/mongo/db/index_builds:multi_index_block_test
    • New: ResumeStateIncludesMultikeyFromSideWrites — asserts the persisted resume state carries
      isMultikey: true and path component 0 for a side-written array document; fails without the
      fix.
  • python buildscripts/resmoke.py run --suites=no_passthrough jstests/noPassthrough/index_builds/resumable_index_build_multikey_from_side_writes.js
    • Asserts isMultiKey: true in explain, correct results for {a: {$gte: 4, $lte: 3}}, and a
      clean validate({full: true}) after resume; fails all three without the fix.
  • Existing resumable_index_build_* suites continue to cover the bulk-builder multikey
    round-trip, including wildcard variants.

Risk

Low. One function on the shutdown-persist path plus a const accessor; failure direction is
strictly conservative; no format changes.

…dex builds

A hybrid index build records the multikey information generated by
concurrent writes only in memory in the IndexBuildInterceptor; the rows in
the side writes table store bare index keys. The resume state persisted on
clean shutdown captures multikey information only from the bulk builder
(i.e. what the collection scan observed), so multikey state recorded by
the interceptor is silently dropped across a restart. Re-draining the side
writes table cannot reconstruct it: each row is applied with an empty set
of multikey paths. The resumed build then commits an index that contains
multikey entries but is not flagged multikey, so the query planner builds
bounds and covered plans that are only legal for non-multikey indexes,
silently returning incorrect results, and validate reports the collection
as invalid.

Fold the interceptor's multikey paths into the multikey state that is
already persisted in IndexStateInfo. On resume the merged state is
restored into the bulk builder, and the existing commit path marks the
index multikey. The change can only widen the multikey state, which is the
always-safe direction, and requires no on-disk format change.
@AlexTalks
AlexTalks self-requested a review July 14, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant