Skip to content

Add msgpack-jackson3 module for Jackson 3.x support - #987

Open
komamitsu wants to merge 4 commits into
msgpack:mainfrom
komamitsu:support-jackson3
Open

Add msgpack-jackson3 module for Jackson 3.x support#987
komamitsu wants to merge 4 commits into
msgpack:mainfrom
komamitsu:support-jackson3

Conversation

@komamitsu

@komamitsu komamitsu commented May 31, 2026

Copy link
Copy Markdown
Member

Summary

  • Adds a new msgpack-jackson3 submodule (jackson-dataformat-msgpack3) that ports the existing Jackson 2.x integration to Jackson 3.x (tools.jackson namespace)
  • Requires Java 17+ and Jackson 3.1.2; the module is excluded from the build on older JDKs so existing Java 8 users are unaffected
  • Includes JMH benchmarks (src/jmh/java) runnable via ./sbt "msgpack-jackson3/jmh:run"
  • Fixes Add Jackson 3 support #933

Key differences from msgpack-jackson

  • Package namespace: com.fasterxml.jacksontools.jackson
  • TokenStreamLocation replaces JsonLocation; byte offset passed as columnNr
  • Lightweight MessagePackWriteContext replaces SimpleStreamWriteContext

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new msgpack-jackson3 submodule that ports the existing Jackson 2.x MessagePack integration to Jackson 3.x (tools.jackson namespace). It is published as a separate artifact (jackson-dataformat-msgpack-jackson3), requires Java 17+, and is conditionally included in the SBT build / CI / release pipelines so existing Java 8 users of msgpack-jackson are unaffected.

Changes:

  • New msgpack-jackson3 module mirroring msgpack-jackson, adapted for Jackson 3 APIs (ObjectWriteContext/ObjectReadContext, TokenStreamLocation, ValueSerializer/ValueDeserializer, builder-style MapperBuilder/TSFBuilder, etc.) and including a lightweight MessagePackWriteContext replacement for SimpleStreamWriteContext.
  • SBT build, CI workflow, and release workflow updated to compile/test/publish the new module only on JDK 17+, plus a new sbt-jmh plugin and JMH benchmarks under src/jmh/java.
  • Numerous regression tests covering tricky edge cases (null keys, write-context state, dup-detection NPEs, root-scalar ordering, multi-root containers, writeNumber(String) NaN/Infinity, etc.).

Reviewed changes

Copilot reviewed 41 out of 42 changed files in this pull request and generated no comments.

Show a summary per file
File Description
build.sbt Adds msgpack-jackson3 project, JMH plugin, Java 17 gating via isJava17Plus.
project/plugins.sbt Adds sbt-jmh plugin.
.github/workflows/CI.yml Skips msgpack-jackson3 tests on JDKs < 17.
.github/workflows/release.yml Splits publishing so jackson3 is only published with JDK 17.
msgpack-jackson3/README.md New documentation for the Jackson 3 module.
msgpack-jackson3/src/main/java/.../MessagePackFactory.java Jackson 3 BinaryTSFactory implementation.
msgpack-jackson3/src/main/java/.../MessagePackFactoryBuilder.java New TSFBuilder for the factory.
msgpack-jackson3/src/main/java/.../MessagePackGenerator.java Port of generator; uses new MessagePackWriteContext, node-buffering model.
msgpack-jackson3/src/main/java/.../MessagePackParser.java Port of parser to ParserMinimalBase; uses MessagePackReadContext.
msgpack-jackson3/src/main/java/.../MessagePackReadContext.java Read context (replaces JsonReadContext).
msgpack-jackson3/src/main/java/.../MessagePackWriteContext.java Lightweight write context.
msgpack-jackson3/src/main/java/.../MessagePackMapper.java ObjectMapper subclass with new builder API.
msgpack-jackson3/src/main/java/.../MessagePackSerializerFactory.java Custom serializer factory installing key serializer.
msgpack-jackson3/src/main/java/.../MessagePackSerializedString.java SerializableString impl tolerant of null values.
msgpack-jackson3/src/main/java/.../MessagePackKeySerializer.java Key serializer using MessagePackSerializedString.
msgpack-jackson3/src/main/java/.../MessagePackExtensionType.java Extension type value + serializer.
msgpack-jackson3/src/main/java/.../ExtensionTypeCustomDeserializers.java Per-type extension deserializer registry.
msgpack-jackson3/src/main/java/.../TimestampExtensionModule.java Module serializing Instant as msgpack timestamp ext type.
msgpack-jackson3/src/main/java/.../JsonArrayFormat.java AnnotationIntrospector forcing ARRAY shape.
msgpack-jackson3/src/main/java/.../PackageVersion.java Hard-coded package version.
msgpack-jackson3/src/main/java/.../Tuple.java Small Tuple helper.
msgpack-jackson3/src/test/java/... Ported test suite plus new regression tests for Jackson 3 specific behavior.
msgpack-jackson3/src/jmh/java/... New JMH benchmarks (read/write/UTF-8) and model classes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@komamitsu
komamitsu requested a review from xerial May 31, 2026 14:27
@komamitsu

Copy link
Copy Markdown
Member Author

@xerial Can you take a quick look at the changes of .github/workflows/* and *.sbt?

@xerial

xerial commented Jun 16, 2026

Copy link
Copy Markdown
Member

@komamitsu Do you still need to maintain msgpack-jackson for Jackson 2.x? If we don't have critical use cases, I'll lean to dropping Jackson 2 support as supporting both Jackson2 and 3 will be overkill for msgpack-java project.

@xerial

xerial commented Jun 16, 2026

Copy link
Copy Markdown
Member

@komamitsu

Copy link
Copy Markdown
Member Author

@xerial Jackson 2 is still actively used, and Jackson 3 requires Java 17+, so not everyone can migrate yet. I think we should support both for a while.

komamitsu added 3 commits July 8, 2026 09:19
Address PR msgpack#987 feedback asking for a clear artifact-name mapping
between jackson-dataformat-msgpack (Jackson 2) and
jackson-dataformat-msgpack-jackson3 (Jackson 3) to reduce upgrade
confusion.
main migrated the build to sbt 2 (feb87ef) without sbt-jmh, since
sbt-jmh 0.4.7 has no sbt-2-compatible artifact. This branch's
msgpack-jackson3 JMH benchmarks depend on sbt-jmh, so after merging
main the plugin fails to resolve. 0.4.8 publishes an sbt_2_3 build.
@komamitsu

Copy link
Copy Markdown
Member Author

@hiSandog Good point. Addressed in 531d3d0

@komamitsu

Copy link
Copy Markdown
Member Author

@xerial I addressed the conflict, PTAL.

BTW, it seems Universal Buffer Test has been skipped since #998 was merged.

@xerial

xerial commented Jul 28, 2026

Copy link
Copy Markdown
Member

@komamitsu Thank you for this PR — the Jackson 3 port itself looks solid, and I agree with your point that we should support both Jackson 2 and 3 for a while, since Jackson 3 requires Java 17+ and many users can't migrate yet.

Before merging, I'd like to propose restructuring the module naming. My concern with jackson-dataformat-msgpack3 is long-term: if Jackson 3 takes over Jackson 2 in the next couple of years, the unmarked name jackson-dataformat-msgpack will be stuck on the legacy version, and Jackson 3 users (the growing cohort) would eventually need a second migration from -msgpack3 back to the plain name. I'd rather put the clean name on the future from day one.

Concretely, I propose we cut msgpack-java 1.0.0 with this structure:

  • jackson-dataformat-msgpack (1.x) = Jackson 3: your Jackson 3 code from this PR becomes the contents of the msgpack-jackson module. To avoid a classpath clash with the Jackson 2 artifact, the Java package moves to org.msgpack.jackson3.dataformat. Since Jackson 3 adopters are already rewriting every com.fasterxml.jackson import to tools.jackson, one more changed import costs them almost nothing — and it means both artifacts can coexist in one application for incremental migration (which was a design goal of Jackson's own tools.jackson rename).
  • jackson2-dataformat-msgpack (1.x) = Jackson 2: the current msgpack-jackson code moves here unchanged, keeping the org.msgpack.jackson.dataformat package. Jackson 2 users who want to keep receiving msgpack-core updates only need a one-line artifactId change — no import changes at all. This module would be maintenance-mode (Jackson 2.x bumps and bug fixes only), documented as such in the README, to be dropped in a future major release when Jackson 2 usage fades.
  • 0.9.x freezes: users who do nothing stay on jackson-dataformat-msgpack 0.9.10 and nothing breaks. We'll keep a 0.9.x branch as an escape hatch for critical fixes, but the plan is zero releases from it — this avoids maintaining two release lines with msgpack-core backports.

If dependabot/renovate proposes 0.9.x → 1.0.0 to a Jackson 2 user, it fails loudly at compile time (the old package is gone from that artifact), and the release notes will point to jackson2-dataformat-msgpack — so there's no silent breakage.

Since msgpack-java's API has been stable for a decade while sitting at 0.x, I think this transition is also the right occasion to finally signal 1.0.0.

Mechanically this reuses essentially all of your work — the Jackson 3 sources just land in msgpack-jackson with the package rename, and the JDK-conditional build exclusion you added carries over (the module still requires Java 17+ while msgpack-core and the Jackson 2 module stay on Java 8). The OSGi bundle symbolic names and export packages would be updated to match.

WDYT? If you're on board, I'm happy to help with the build.sbt / workflow changes for the restructuring.

@xerial

xerial commented Jul 28, 2026

Copy link
Copy Markdown
Member

I've opened #1012 implementing the restructuring proposed above, built on top of this PR's commits (so all the Jackson 3 port work is preserved and attributed). @komamitsu PTAL — if you prefer this direction we can continue the review there; if not, happy to discuss further here.

While enabling the module restructuring I also found that the Jackson 2 module's JUnit 5 tests have been silently skipped on main (no Jupiter test interface in the sbt build — likely related to the skipped Universal Buffer Test you noticed), and fixing that surfaced a small JsonLocation regression from #903. Both are fixed in #1012.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Jackson 3 support

3 participants