Replace jackson OTLP json serialization with handrolled version

[jack-berg]

Related to #8533.

Jackson 3 was released and requires java 17.

We rely on jackson in a variety of places. This PR concerns the serialization functionality we rely on for opentelemetry-exporter-logging-otlp, opentelemetry-exporter-otlp (officially json is not supproted - see #5833).

Rather than try to accommodate different versions of jackson through something like SPI, its simpler to just replace jackson with a hand rolled version, at least for the JSON serialization bits.

We have precedent for hand rolled JSON serialization in JsonEncoding, which is used by Value. However, this operates in very different context than OTLP marshaling and probably shouldn't be merged together.

[codecov]

Codecov Report

❌ Patch coverage is 96.98276% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.02%. Comparing base (0225a99) to head (3ac38d9).
⚠️ Report is 13 commits behind head on main.

Files with missing lines	Patch %	Lines
...exporter/internal/marshal/JsonBufferedEncoder.java	97.72%	2 Missing and 2 partials ⚠️
...etry/exporter/internal/marshal/JsonSerializer.java	93.75%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8545      +/-   ##
============================================
+ Coverage     90.97%   91.02%   +0.05%     
- Complexity    10206    10256      +50     
============================================
  Files          1013     1013              
  Lines         27166    27310     +144     
  Branches       3182     3200      +18     
============================================
+ Hits          24713    24860     +147     
+ Misses         1729     1725       -4     
- Partials        724      725       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[brunobat]

Well... I think we have been working on the same :)
See: #8546
I have a bunch of tests in there if you want to use them.

[jack-berg]

Well... I think we have been working on the same :)

Ah! Too bad 😅. I checked out your code and added a little more test coverage mine was missing.

I like the look of this though.

This shouldn't be the last location of the JSON classes. They need to be placed on a SDK common lib to allow consumption by Declarative Config.

Not sure declarative config can easily go in this direction. It needs to do a couple of things which are harder than JSON serialization:

• Parse YAML
• Convert from generic Object to POJO

It actually doesn't do any serialization at all. So I think the serialization code could actually live in opentelemetry-exporter-common long term.

[opentelemetry-pr-dashboard]

This PR has review comments. Review suggestions, whether from maintainers or automated reviewers, aren't always correct or required. Please evaluate each comment on its merits, then make sure each thread has a clear outcome.

For example, link to the commit if you applied a suggestion, explain why it wasn't applied, or ask a follow-up question.

Automation flags a PR for human review once every review thread has a reply or is marked as resolved.

Status across open PRs is visible on the pull request dashboard.

[brunobat]

ok. Closing my PRs.

[zeitlinger]

I think the current tests cover the main behavior well, but since this is low-level serialization code and it adds a fair amount of custom logic, I’d feel better with a few more edge-case tests before merging.

In particular, I think it would be good to add coverage for:

• buffer flushing once output exceeds the internal 4096-byte buffer
• nesting deep enough to grow the counts array past 16
• invalid surrogate handling in writeQuoted() (lone low surrogate / unmatched high surrogate)
• the close() IOException path

The happy-path coverage looks solid already; I’m mainly asking for the edge cases around the new custom writer.

(generated by claude)

[breedx-splk]

There's a long-standing challenge in json marshaling with self-referential objects. Fortunately, this PR doesn't introduce generic object serialization, so you can't just pass any old Object into it. That helps. But I did fine one very edge-casey thing: It's possible to create self-referential marshaler demonstrate a change in behavior with this PR.

ProtoFieldInfo fieldInfo = ProtoFieldInfo.create(1, 0, "self");
Marshaler marshaler =
    new Marshaler() {
      @Override
      public int getBinarySerializedSize() {
        return 0;
      }

      @Override
      protected void writeTo(Serializer output) throws IOException {
        output.serializeMessage(fieldInfo, this);
      }
    };
marshaler.writeJsonTo(new ByteArrayOutputStream());

That will end up throwing a StackOverflowError with this PR, whereas Jackson has an explicit check for a certain depth and gives up by throwing some Jackson-specific exception which is rooted in IOException. Note the difference between Error and Exception here.

Normally I'd say that this change in behavior is a real problem, but I can't find any genuine production code paths to attain this...so maybe it's not a big deal. I just thought I'd bring it up in case others feel differently.

[breedx-splk]

I'd like to see another test that just passes nulls for each of the types too.

[jack-berg]

We've got nullaway protecting against this. If there is some way for null to reach here, I would agree.

[breedx-splk]

One small name change could help better convey that this is only called for values, and not for field names or other possible uses. Maybe it's not necessary given the argument name, just an idea.

Suggested change

	void writeString(String value) throws IOException {
	void writeStringValue(String value) throws IOException {

[jack-berg]

There are several other methods which write values and do not include the *Value suffix. I think I'd like to skip this one.

[breedx-splk]

This is great! I'm basically an approval, will just give a little time for response to comments.

[jack-berg]

There's a long-standing challenge in json marshaling with self-referential objects.

I did find a way to create one of these situations with Value. I added fixed depth limit of 100 to mitigate, and corresponding tests to verify. Jackson has a default max depth of 1000 in v2, and 500 and v3. I went with 100 here. Its easier to loosen restrictions later than it is to clamp them down.

[jack-berg]

buffer flushing once output exceeds the internal 4096-byte buffer
nesting deep enough to grow the counts array past 16
invalid surrogate handling in writeQuoted() (lone low surrogate / unmatched high surrogate)
the close() IOException path

Tests added for all!