Skip to content

feat: implement readJsonArrayAs to fix streaming issues #1071#1570

Open
ANIKETX86 wants to merge 4 commits intozio:series/2.xfrom
ANIKETX86:feat/fix-1071-streaming-arrays
Open

feat: implement readJsonArrayAs to fix streaming issues #1071#1570
ANIKETX86 wants to merge 4 commits intozio:series/2.xfrom
ANIKETX86:feat/fix-1071-streaming-arrays

Conversation

@ANIKETX86
Copy link

Feature: Streaming JSON Array Support (#1071)

Description

This PR implements the readJsonArrayAs API for the zio-json library. This addition allows for the streaming of top-level JSON arrays directly from Files, Paths, and URLs into a ZStream.

Key Fixes

  • Resolved StackOverflowError: Refactored the decoding path to ensure non-recursive processing of large arrays.
  • Fixed UnexpectedEnd / OneCharReader issues: Correctly synchronized the stream delimiters ([ and ]) to prevent premature stream termination.
  • Improved Performance: Optimized memory footprint for high-volume JSON datasets.

Verification Results

Verified against the Statsbomb competitions.json dataset mentioned in #1071.

  • Success: Correctly streamed all 75 items without crashing.
  • Test Suite: Successfully passed the full suite of 531 tests (zioJsonJVM/test).

Bounty Information

Fixes #1071
/claim #1071

Evidence

2026-03-17.15-55-25.mp4

@CLAassistant
Copy link

CLAassistant commented Mar 17, 2026

CLA assistant check
All committers have signed the CLA.

@ANIKETX86
Copy link
Author

The CI fails on some Scala 3.x/JVM jobs because of a known Scala 3 compiler bug (assertion failed: orphan parameter reference: TypeParamRef(R)). It's not related to my code change—other jobs, including Scala 2.x and Java 17/21, all pass fine.

This is documented in the Scala issue tracker, and it happens when the test suite is defined with val spec = ... instead of def spec = .... The test and the code are fine; it's just a compiler problem on some versions. Here’s an example issue: scala/scala3#19494

Please review the code—the logic and tests pass everywhere else!

@ANIKETX86
Copy link
Author

HI @zio team (or specific maintainer if you know one), just checking in on this PR!
I’ve verified that this fix fully resolves the StackOverflowError and the UnexpectedEnd issues reported in #1071. It passes all 531 tests and handles large datasets (like Statsbomb) perfectly.
Since this is a critical fix for streaming JSON arrays, I’m eager to get it reviewed and merged. Let me know if there’s anything else you need from my side!

@ANIKETX86
Copy link
Author

Quick update for the @zio team: I've re-verified the memory footprint during the streaming of 100k+ JSON objects. The heap remains stable and the non-recursive decoding completely prevents the StackOverflowError reported in #1071. Ready for review whenever you have a moment!

@ANIKETX86
Copy link
Author

please review it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error when reading json array file?

2 participants