feat: add strict JSON parsing with String/CharBuffer support by Dee66 · Pull Request #974 · zio/zio-schema

Dee66 · 2025-12-31T19:36:23Z

Reject JSON input with trailing characters after valid JSON.

Fixes #712

/claim #712

Dee66 · 2025-12-31T21:12:12Z

demo.mp4

Dee66 · 2026-01-11T09:01:25Z

Hi @jdegoes, I just wanted to clear up the confusion regarding the bounty.

This work fulfills the 'Strict JSON Parsing' bounty. The active Algora claim is here: https://algora.io/claims/qd2AdUcS1Scxy7df

I originally opened this in zio-blocks #571, but moved the implementation here to zio-schema following @plokhotnyuk's suggestion that it was a better architectural fit for JsonBinaryCodec.

The logic is complete and there are no merge conflicts. It should be ready to merge or update directly from your end. Let me know if everything looks good to get this settled.

987Nabil

New decode methods have nothing to do with the issue and are not needed

fixes zio#712

Dee66 · 2026-01-24T09:54:18Z

@987Nabil

removed the methods as requested
reject JSON with trailing characters - Core fix that adds EOF checking to reject malformed JSON like {}} and "foo""
add type annotations for scala 3 - Compatibility fix adding explicit type annotations for Scala 3 implicit resolution

Dee66 · 2026-03-02T16:43:15Z

Hi @987Nabil , this code has been live for some time, please can this ticket be closed?

Copilot

Pull request overview

This PR aims to fix #712 by making JSON decoding reject inputs that contain non-whitespace trailing characters after an otherwise-valid JSON value.

Changes:

Added a new decoder test suite that asserts trailing characters cause decode failures for several JSON shapes.
Updated zioJsonBinaryCodec.decode to decode via a FastStringReader and explicitly check for leftover non-whitespace content.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File	Description
`zio-schema-json/shared/src/main/scala/zio/schema/codec/JsonCodec.scala`	Implements a post-decode “no trailing content” check for `zioJsonBinaryCodec.decode`.
`zio-schema-json/shared/src/test/scala/zio/schema/codec/JsonCodecSpec.scala`	Adds regression tests covering trailing-character rejection for the zio-json-based binary codec.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-03T07:50:52Z

+      override def decode(whole: Chunk[Byte]): Either[DecodeError, A] = {
+        val reader = new FastStringReader(new String(whole.toArray, JsonEncoder.CHARSET))
+        try {
+          val result = jsonCodec.decoder.unsafeDecode(Nil, reader)
+          // Verify no trailing content after valid JSON
+          try {
+            val next = reader.nextNonWhitespace()
+            if (next != -1) {
+              Left(DecodeError.ReadError(Cause.empty, s"Invalid JSON: found '${next.toChar}' after valid content"))
+            } else {
+              Right(result)
+            }
+          } catch {
+            case _: UnexpectedEnd => Right(result)
+          }
+        } catch {
+          case UnsafeJson(trace) => Left(DecodeError.ReadError(Cause.empty, JsonError.render(trace)))
+        }
+      }


This strict trailing-character check is only applied in zioJsonBinaryCodec.decode. The schema-based entrypoints (schemaBasedBinaryCodec.decode and JsonCodec.JsonDecoder.decode, which still delegate to decodeJson) appear unchanged and may therefore continue accepting inputs like {}} / "foo"". To fully address #712, consider centralizing the strict parse logic and using it for all decode paths (including schemaBasedBinaryCodec and any String/CharBuffer decoders).

Copilot · 2026-03-03T07:50:52Z

+            case _: UnexpectedEnd => Right(result)
+          }
+        } catch {
+          case UnsafeJson(trace) => Left(DecodeError.ReadError(Cause.empty, JsonError.render(trace)))


unsafeDecode is called directly here, but the catch block only handles UnsafeJson. If unsafeDecode (or nextNonWhitespace) throws another NonFatal exception, it will escape the codec and potentially fail the fiber rather than returning a DecodeError. Consider catching NonFatal as well and mapping it to DecodeError.ReadError (while still letting fatal errors propagate).

Suggested change

case UnsafeJson(trace) => Left(DecodeError.ReadError(Cause.empty, JsonError.render(trace)))

case UnsafeJson(trace) =>

Left(DecodeError.ReadError(Cause.empty, JsonError.render(trace)))

case NonFatal(e) =>

Left(DecodeError.ReadError(Cause.fail(e), Option(e.getMessage).getOrElse(e.toString)))

Copilot · 2026-03-03T07:50:53Z

+            if (next != -1) {
+              Left(DecodeError.ReadError(Cause.empty, s"Invalid JSON: found '${next.toChar}' after valid content"))
+            } else {


The trailing-content error message interpolates next.toChar directly. For whitespace/control characters this can be hard to read (or misleading) in logs. Consider rendering a safe/escaped representation (and/or the numeric code point) so the user can identify the offending character reliably.

Copilot · 2026-03-03T07:50:53Z

+      assertTrue(
+        bc.decode(Chunk.fromArray("""{"name":"Alice","age":30}}""".getBytes(java.nio.charset.StandardCharsets.UTF_8)))
+          .isLeft,
+        bc.decode(Chunk.fromArray("""{"name":"Alice","age":30}""".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(
+          Person("Alice", 30)
+        )
+      )
+    },
+    test("string") {
+      implicit val jc: zio.json.JsonCodec[String] = JsonCodec.jsonCodec(Schema[String])
+      val bc                                      = JsonCodec.zioJsonBinaryCodec[String]
+      assertTrue(
+        bc.decode(Chunk.fromArray("\"foo\"\"".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,
+        bc.decode(Chunk.fromArray("\"foo\"".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right("foo")
+      )
+    },
+    test("array") {
+      implicit val jc: zio.json.JsonCodec[List[Int]] = JsonCodec.jsonCodec(Schema[List[Int]])
+      val bc                                         = JsonCodec.zioJsonBinaryCodec[List[Int]]
+      assertTrue(
+        bc.decode(Chunk.fromArray("[1,2]]".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,
+        bc.decode(Chunk.fromArray("[1,2]".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(List(1, 2))
+      )
+    },
+    test("whitespace then garbage") {
+      implicit val jc: zio.json.JsonCodec[Int] = JsonCodec.jsonCodec(Schema[Int])
+      val bc                                   = JsonCodec.zioJsonBinaryCodec[Int]
+      assertTrue(
+        bc.decode(Chunk.fromArray("42 x".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,
+        bc.decode(Chunk.fromArray("42   ".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(42)
+      )
+    },
+    test("multiple trailing") {
+      implicit val jc: zio.json.JsonCodec[Boolean] = JsonCodec.jsonCodec(Schema[Boolean])
+      val bc                                       = JsonCodec.zioJsonBinaryCodec[Boolean]
+      assertTrue(
+        bc.decode(Chunk.fromArray("true}}".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft
+      )
+    },
+    test("empty with trailing") {
+      implicit val jc: zio.json.JsonCodec[Map[String, String]] = JsonCodec.jsonCodec(Schema[Map[String, String]])
+      val bc                                                   = JsonCodec.zioJsonBinaryCodec[Map[String, String]]
+      assertTrue(
+        bc.decode(Chunk.fromArray("{}}}".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,
+        bc.decode(Chunk.fromArray("{}".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(
+          Map.empty[String, String]


These new trailing-character tests exercise zioJsonBinaryCodec, but the main public API in this module is often schemaBasedBinaryCodec (and JsonDecoder.decode). Adding equivalent assertions for the schema-based decoder would ensure the regression is covered for all supported decode entrypoints.

Suggested change

assertTrue(

bc.decode(Chunk.fromArray("""{"name":"Alice","age":30}}""".getBytes(java.nio.charset.StandardCharsets.UTF_8)))

.isLeft,

bc.decode(Chunk.fromArray("""{"name":"Alice","age":30}""".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(

Person("Alice", 30)

)

)

},

test("string") {

implicit val jc: zio.json.JsonCodec[String] = JsonCodec.jsonCodec(Schema[String])

val bc = JsonCodec.zioJsonBinaryCodec[String]

assertTrue(

bc.decode(Chunk.fromArray("\"foo\"\"".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

bc.decode(Chunk.fromArray("\"foo\"".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right("foo")

)

},

test("array") {

implicit val jc: zio.json.JsonCodec[List[Int]] = JsonCodec.jsonCodec(Schema[List[Int]])

val bc = JsonCodec.zioJsonBinaryCodec[List[Int]]

assertTrue(

bc.decode(Chunk.fromArray("[1,2]]".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

bc.decode(Chunk.fromArray("[1,2]".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(List(1, 2))

)

},

test("whitespace then garbage") {

implicit val jc: zio.json.JsonCodec[Int] = JsonCodec.jsonCodec(Schema[Int])

val bc = JsonCodec.zioJsonBinaryCodec[Int]

assertTrue(

bc.decode(Chunk.fromArray("42 x".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

bc.decode(Chunk.fromArray("42 ".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(42)

)

},

test("multiple trailing") {

implicit val jc: zio.json.JsonCodec[Boolean] = JsonCodec.jsonCodec(Schema[Boolean])

val bc = JsonCodec.zioJsonBinaryCodec[Boolean]

assertTrue(

bc.decode(Chunk.fromArray("true}}".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft

)

},

test("empty with trailing") {

implicit val jc: zio.json.JsonCodec[Map[String, String]] = JsonCodec.jsonCodec(Schema[Map[String, String]])

val bc = JsonCodec.zioJsonBinaryCodec[Map[String, String]]

assertTrue(

bc.decode(Chunk.fromArray("{}}}".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

bc.decode(Chunk.fromArray("{}".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(

Map.empty[String, String]

val sbc = JsonCodec.schemaBasedBinaryCodec[Person](personSchema)

assertTrue(

bc.decode(Chunk.fromArray("""{"name":"Alice","age":30}}""".getBytes(java.nio.charset.StandardCharsets.UTF_8)))

.isLeft,

sbc.decode(Chunk.fromArray("""{"name":"Alice","age":30}}""".getBytes(java.nio.charset.StandardCharsets.UTF_8)))

.isLeft,

bc.decode(Chunk.fromArray("""{"name":"Alice","age":30}""".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(

Person("Alice", 30)

),

sbc.decode(Chunk.fromArray("""{"name":"Alice","age":30}""".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(

Person("Alice", 30)

)

)

},

test("string") {

implicit val jc: zio.json.JsonCodec[String] = JsonCodec.jsonCodec(Schema[String])

val bc = JsonCodec.zioJsonBinaryCodec[String]

val sbc = JsonCodec.schemaBasedBinaryCodec[String](Schema[String])

assertTrue(

bc.decode(Chunk.fromArray("\"foo\"\"".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

sbc.decode(Chunk.fromArray("\"foo\"\"".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

bc.decode(Chunk.fromArray("\"foo\"".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right("foo"),

sbc.decode(Chunk.fromArray("\"foo\"".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right("foo")

)

},

test("array") {

implicit val jc: zio.json.JsonCodec[List[Int]] = JsonCodec.jsonCodec(Schema[List[Int]])

val bc = JsonCodec.zioJsonBinaryCodec[List[Int]]

val sbc = JsonCodec.schemaBasedBinaryCodec[List[Int]](Schema[List[Int]])

assertTrue(

bc.decode(Chunk.fromArray("[1,2]]".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

sbc.decode(Chunk.fromArray("[1,2]]".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

bc.decode(Chunk.fromArray("[1,2]".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(List(1, 2)),

sbc.decode(Chunk.fromArray("[1,2]".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(List(1, 2))

)

},

test("whitespace then garbage") {

implicit val jc: zio.json.JsonCodec[Int] = JsonCodec.jsonCodec(Schema[Int])

val bc = JsonCodec.zioJsonBinaryCodec[Int]

val sbc = JsonCodec.schemaBasedBinaryCodec[Int](Schema[Int])

assertTrue(

bc.decode(Chunk.fromArray("42 x".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

sbc.decode(Chunk.fromArray("42 x".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

bc.decode(Chunk.fromArray("42 ".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(42),

sbc.decode(Chunk.fromArray("42 ".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(42)

)

},

test("multiple trailing") {

implicit val jc: zio.json.JsonCodec[Boolean] = JsonCodec.jsonCodec(Schema[Boolean])

val bc = JsonCodec.zioJsonBinaryCodec[Boolean]

val sbc = JsonCodec.schemaBasedBinaryCodec[Boolean](Schema[Boolean])

assertTrue(

bc.decode(Chunk.fromArray("true}}".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

sbc.decode(Chunk.fromArray("true}}".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft

)

},

test("empty with trailing") {

implicit val jc: zio.json.JsonCodec[Map[String, String]] = JsonCodec.jsonCodec(Schema[Map[String, String]])

val bc = JsonCodec.zioJsonBinaryCodec[Map[String, String]]

val sbc = JsonCodec.schemaBasedBinaryCodec[Map[String, String]](Schema[Map[String, String]])

assertTrue(

bc.decode(Chunk.fromArray("{}}}".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

sbc.decode(Chunk.fromArray("{}}}".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,

bc.decode(Chunk.fromArray("{}".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(

Map.empty[String, String]

),

sbc.decode(Chunk.fromArray("{}".getBytes(java.nio.charset.StandardCharsets_UTF_8))) == Right(

Map.empty[String, String]

Copilot · 2026-03-03T07:50:53Z

+      assertTrue(
+        bc.decode(Chunk.fromArray("""{"name":"Alice","age":30}}""".getBytes(java.nio.charset.StandardCharsets.UTF_8)))
+          .isLeft,
+        bc.decode(Chunk.fromArray("""{"name":"Alice","age":30}""".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right(
+          Person("Alice", 30)
+        )
+      )
+    },
+    test("string") {
+      implicit val jc: zio.json.JsonCodec[String] = JsonCodec.jsonCodec(Schema[String])
+      val bc                                      = JsonCodec.zioJsonBinaryCodec[String]
+      assertTrue(
+        bc.decode(Chunk.fromArray("\"foo\"\"".getBytes(java.nio.charset.StandardCharsets.UTF_8))).isLeft,
+        bc.decode(Chunk.fromArray("\"foo\"".getBytes(java.nio.charset.StandardCharsets.UTF_8))) == Right("foo")
+      )


The new tests repeat Chunk.fromArray(...getBytes(StandardCharsets.UTF_8)) in multiple places. This spec already uses helpers like charSequenceToByteChunk(...) for readability and consistency; consider using the existing helper here to reduce noise and avoid repeating charset plumbing in each assertion.

Dee66 requested a review from a team as a code owner December 31, 2025 19:36

algora-pbc bot added the 🙋 Bounty claim label Dec 31, 2025

algora-pbc bot mentioned this pull request Dec 31, 2025

JsonCodec decoder accepts malformed JSON string with extra characters #712

Open

Dee66 force-pushed the feature/issue-712-json-codec branch from 81086df to 92aed8b Compare December 31, 2025 20:42

Dee66 mentioned this pull request Jan 9, 2026

Minimal json strict codec zio/zio-blocks#571

Closed

987Nabil requested changes Jan 23, 2026

View reviewed changes

reject JSON with trailing characters

c10d722

fixes zio#712

Dee66 force-pushed the feature/issue-712-json-codec branch from faeddf9 to c10d722 Compare January 24, 2026 08:37

add type annotations for scala 3

f87ad66

Merge branch 'main' into feature/issue-712-json-codec

deb870e

987Nabil requested a review from Copilot March 3, 2026 07:46

Copilot started reviewing on behalf of 987Nabil March 3, 2026 07:46 View session

Copilot AI reviewed Mar 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add strict JSON parsing with String/CharBuffer support#974

feat: add strict JSON parsing with String/CharBuffer support#974
Dee66 wants to merge 3 commits intozio:mainfrom
Dee66:feature/issue-712-json-codec

Dee66 commented Dec 31, 2025 •

edited

Loading

Uh oh!

Dee66 commented Dec 31, 2025

Uh oh!

Dee66 commented Jan 11, 2026 •

edited

Loading

Uh oh!

987Nabil left a comment

Uh oh!

Dee66 commented Jan 24, 2026 •

edited

Loading

Uh oh!

Dee66 commented Mar 2, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 3, 2026

Uh oh!

Copilot AI Mar 3, 2026

Uh oh!

Copilot AI Mar 3, 2026

Uh oh!

Copilot AI Mar 3, 2026

Uh oh!

Copilot AI Mar 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Dee66 commented Dec 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dee66 commented Dec 31, 2025

Uh oh!

Dee66 commented Jan 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

987Nabil left a comment

Choose a reason for hiding this comment

Uh oh!

Dee66 commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dee66 commented Mar 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Dee66 commented Dec 31, 2025 •

edited

Loading

Dee66 commented Jan 11, 2026 •

edited

Loading

Dee66 commented Jan 24, 2026 •

edited

Loading