Skip to content

Conversation

@owanio1992
Copy link

@owanio1992 owanio1992 commented Nov 1, 2025

feature source PR: fluent/fluent-bit#10665
This commit adds the json.escape_unicode option to the service configuration for both classic and YAML modes.
This option allows users to control the encoding of raw UTF-8 strings when serializing MessagePack records to JSON.

@owanio1992 owanio1992 requested review from a team as code owners November 1, 2025 07:36
@owanio1992 owanio1992 marked this pull request as draft November 1, 2025 07:39
@owanio1992 owanio1992 force-pushed the fix/missing_doc_#10665 branch from 50b5c77 to 35afdc1 Compare November 1, 2025 07:39
@owanio1992 owanio1992 changed the title Add json_encoding option to service configuration Add json.escape_unicode option to service configuration Nov 1, 2025
@owanio1992 owanio1992 marked this pull request as ready for review November 1, 2025 07:40
Copy link
Contributor

@cosmo0920 cosmo0920 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

json.escape_unicode handles boolean value. So,we need to use with On/true or Off/false for specifying escaping non-ASCII characters with Unicode code points.

@cosmo0920
Copy link
Contributor

@owanio1992 Could you add Signed-off line in your commit? Under the fluent organization repos, we request to use commit signs for every contributors.

@owanio1992 owanio1992 force-pushed the fix/missing_doc_#10665 branch 2 times, most recently from 7327cc7 to 99100bd Compare November 2, 2025 13:00
@esmerel
Copy link
Contributor

esmerel commented Nov 3, 2025

I'll come back for a tech writing review once @cosmo0920 's requested changes are done =)

Copy link
Contributor

@cosmo0920 cosmo0920 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For technical perspective, it looks good.
But DCO still complains that the author and the commiter information differs like:

Commit sha: 99100bd, Author: owan, Committer: owan; Expected "owan chihhao.wang@zyxel.com.tw", but got "owanio1992 owan.io1992@gmail.com".

@eschabell eschabell self-assigned this Nov 4, 2025
@eschabell eschabell added waiting-for-user Waiting for user/contributors feedback or requested changes dco DCO failing labels Nov 4, 2025
@eschabell
Copy link
Collaborator

@owanio1992 if you can look at the DCO failures this can be reviewed by @esmerel.

@eschabell eschabell requested a review from esmerel November 4, 2025 13:52
@owanio1992 owanio1992 force-pushed the fix/missing_doc_#10665 branch 2 times, most recently from b745e02 to 2a4e588 Compare November 4, 2025 14:37
@owanio1992 owanio1992 marked this pull request as draft November 4, 2025 14:40
@owanio1992 owanio1992 force-pushed the fix/missing_doc_#10665 branch 5 times, most recently from 2e06f6b to 9daaeb3 Compare November 4, 2025 15:04
Signed-off-by: owanio1992 <owan.io1992@gmail.com>
@owanio1992 owanio1992 force-pushed the fix/missing_doc_#10665 branch from 9daaeb3 to 9c63970 Compare November 4, 2025 15:06
@owanio1992 owanio1992 marked this pull request as ready for review November 4, 2025 15:06
@owanio1992
Copy link
Author

@cosmo0920 I update description and default value, please review again
thank you

Copy link
Contributor

@esmerel esmerel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 comments, same question on each. otherwise this looks fine to me

| `scheduler.cap` | Set a maximum retry time in seconds. Supported in v1.8.7 and greater. | `2000` |
| `scheduler.base` | Set a base of exponential backoff. Supported in v1.8.7 and greater. | `5` |
| `json.convert_nan_to_null` | If enabled, `NaN` converts to `null` when Fluent Bit converts `msgpack` to `json`. | `false` |
| `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as \uXXXX sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as \uXXXX sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` |
| `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as `\uXXXX` sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` |

This probably should be in code fence, but I'm not sure if the escape character needs to be in there or not

| `scheduler.cap` | Set a maximum retry time in seconds. Supported from v1.8.7. | `2000` |
| `scheduler.base` | Sets the base of exponential backoff. Supported from v1.8.7. | `5` |
| `json.convert_nan_to_null` | If enabled, NaN is converted to null when Fluent Bit converts `msgpack` to JSON. | `false` |
| `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as \uXXXX sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as \uXXXX sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` |
| `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as `\uXXXX` sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` |

same here

| `scheduler.cap` | Sets a maximum retry time in seconds. Supported in v1.8.7 and greater. | `2000` |
| `scheduler.base` | Sets the base of exponential backoff. Supported in v1.8.7 and greater. | `5` |
| `json.convert_nan_to_null` | If enabled, `NaN` is converted to `null` when Fluent Bit converts `msgpack` to `json`. | `false` |
| `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as \uXXXX sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as \uXXXX sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` |
| `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as `\uXXXX` sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` |

Copy link
Contributor

@cosmo0920 cosmo0920 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For technical side, it looks good to me. Thank you for your contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco DCO failing waiting-for-user Waiting for user/contributors feedback or requested changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants