Skip to content

Draft NDJSON support#81

Open
mstackhouse wants to merge 1 commit intomainfrom
ndjson_support
Open

Draft NDJSON support#81
mstackhouse wants to merge 1 commit intomainfrom
ndjson_support

Conversation

@mstackhouse
Copy link
Copy Markdown
Member

I had claude take a crack at support for NDJSON. This isn't going to address performance, specifically for writing, because it's bypassing yyjsonr's implementation. But I'm thinking that I can try contributing to yyjsonr to allow it to write out ndjson rows for dataframes the way that we need it. Once that's done, we could alter the implementation to use the yyjsonr defaults, or just wait to merge this then.

Here's an example of using the current implementation:

library(datasetjson)
ae_orig <- read_dataset_ndjson("https://github.com/cdisc-org/DataExchange-DatasetJson/raw/refs/heads/master/examples/sdtm/ae.ndjson")

infile <- tempfile()
write_dataset_ndjson(ae_orig, infile)

ae_comp <- read_dataset_ndjson(infile, chunk_size=10)

diffdf::diffdf(ae_orig, ae_comp)
#> No issues were found!

Created on 2026-02-04 with reprex v2.1.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant