Heavy heap usage on JavaScript

[patricklucas]

This is not really a bug with xmlutil, but there are possibly ways to mitigate the issue here.

Short version: Kotlin's CharArray.concatToString JavaScript implementation builds up strings character-by-character using +=, and KtXmlReader.text/KtXmlReader.get() calls concatToString on the buffer, resulting in an object allocated and retained for every byte in every text section.

I've opened an issue on Kotlin here: https://youtrack.jetbrains.com/issue/KT-84791/Many-string-operations-allocate-for-each-character

Per my suggestion at the bottom of that issue, I wonder if an improvement to xmlutil would be to induce string Cons compaction on JavaScript in particular immediately after the call to concatToString in KtXmlReader.

[pdvrieze]

Thanks for letting me know. I'm not that much of a JS person so I hadn't looked into it in detail, but obviously this approach is far from optimal. I'll have a look at what I can do. The whole point of concatToString is exactly to allow for fast copies (it is a nice speed-up on the JVM over doing it for individual characters), but creating many temporary strings is clearly not the optimal approach.

[pdvrieze]

I'll look at some workarounds. They seem to exist, but indeed it would be better if the platform/standard library did it itself. It seems that 2 copies are still needed (from full array to slice, and from slice to string).

Looking at the issue, the best approach may be to look at just using an underlying string. In the meantime, it may be an option to use DOM based parsing instead (rather than the generic implementation).

[pdvrieze]

Hi @patricklucas , you may want to try to check out a snapshot. I have redone quite some bits of the generic parser. It now has the ability to work directly on a string source and use substrings for those cases (limited intermediate stringbuilders). Please note however that if you have multi-byte line endings (windows CRLF) that occur inside text content this will still trigger a StringBuilder to be used to accumulate the text (normalization is needed); (I haven't worked on creating a javascript specific accumulator yet).

[patricklucas]

Thanks @pdvrieze, it's looking quite good. My benchmark shows a ~22% reduction in retained memory, from 2.25 GiB -> 1.76 GiB, and a 7% speedup.

This code runs both on the server and in the browser, FYI. The benchmark is in Node, so it would be using the generic parser. I've been meaning to try out the generic parser in the browser to compare it to the DOM parser but haven't gotten to it yet.

Minor note regarding the upgrade to 1.0.0-SNAPSHOT: the deprecation warning on recommended_0_91_0 says "Use the new 1_0 accessors instead, it is equivalent", but I noticed via some failing tests that recommended_1_0 sets xmlDeclMode to IfRequired vs. the previous value of Minimal. This wasn't a big problem for me, but just wanted to let you know.

[pdvrieze]

@patricklucas It seems that the fix for stringbuilder has gone in for Kotlin 2.3.20. That should help a lot too.

[patricklucas]

I think it missed 2.3.20 by a couple of weeks, but hopefully it will be included in 2.4.0, due out later this month.

[pdvrieze]

I think it missed 2.3.20 by a couple of weeks, but hopefully it will be included in 2.4.0, due out later this month.

I looked at the source (I was attempting to write an alternative, but found it unneeded). It is still not optimal for the use case, but a lot better.