Include namespace declarations on root elemant at serialization time

[patricklucas]

I am aware I can use the @XmlNamespaceDeclSpecs annotation to include namespace declarations on an element, but is there a way to tack those on at serialization time rather than using an annotation?

My use-case is that part of the element structure is described as xs:any, and I'm successfully using CompactFragment to handle this content, and I would like consumers of the library containing the serializable classes to be able to add namespace declarations to the root element when serializing so that the declaration doesn't have to be included on every single CompactFragment instance that uses that namespace.

In my case, a document can have thousands of such inner elements, and it would be nice to have the namespace declared just once. The library itself is not aware of what namespaces a consumer might want to use, so it cannot preemptively include them using @XmlNamespaceDeclSpecs.

I traced the code just a bit, and found that ultimately the decls are exposed in the descriptor via a call to config.policy.elementNamespaceDecls, so perhaps there is a way (or could be a way) to provide additional namespace declarations on the serializer config?

[pdvrieze]

The way to do this (for serialization) would be the isCollectingNSAttributes option in the configuration (not policy). It is off by default because it introduces additional overhead on writing (it has a two-pass approach where the first pass collects all used namespaces, and then writes the actual elements).

[patricklucas]

I did see that in the docs and tried it out, but it seemed to have no effect (I should have mentioned this in my original post).

My assumption was that it only works for namespace-qualified elements known to the descriptor, and wouldn't pick up namespaces on CompactFragments, but maybe that's wrong. After enabling the flag, I confirmed that the relevant codepath was followed, but the call to collectNamespaces only contained the root element's namespace.

I'll try it out again and make a reproducer if necessary.

That said, even if this works, I could imagine the approach suggested in my original post is still desirable functionality for a future release given it should avoid the second pass while still allowing configurability of namespace declarations.

[patricklucas]

I've put together a test file that seems to show that isCollectingNSAttributes indeed doesn't work for CompactFragments—though I did notice that whether or not I set it, the namespace declarations do get elevated by one level, from the CompactFragments themselves onto their parent element, which I did not expect.

However my test seems to have revealed another issue: somehow, it seems that one test encoding with a particular root element prefix leaks that prefix into subsequent tests.

When I run this test on its own, it passes, but when I run the whole file and it runs after a previous test which calls baseXmlConfig.encodeToString(rootElement, "foo2"), this test fails because the elements in the output that should have prefix foo instead have prefix foo2:

  @Test
  fun `test non-CompactFragment element with other namespace`() {
    val rootElement = RootElement("hello", listOf(),  Other("text"))
    log.info { rootElement }

    val output = baseXmlConfig.encodeToString(rootElement, "foo")
    log.info { output }

    val expectedOutput = """
      <?xml version='1.0' ?>
      <foo:rootElement xmlns:foo="http://example.com/foo" someAttribute="hello">
        <baz:other xmlns:baz="http://example.com/baz">text</baz:other>
      </foo:rootElement>
    """.trimIndent()

    assertThat(output).isEqualTo(expectedOutput)
  }

I've attached the test file. It uses assertk and kotlin-logging but usages of both can be easily replaced/removed.

Main relevant tests:

• test reserialize using isCollectingNSAttributes: tests that namespace decls from CompactFragments are picked up by isCollectingNSAttributes, fails
• test encoding leaks prefix: demonstrates the issue that encoding with a prefix affects subsequent calls

package com.example

import assertk.assertThat
import assertk.assertions.containsExactly
import assertk.assertions.isEqualTo
import io.github.oshai.kotlinlogging.KotlinLogging
import kotlinx.serialization.Serializable
import nl.adaptivity.xmlutil.XmlEvent
import nl.adaptivity.xmlutil.core.XmlVersion
import nl.adaptivity.xmlutil.serialization.XML
import nl.adaptivity.xmlutil.serialization.XmlElement
import nl.adaptivity.xmlutil.serialization.XmlSerialName
import nl.adaptivity.xmlutil.serialization.XmlValue
import nl.adaptivity.xmlutil.util.CompactFragment
import org.junit.jupiter.api.Test

private val log = KotlinLogging.logger {}

private val baseXmlConfig = XML {
  recommended_0_91_0 { pedantic = true }

  indentString = " ".repeat(2)
  xmlVersion = XmlVersion.XML10
}

class TestIt {

  @Test
  fun `test deserialize with ns decl on leaf elements`() {
    val input =
        """
      <?xml version='1.0' ?>
      <foo:rootElement xmlns:foo="http://example.com/foo" someAttribute="hello">
        <foo:childElement>
          <bar:thing1 xmlns:bar="http://example.com/bar">test</bar:thing1>
          <bar:thing2 xmlns:bar="http://example.com/bar">test</bar:thing2>
        </foo:childElement>
        <foo:childElement>
          <bar:thing3 xmlns:bar="http://example.com/bar">test</bar:thing3>
        </foo:childElement>
      </foo:rootElement>
    """
            .trimIndent()

    val rootElement = baseXmlConfig.decodeFromString<RootElement>(input, rootName = null)
    log.info { rootElement }

    assertThat(rootElement.someAttribute).isEqualTo("hello")
    assertThat(rootElement.childElements.flatMap { it.list })
        .containsExactly(
            CompactFragment("<bar:thing1 xmlns:bar=\"http://example.com/bar\">test</bar:thing1>"),
            CompactFragment("<bar:thing2 xmlns:bar=\"http://example.com/bar\">test</bar:thing2>"),
            CompactFragment("<bar:thing3 xmlns:bar=\"http://example.com/bar\">test</bar:thing3>"),
        )
  }

  @Test
  fun `test deserialize with ns decl on root element`() {
    val input =
        """
      <?xml version='1.0' ?>
      <foo:rootElement xmlns:foo="http://example.com/foo" xmlns:bar="http://example.com/bar" someAttribute="hello">
        <foo:childElement>
          <bar:thing1>test</bar:thing1>
          <bar:thing2>test</bar:thing2>
        </foo:childElement>
        <foo:childElement>
          <bar:thing3>test</bar:thing3>
        </foo:childElement>
      </foo:rootElement>
    """
            .trimIndent()

    val rootElement = baseXmlConfig.decodeFromString<RootElement>(input, rootName = null)
    log.info { rootElement }

    assertThat(rootElement.someAttribute).isEqualTo("hello")
    assertThat(rootElement.childElements.flatMap { it.list })
        .containsExactly(
            CompactFragment(
                namespaces = listOf(XmlEvent.NamespaceImpl("bar", "http://example.com/bar")),
                "<bar:thing1>test</bar:thing1>"),
            CompactFragment(
                namespaces = listOf(XmlEvent.NamespaceImpl("bar", "http://example.com/bar")),
                "<bar:thing2>test</bar:thing2>"),
            CompactFragment(
                namespaces = listOf(XmlEvent.NamespaceImpl("bar", "http://example.com/bar")),
                "<bar:thing3>test</bar:thing3>"),
        )
  }

  @Test
  fun `test reserialize with ns decl on leaf elements`() {
    val input =
        """
      <?xml version='1.0' ?>
      <foo:rootElement xmlns:foo="http://example.com/foo" someAttribute="hello">
        <foo:childElement>
          <bar:thing1 xmlns:bar="http://example.com/bar">test</bar:thing1>
          <bar:thing2 xmlns:bar="http://example.com/bar">test</bar:thing2>
        </foo:childElement>
        <foo:childElement>
          <bar:thing3 xmlns:bar="http://example.com/bar">test</bar:thing3>
        </foo:childElement>
      </foo:rootElement>
    """
            .trimIndent()

    val rootElement = baseXmlConfig.decodeFromString<RootElement>(input, rootName = null)
    log.info { rootElement }

    val output = baseXmlConfig.encodeToString(rootElement, "foo2")
    log.info { output }

    val expectedOutput =
        """
      <?xml version='1.0' ?>
      <foo2:rootElement xmlns:foo2="http://example.com/foo" someAttribute="hello">
        <foo2:childElement>
          <bar:thing1 xmlns:bar="http://example.com/bar">test</bar:thing1>
          <bar:thing2 xmlns:bar="http://example.com/bar">test</bar:thing2>
        </foo2:childElement>
        <foo2:childElement>
          <bar:thing3 xmlns:bar="http://example.com/bar">test</bar:thing3>
        </foo2:childElement>
      </foo2:rootElement>
    """
            .trimIndent()

    assertThat(output).isEqualTo(expectedOutput)
  }

  @Test
  fun `test reserialize with ns decl on root element`() {
    val input =
        """
      <?xml version='1.0' ?>
      <foo:rootElement xmlns:foo="http://example.com/foo" xmlns:bar="http://example.com/bar" someAttribute="hello">
        <foo:childElement>
          <bar:thing1>test</bar:thing1>
          <bar:thing2>test</bar:thing2>
        </foo:childElement>
        <foo:childElement>
          <bar:thing3>test</bar:thing3>
        </foo:childElement>
      </foo:rootElement>
    """
            .trimIndent()

    val rootElement = baseXmlConfig.decodeFromString<RootElement>(input, rootName = null)
    log.info { rootElement }

    val output = baseXmlConfig.encodeToString(rootElement, "foo2")
    log.info { output }

    val expectedOutput =
        """
      <?xml version='1.0' ?>
      <foo2:rootElement xmlns:foo2="http://example.com/foo" someAttribute="hello">
        <foo2:childElement xmlns:bar="http://example.com/bar">
          <bar:thing1>test</bar:thing1>
          <bar:thing2>test</bar:thing2>
        </foo2:childElement>
        <foo2:childElement xmlns:bar="http://example.com/bar">
          <bar:thing3>test</bar:thing3>
        </foo2:childElement>
      </foo2:rootElement>
    """
            .trimIndent()

    assertThat(output).isEqualTo(expectedOutput)
  }

  @Test
  fun `test reserialize using isCollectingNSAttributes`() {
    val input =
        """
      <?xml version='1.0' ?>
      <foo:rootElement xmlns:foo="http://example.com/foo" xmlns:bar="http://example.com/bar" someAttribute="hello">
        <foo:childElement>
          <bar:thing1>test</bar:thing1>
          <bar:thing2>test</bar:thing2>
        </foo:childElement>
        <foo:childElement>
          <bar:thing3>test</bar:thing3>
        </foo:childElement>
      </foo:rootElement>
    """
            .trimIndent()

    val thisXmlConfig = baseXmlConfig.copy { isCollectingNSAttributes = true }

    val rootElement = thisXmlConfig.decodeFromString<RootElement>(input, rootName = null)
    log.info { rootElement }

    val output = thisXmlConfig.encodeToString(rootElement, "foo2")
    log.info { output }

    val expectedOutput =
        """
      <?xml version='1.0' ?>
      <foo2:rootElement xmlns:foo2="http://example.com/foo" xmlns:bar="http://example.com/bar" someAttribute="hello">
        <foo2:childElement>
          <bar:thing1>test</bar:thing1>
          <bar:thing2>test</bar:thing2>
        </foo2:childElement>
        <foo2:childElement>
          <bar:thing3>test</bar:thing3>
        </foo2:childElement>
      </foo2:rootElement>
    """
            .trimIndent()

    // Fails, output actually matches `test reserialize with ns decl on root element`
    assertThat(output).isEqualTo(expectedOutput)
  }

  @Test
  fun `test non-CompactFragment element with other namespace`() {
    val rootElement = RootElement("hello", listOf(), Other("text"))
    log.info { rootElement }

    val output = baseXmlConfig.encodeToString(rootElement, "foo")
    log.info { output }

    val expectedOutput =
        """
      <?xml version='1.0' ?>
      <foo:rootElement xmlns:foo="http://example.com/foo" someAttribute="hello">
        <baz:other xmlns:baz="http://example.com/baz">text</baz:other>
      </foo:rootElement>
    """
            .trimIndent()

    assertThat(output).isEqualTo(expectedOutput)
  }

  @Test
  fun `test non-CompactFragment element with other namespace and isCollectingNSAttributes`() {
    val rootElement = RootElement("hello", listOf(), Other("text"))
    log.info { rootElement }

    val thisXmlConfig = baseXmlConfig.copy { isCollectingNSAttributes = true }

    val output = thisXmlConfig.encodeToString(rootElement, "foo")
    log.info { output }

    val expectedOutput =
        """
      <?xml version='1.0' ?>
      <foo:rootElement xmlns:foo="http://example.com/foo" xmlns:baz="http://example.com/baz" someAttribute="hello">
        <baz:other>text</baz:other>
      </foo:rootElement>
    """
            .trimIndent()

    assertThat(output).isEqualTo(expectedOutput)
  }

  @Test
  fun `test encoding leaks prefix`() {
    val rootElement = RootElement("hello", listOf(), Other("text"))
    log.info { rootElement }

    val output1 = baseXmlConfig.encodeToString(rootElement, "foo1")
    log.info { output1 }

    val expectedOutput1 =
        """
      <?xml version='1.0' ?>
      <foo1:rootElement xmlns:foo1="http://example.com/foo" someAttribute="hello">
        <baz:other xmlns:baz="http://example.com/baz">text</baz:other>
      </foo1:rootElement>
    """
            .trimIndent()

    assertThat(output1).isEqualTo(expectedOutput1)

    val output2 = baseXmlConfig.encodeToString(rootElement, "foo2")
    log.info { output2 }

    val expectedOutput2 =
        """
      <?xml version='1.0' ?>
      <foo2:rootElement xmlns:foo2="http://example.com/foo" someAttribute="hello">
        <baz:other xmlns:baz="http://example.com/baz">text</baz:other>
      </foo2:rootElement>
    """
            .trimIndent()

    // Fails, as output uses prefix foo1 instead of the prefix foo2
    assertThat(output2).isEqualTo(expectedOutput2)
  }
}

private const val NS = "http://example.com/foo"

@Serializable
@XmlSerialName("rootElement", namespace = NS)
data class RootElement(
    @XmlElement(false) val someAttribute: String,
    val childElements: List<ChildElement>,
    val other: Other?,
)

@Serializable
@XmlSerialName("childElement", namespace = NS)
data class ChildElement(@XmlValue val list: List<CompactFragment>)

@Serializable
@XmlSerialName("other", namespace = "http://example.com/baz", prefix = "baz")
data class Other(@XmlValue val text: String)