Improve the jOOQ transaction experience

[lukaseder]

When using Spring Boot with jOOQ, a common source of confusion is how to use transactions. There are mainly three options:

• Ignore jOOQ's transaction APIs, and do everything Spring style (using @Transactional annotations or programmatically, using a TransactionManager
• Ignore Spring's transaction APIs and use jOOQ's APIs only
• Mix the two APIs

The first case doesn't lead to any integration problems, but the second and third ones do. For example, the org.springframework.boot.autoconfigure.jooq.SpringTransactionProvider does not work with jOOQ's asynchronous transactions as can be seen in this issue here: jOOQ/jOOQ#10850 (helpful MCVE here: https://github.com/anushreegupta44/jOOQ-mcve). The workaround is to manually remove the SpringTransactionProvider again from the jOOQ Configuration, but users can reasonably expect this to work out of the box, especially when there are no Spring transactions involved.

The SpringTransactionProvider was originally implemented as a bridge to allow for mixing Spring transactions (e.g. based on @Transactional annotations), and then in some methods, use jOOQ's transaction APIs, which delegate to Spring, but it's probably not implemented correctly for all cases, nor am I sure whether it should always be added to Spring Boot's jOOQ auto-configuration, because when people want to use jOOQ's transaction APIs, it's not necessary.

I'm not sure what the best approach here is to improve the out-of-the-box integration experience for all three of the above cases... My intutition tells me that the SpringTransactionProvider should get an instance of the org.jooq.impl.DefaultTransactionProvider and delegate to it for all three methods (begin, commit, rollback) when TransactionSynchronizationManager.isActualTransactionActive() is false, but that may not be sufficient for asynchronous transactions.

Thoughts?

[wilkinsona]

My first thoughts are that I'd need to learn more about jOOQ's transaction support to form an opinion. @michael-simons as someone who knows Spring and jOOQ pretty well, your thoughts on this would be more than welcome if you have the time.

[michael-simons]

I can only answer from Neo4j's perspective:
Neo4j-OGM added something like you have on top to integrate driver transaction with Spring Transaction and it is a major pain. Something fails all the time.

We decided to provide both imperative and reactive, native Spring Transaction managers, that opt all in.
If people don't like it, they can bail out and use our driver transaction (imperative, async or reactive).

I'm advocating internally for exact that approach.

[lukaseder]

If people don't like it, they can bail out and use our driver transaction (imperative, async or reactive).

How does bailing out work in your case? Is that something that has to be done explicitly via extra configuration? Or is it automatic, once they use your own native transaction APIs?

[michael-simons]

If someone goes down into the driver layer - same level as pure JDBC connection - it’s the driver’s transaction that is leading. A potentially outer one coming from the application container won’t change a thing. In short: Yes to the „or"

…

Am 05.11.2020 um 21:52 schrieb Lukas Eder ***@***.***>: If people don't like it, they can bail out and use our driver transaction (imperative, async or reactive). How does bailing out work in your case? Is that something that has to be done explicitly via extra configuration? Or is it automatic, once they use your own native transaction APIs? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24049 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEAQL3VWDZLUI2J4VKZWHLSOMGAPANCNFSM4TLGBZMA>.

[lukaseder]

Oh interesting, so you start a new one despite there possibly already being one from the container. I kinda tend to think that whoever's first, wins, and the other "party" needs to make sure their transaction is properly nested, unless a new one is requested explicitly... That way, users can nest A in B or B in A, irrespective of what library/framework created A and/or B

[michael-simons]

What does a JDBC Connection do when a user grabs one from their driver and uses it inside the container connection, not knowing that the data source is managed? Lukas Eder <notifications@github.com> schrieb am Do. 5. Nov. 2020 um 22:06:

…

Oh interesting, so you start a new one despite there possibly already being one from the container. I kinda tend to think that whoever's first, wins, and the other "party" needs to make sure their transaction is properly nested, unless a new one is requested explicitly... That way, users can nest A in B or B in A, irrespective of what library/framework created A and/or B — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#24049 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEAQL64YYB66YBNBQMMXCLSOMHT5ANCNFSM4TLGBZMA> .

[lukaseder]

Sure, there's always a way to bypass things.

But the Spring Boot jOOQ starter configures jOOQ with a managed DataSource and a SpringTransactionProvider, which doesn't work correctly for some cases. This isn't about calling JDBC API on unmanaged data sources. This is about calling jOOQ API which can be reasonably expected to operate on the managed data source. Something that is supposed to work out of the box currently isn't working, so combining Spring Boot and jOOQ (in this case) leads to more setup effort rather than less setup effort.

My question here is whether jOOQ needs to be fixed, or the SpringTransactionProvider, or something entirely different. I tend to think the SpringTransactionProvider is not up to date, but before I look into providing a PR, I wanted to collect a few opinions.

[wilkinsona]

My intutition tells me that the SpringTransactionProvider should get an instance of the org.jooq.impl.DefaultTransactionProvider and delegate to it for all three methods (begin, commit, rollback) when TransactionSynchronizationManager.isActualTransactionActive() is false, but that may not be sufficient for asynchronous transactions.

Would this not make this significantly more complicated? Assuming I've understood things correctly, won't it result in either Spring's PlatformTransactionManager or jOOQ managing the current transaction? The current intent of SpringTransactionProvider is to ensure that Spring's PlatformTransactionManager is solely responsible for transaction management.

As far as I can tell, the crux of the problem is jOOQ's async transaction support and the lack of good mapping to Spring's imperative transaction support which is ThreadLocal-based. The MCVE doesn't seem to gain anything from adapting jOOQ's transactions to Spring's PlatformTransactionManager and is, in fact, hurt by it.

Rather than trying to allow users to mix and match the two APIs (which will be complex and may not even be possible), I wonder if we'd be better providing a configuration property that allows uses to specify whether they want to use Spring's transaction management or jOOQ's transaction management. Basically make it easier to opt out of (or into if we flip the default) SpringTransactionProvider being configured.

[lukaseder]

Would this not make this significantly more complicated?

Yes, absolutely, for us maintainers. But not necessarily for the users.

Assuming I've understood things correctly, won't it result in either Spring's PlatformTransactionManager or jOOQ managing the current transaction?

Depending on who starts things, Spring or jOOQ would manage the current transaction, and the "other side" would have to pick it up.

Looking at it more closely, jOOQ would actually not start the current transaction itself, but the SpringTransactionProvider SPI implementation would start a "jOOQ aware Spring transaction", if that makes sense.

The current intent of SpringTransactionProvider is to ensure that Spring's PlatformTransactionManager is solely responsible for transaction management.

That, and that jOOQ's sync transaction APIs can make use of it, as clients of Spring's PlatformTransactionManager

As far as I can tell, the crux of the problem is jOOQ's async transaction support and the lack of good mapping to Spring's imperative transaction support which is ThreadLocal-based.

Well, if there were any plans of offering "imperative" (or rather "programmatic") transaction support that is not ThreadLocal based in Spring, those could be used by SpringTransactionProvider, too. But I don't know Spring well enough to know if this is already possible, or will be possible in the near future. It would be my preferred solution.

The MCVE doesn't seem to gain anything from adapting jOOQ's transactions to Spring's PlatformTransactionManager and is, in fact, hurt by it.

Yes

Rather than trying to allow users to mix and match the two APIs (which will be complex and may not even be possible)

My recommendation is to generally not mix things, but people will inevitably try it.

Of course, we could also change course here, and disallow mixing things per documentation and implementation, but that would probably break quite a few applications out there, at least in the sync case. A possibility would be to log a warning, about mixing not being recommended.

In the async case, jOOQ could just check the TransactionProvider class name and log a warning or throw an UnsupportedOperationException, if it is SpringTransactionProvider, with some explanations.

I wonder if we'd be better providing a configuration property that allows uses to specify whether they want to use Spring's transaction management or jOOQ's transaction management. Basically make it easier to opt out of (or into if we flip the default) SpringTransactionProvider being configured.

That would be great in addition to what I'm suggesting (and the above thrown exception). Personally, I wouldn't flip the default. If someone uses jOOQ with Spring (Boot), they're very likely to want to use Spring for transaction management (mostly via @Transactional annotations), or they already use Spring and want to start using jOOQ. I think that happens more often than someone already using jOOQ and starting to want to use Spring.

But it will still mean that people will first have to understand all of this, and make an informed decision, possibly when evaluating things for the first time. Until that decision, things may simply seem not to work. If users are not jOOQ and/or Spring (Boot) power users, they might already be deep down in some rabbit hole, finding it hard to recover from the "mistake" of mixing the two APIs, at least, that's how I perceive it from my support case experience.