Add trio implementation

[aaugustin]

No description provided.

[agronholm]

So, while working on my own version, I started wondering if the Assembler class was actually something that should be left as an implementation detail rather than part of the API? I didn't really understand why it was a separate thing from the connection classes.

Another big change I had to make was the introduction of pytest as a test dependency, as testing with plain unittest would require a custom harness which I feel is pointless when AnyIO's pytest plugin contains all the necessary machinery to support async tests.

[aaugustin]

@agronholm The Assembler is an implementation detail: it isn't documented in the API. It could live in the connection module. It happens to be an independent piece with medium complexity, which justifies giving it its own module. That keeps the connection module simpler. This is a personal preference in how I organize the code with no user-visible consequences.

[aaugustin]

websockets got its own async test harness before there was one in the stdlib. The legacy implementation still uses it. I switched to unittest.IsolatedAsyncioTestCase in the new asyncio implementation.

I chose to stick to vanilla choices everywhere. websockets has no required dependencies, even for development. I am clearly biased by:

• my background in Django, which didn't have any dependencies for an extremely long time;
• my desire to avoid debugging dependencies :-) I got my share of debugging with asyncio ;-)

[agronholm]

OK, so is the introduction of pytest as a test dependency going to be a problem? I mean, I have plenty of experience in the best practices front so I can do all the heavy lifting in that regard (packaging, CI etc.).

[agronholm]

my background in Django, which didn't have any dependencies for an extremely long time;

Incidentally, this is the primary reason why I steer clear of Django.

[aaugustin]

Mastery of pytest isn't the problem. I used it in a professional context before. I know how to configure it. Actually, I spent a lot more time deep in the guts of the fixtures system than I ever wanted...

It's simply not suitable considering my goals and constraints for this project. Keeping complexity and the amount of third-party code that I may have to understand or debug to the bare minimum ranks very high on the list.

You're welcome to use it as a test runner though.

[agronholm]

Sure, it's your project, your rules. But I've just been thinking how much of the maintenance burden you could shed by relying on external projects rather than inventing everything yourself. Just my two cents.

[agronholm]

So are you going to continue with your work of adding a Trio-specific back-end? The pushes seem to indicate as much. Would you not rather let me finish my work on the AnyIO back-end?

[aaugustin]

Short, directional answer because it's 11:30pm here:

• I don't see myself reviewing a PR adding an entire implementation. There's a lot of details to get right; I find it quite hard to get them right myself; checking that someone else got them right feels impossible to me. We didn't talk before you started this work and I don't have a plan to integrate it. Of course you can publish websockets-anyio or fork.
• Since the asyncio backend is already there and there's no good reason to remove it, at this point, it's more straightforward to add just trio than to add anyio + trio. I don't see a benefit in adding anyio and I see a downside - more moving parts going foward.

[agronholm]

Since the asyncio backend is already there and there's no good reason to remove it, at this point, it's more straightforward to add just trio than to add anyio + trio. I don't see a benefit in adding anyio and I see a downside - more moving parts going foward.

I don't understand. Why even suggest adding anyio AND trio back-ends when the former caters for both Trio and asyncio users? The biggest reason would be fewer parts to maintain, not more!

[aaugustin]

Indeed, if I finish the trio implementation within a reasonable timeframe, there's little point in an anyio implementation.

As all things open-source, completion is highly dependent on everything else in my life; it isn't done until it's done :-)

---

I see your frustration that I'm not taking advantage of anyio when you built it exactly for people like me. It's fine to disagree. However, I tried to understand where our views diverge to make sure I'm not missing something. Going straight to the point:

1/ I don't have a significant cost of maintenance for code that I control. I invest upfront in getting it right. The cost of maintenance is driven by changes in code outside of my control, whether intentional or not. asyncio is outside of my control and has been painful in this regard. trio and anyio are outside of my control; they may be less painful as asyncio but still it's more code outside of my control. (Of course anyio is under your control so you have the opposite perspective here.)

2/ This is a hobby project for me. I spend time writing code in websockets because I like coding and I don't get the opportunity to write meaningful code in my current job. Some people do crosswords; I do open-source. It's time well spent, even if others have written similar code before. I wrote an unconventional HTTP parser; it does exactly what I want and never caused problems; I found that interesting. Conversely, I get less joy of creation from diagnosing what changed in someone else's code and suddenly caused websockets to misbehave or from reviewing someone else's contribution so I'm not looking to spend more time on these activities.

[agronholm]

Indeed, if I finish the trio implementation within a reasonable timeframe, there's little point in an anyio implementation.

As all things open-source, completion is highly dependent on everything else in my life; it isn't done until it's done :-)

I see your frustration that I'm not taking advantage of anyio when you built it exactly for people like me. It's fine to disagree. However, I tried to understand where our views diverge to make sure I'm not missing something. Going straight to the point:

1/ I don't have a significant cost of maintenance for code that I control. I invest upfront in getting it right. The cost of maintenance is driven by changes in code outside of my control, whether intentional or not. asyncio is outside of my control and has been painful in this regard. trio and anyio are outside of my control; they may be less painful as asyncio but still it's more code outside of my control. (Of course anyio is under your control so you have the opposite perspective here.)

Funnily enough, you're making my point for me. If you proceed with your current plan of adding a Trio backend, you will end up with two async back-ends, either one of which may break when the upstream makes changes. But if there was only the AnyIO back-end, then websockets would "just work" on both asyncio and Trio (and any other event loop it will support in the future; there have been talks of that), and you'd only have to worry about breakages occurring in AnyIO (which are quite rare these days), as it will shield you against most upstream changes.

[aaugustin]

Funnily enough, you're making my point for me.

This sounds like you're talking down to me. Please, let's have a constructive conversation or leave it there.

I'm clear on where I disagree with you. I'm trying — and, for now, failing — to land this conversation on a respectful agreement to disagree.

You're saying that using anyio will result in a lower maintenance cost. I don't buy it. I don't think there's a clear cut winner between:

1. Depending directly on asyncio and trio

• allows providing native APIs for both -> better UX IMO
• vulnerable to changes in asyncio and trio but debugging involves only one layer (debugging usually means reading the source)
• when appropriate, I can push back with "it's not me; it's asyncio / it's trio" as the user is actively choosing asyncio or trio

2. Depending directly on anyio and indirectly on asyncio and trio

• allows providing a unified API for both -> some might prefer this
• users are exposed structured concurrency (when many are just learning asyncio) -> leads to questions in the issue tracker
• vulnerable to changes in anyio and asyncio and trio, although anyio might shield some of them, but debugging involves two layers

(This isn't a full decision matrix; that's not my point; my point is that you have pros and cons on both sides.)

It's similar building a Swift and Kotlin apps or a React Native app: both options are valid, depending on your preferences and constraints.

---

Based on how this conversation went, it doesn't look like there's a path for smooth collaboration. Therefore, I won't consider a PR adding an anyio backend to websockets.

[agronholm]

Alright, thank you for the elaborate response. At least I tried. And sorry if I sounded like I was talking down to you – it was never my intention.

[aaugustin]

No worries. Feels like we fell into a typical trap of open-source collaboration :-(

If you have a WIP version of your anyio implementation somewhere, I'm genuinely interested in reading it because it will help me get a better grasp of anyio and structured concurrency (which I have no experience with).

[agronholm]

If you have a WIP version of your anyio implementation somewhere, I'm genuinely interested in reading it because it will help me get a better grasp of anyio and structured concurrency (which I have no experience with).

Here it is: https://github.com/agronholm/websockets/tree/anyio

[aaugustin]

Green build. Woohoo. Still a few fixes + full proof-reading necessary.

[aaugustin]

The first six commits were merged in #1669.

[fjarri]

Just curious, why trio specifically instead of anyio which would cover the existing asyncio implementation as well?

[aaugustin]

There's an extensive discussion of this question with the author of anyio in the discussion above. The very short version is:

• the asyncio implementation is quite battle-tested at this point; I do not wish to rewrite it on top of anyio and go through the whole process of making it robust again; (I've already done it twice!)
• I prefer minimizing the number of layers I depend on, as many bugs filed against websockets are actually bugs in lower layers, currently "Python" and "asyncio".

[aaugustin]

Also, since this is a hobby project, when two paths that are likely to lead to a similar result, "which one I'll be happiest to implement and maintain" is the decisive factor. Implementation cost is fairly irrelevant as I'm not interested in minimizing the time I spend enjoying my hobby :-)