Explanation for developers

[nickaxgit]

I hope this can become something useful to others - I have just spent a good few days getting a (fairly) good understanding of the structure - I have pointed out some of the areas of (my) confusion .... I think there is scope to improve the comments and possible rename a few things. This isn't meant to be a critique of the code just the fresh perspective of someone attempting to pick it up.
I hope to make a PR in the next few days with a working UDPConn

A dissection of Stacks

This is a very elegant implementation - but multi-layered and fairly hard to get your head around with the existing documentation
hopefully this will help future travellers - I may have some things wrong here - please correct me

Some of the entities and methods at different levels of the stack differ only in scope (and therefore case)

The structure appears to be kind of 'fractal' or 'self similar' at the, stack and connection level - this confused me, but recvEth and HandlEth are kindof 'bubbled' through - understanding this helps. (SendEth - might be a better name that HandleEth, but I suspect the naming is because of the way it is called)

Conn(ection) and sock(et) are used interchangeably - socket being used more as an interface (defining the 'shape' of the thing) and conn being a reference to one -- it might help to think of 'socket' as the class and 'conn' as an instance of the class - but the naming isn't entirely consistent with that.

Another area for confusion is (for example) handleEth, PortStack has a public HandleEth and a private handleEth, but also the interface Socket defines HandleEth, and ports (tcpPort and udpPort) implement HandleEth

portStack.handleEth is responsible for sending data (taking it from the socket/connection's ring buffers, and presenting to the NIC) - this is probably an recognised pattern, where the driver is invoking HandleEth to 'pull' data to be sent (rather than the 'higher' levels 'pushing' it) - this would be more obvious if it was better commented, consistency in the buffer name (response[] vs dst[]) would also help, in Go slices are passed by reference which is how/why the buffer can be passed in this way, but again it's not entirely obvious.

The NIC driver calls the portStacks handleEth() whenever it is ready to send data

PortStack.handleEth - calls HandleEth on each connection object with pending data - passing them byy reference, a buffer (a slice of bytes) to fill with outbound data.

The ports HandleEth method, ultimately invokes, the port's, handler's send() method -- again passing a buffer to fill (dst[])

Note that the "Conns" any/all UDPConn/TCPconns (and their ring buffers) do not reside in the PortStack - in fact the library holds no reference to it/them at all - it IS the 'Conn' object the is returned to the user space code and the calling application holds the reference. (this confused me for a long time as there is quite a bit of code relating to the "conn" with no obvious bridge to the ports side of things.
All the members of the Conn structures are private to prevent external manipulation

Each Conn 'connection' (aka Sock) has a pair of ring buffers (TX[] and RX[]) - for outbound and inbound data
It also has a reference to its PortStack, It holds the remoteIP and localPort

Type PortStack

There is a single instance (I know this isn't OO, but we are probably all familiar with the terminology) of PortStack, per device (network adapter) .. thus, usually one.

Within each/the PortStack, are some summary counts of total packets, dropped, processed etc, along with some properties of the local device - Mac address, IP etc.

Importantly the PortStack also holds a slice of UDP and an slice of TCP Ports (of type udpPort and tcpPort respectively) - I can't actually see how these are populated/extended ?? - findAvailPort doesn't appear to do it

The structure of udpPort and tcpPort are essentially identical each has a handler (of type iudphandler and itcphandler respectively) and a port (number),

Both implement the Socket interface, which includes HandleEth() and a close() method. HandleEth() is responsible for sending packets

Conversely PortStack.RcvEth is responsible for processing arriving packets..

RecvEth reads (or rather is called with) the Ethernet frames.. it checks their checksums, headers etc, and the invokes recv() on the correct port's handler (which implements either iudphandler or itcphandler)

A Socket is a thing that that has HandleEth() and a Close() method UDPConn and TCPConn are two fine examples

The PortStack has a HandleEth() and a handleEth() AND each UDPConn or TCPConn has a HandleEth()

(repeat)
UDPConn has a send() method - this takes data from the sockets TX ring buffer and places it into a 'response' buffer
This is how the driver calls for udp packets to be sent

Opening 'connections'

A TCPConn "connection" is created by invoking stacks.NEWTCPConn() this is a static, 'factory method' (although none of those terms strictly apply to GO)
It accepts two parameters a Stack and some buffer size config , it returns a connection object that holds a reference to the portStack and initialises a set of tx and rx ring buffers.
No source or destination ports or addresses are defined yet - it's an 'empty' connection object.

OpenDialDCP is invoked on a TCPConnection (sock) - a local port number, and a remote address and port are provided by the caller. (sock is used as a object name, the type is a TCP/UDP Conn, I find this confusing - it is a socket, or a connection?? - what's the difference)
The connections "openStack" method is then invoked .. this calls OpenTCP on the connections portStack, passing the localportnum and the connection(sock) AS AS HANDLER

OpenDialTCP
OpenStack
stack.OpenTCP
findAvailablePort
port.open(portnum,connection) .... binds it to its connection (and the ring buffers therein) - making the connection the handler for this portnum

Incoming data

RecvEth on the PortStack is invoked (presumably by the underlying NIC driver) - a slice of bytes containing an ethernet frame is RX'd
headers and checksums are checked and valid packets have their destination port inspected.
This port is used to find the Port in the portStack's []UDPPorts or []TCPPorts - and recv method of the handler of that port is invoked, passing up a IP packet (UDP or TCP)
(it's worth nothing that the port itself has no implementation of recv … the implementation is provided by the handler 'bound' to the port when it is opened - The handler is a thing that impelements the iudp/itcp interface, ... ie a tcpConn or a udpConn - the conn that was created by user code with NewUDP/TCPConn)

Outgoing data
Something (presumably the NIC Driver) calls HandleEth() on the PortStack
HandleEth reads (or rather is called with) the Ethernet frames.. checks their checksums, headers etc, and the invokes send() on the correct ports handler (which is a Conn implementing either iudphandler or itcphandler -it referred to as a socket (which is confusing))

Common.go and setupWithDHCP

This part is actually (relatively) easy to understand

there are two important bit line 85 (or thereabouts)

dev.RecvEthHandle(stack.RecvEth) // Set the device to pass incomming Ethernet packets to the portStack's RecvEth method

and the main nicLoop

// Begin asynchronous (outbound) packet handling.
go nicLoop(dev, stack)

which populates a queue of outbound packets - by calling portStack's HandleEth() - thusly:-

lenBuf[i], err = Stack.HandleEth(buf[:]) //<- THIS is where we suck a packet out of the tx ring buffer, and place it (by reference) into the queue (for sending)

and then ranges over that queue to transmit the packets with this line:-

err := dev.SendEth(queue[i][:n]) //this is invoking the send() method, on the device sending an ethernet packet - in just the way you might imagine

I have not gone into the inner workings of device.pollOne or device.SendEth - presumably these are sending and receiving individual packets to the hardware over SPI

[soypat]

@nickaxgit Thank you for the detailed break down- at a glance this looks like its hitting the nail on the head on most topics.

I will leave a few notes to complement your observations

(SendEth - might be a better name that HandleEth, but I suspect the naming is because of the way it is called)

SendEth might sound good though it does not describe what is actually going on. The PortStack is not sending an ethernet packet, just populating the buffer with the next ethernet packet to send. It is the responsibility of the caller to then send the ethernet packet over the wifi/ethernet driver. There might be room for a better name than SendEth. I am not a fan of HandleEth either.

Conn(ection) and sock(et) are used interchangeably

A Conn refers to something that implements Go's net.Conn interface. So in this sense a Conn is abstract. A socket... is well... not very well defined in this library. We don't expose the word "socket" at all to users. I've often thought about Sockets as the lower level concept of a conn, i.e: "Socket occupies a port of a PortStack" (still unconvinced how to define socket, networking is hard)

Another area for confusion is (for example) handleEth, PortStack has a public HandleEth and a private handleEth, but also the interface Socket defines HandleEth, and ports (tcpPort and udpPort) implement HandleEth

How could this be improved? It seems to me that it is opportune to name them identically since they perform similar actions: filling a ethernet buffer with the response.

portStack.handleEth is responsible for sending data

Again, "sending" is not exactly what it is doing. It is populating the ethernet buffer with the response, but no actual sending is happening.

(sock is used as a object name, the type is a TCP/UDP Conn, I find this confusing - it is a socket, or a connection?? - what's the difference)

Ahh yes, you are correct in being confused. The naming is inconsistent because the concept of a Socket is loosely defined in this repository. I like using sock internally as a variable name to refer to anything that holds a port of PortStack.

The connections "openStack" method is then invoked

This "registers" the handler to the port.

(presumably by the underlying NIC driver)

This should be done by middleware or by a user that orchestrates the driver and seqs.PortStack in unison. This orchestration is what allows seqs to be extremely lightweight and run on a single goroutine, making it ideal for embedded systems.

(it's worth nothing that the port itself has no implementation of recv … the implementation is provided by the handler 'bound' to the port when it is opened - The handler is a thing that impelements the iudp/itcp interface, ... ie a tcpConn or a udpConn - the conn that was created by user code with NewUDP/TCPConn)

Interestingly (or not so) this layer of indirection saves a BUNCH of memory when ports are not in use due to how large the port buffers would be otherwise. We may want to change it in the future to avoid this indirection once we get zero-copy-rx working so that sockets/conns are much smaller.

I have not gone into the inner workings of device.pollOne or device.SendEth - presumably these are sending and receiving individual packets to the hardware over SPI

That is correct. These methods are cyw43439 functions. There really is no need to dive into that side if working on seqs.

[nickaxgit]

HandleEth -> GetNextOutboundPacket ?? (just brainstorming) -

" It is the responsibility of the caller to then send the ethernet packet over the wifi/ethernet driver. "
That's confusing (to me) .. I see the caller (of handleEth) as the NICloop .. effectively polling the connection for queued data..
The userspace code will be doing some form of connection.write() to populate those queues (the TX ring buffer) .. although they are oblivious to that.

Just to re-iterate - I'm not saying anything here is bad - just that some of the naming, comments and consistency could be slightly improved to make it more 'accessible' - I'm sure its pretty darned efficient