Python 3 secure mode is not functional

[janxious]

This is a bummer, and was hard to track down. The way non-blocking SSL-wrapped sockets function changed between python 2 and 3, and we need to change some stuff, or re-write how we do communication in python 3 (cpython AND pypy).

---

From @jason-o-matic:

I managed to reproduce it with a more basic example (I’ll paste below). It looks to me like the behavior changed so the ssl wrapping copies the original socket and closes the old one or something. I think we’ll need to update test_connection to do a different thing for an ssl socket, like they talk about here: https://docs.python.org/3/library/ssl.html#notes-on-non-blocking-sockets

I think we should add an ssl test, probably involving a test server locally. I don’t really want to write a test server in every language, and one already exists in the ruby agent, so maybe the ebst move is to update the test server in the ruby agent to handle ssl and then fire up that test server from the python agent for the tests.

Another thing that’s worrysome is how much of a breaking change in python this seems to be. It might be worth seeing if we can update the python agent tests to test multiple versions of python 3 if possible. Maybe there’s a thing like rbenv for python.

dumb code that currently results in an ssl error that I think means the connection is alive but would block at the moment:

import errno
import os
import time

import socket, ssl
import fcntl

HOST, PORT = 'collector.instrumentalapp.com', 8001

def handle(conn):
    # conn.write(b'hello\nauthenticate PROJECT_TOKEN\nincrement test.metric 1 1497560816\n')
    conn.send(bytes('hello\nauthenticate PROJECT_TOKEN\nincrement test.metric 1 1497560816\n', "ASCII"))
    conn.send(bytes('test.metric 1 %s\n' % round(time.time()), "ASCII"))
    print(conn.recv().decode())

def main():
    sock = socket.socket(socket.AF_INET)
    print(sock)
    context = ssl.create_default_context()
    print(sock)
    conn = context.wrap_socket(sock, server_hostname=HOST)
    print(sock)
    try:
        conn.connect((HOST, PORT))
        print(sock)
        conn.settimeout(10)
        handle(conn)
        print(sock)
        conn.settimeout(None)
        print(sock)
        conn.send(bytes('test.metric 1 %s\n' % round(time.time()), "ASCII"))
        print(sock)
        # s = sock
        s = conn
        # fcntl.fcntl(s, fcntl.F_SETFL, os.O_NONBLOCK)
        s.setblocking(0)
        try:
            msg = s.recv(1)
            return msg == ""
        except socket.error as e:
            raise
            err = e.args[0]
            if err == errno.EAGAIN or err == errno.EWOULDBLOCK:
                return True
            else:
                print("Socket connection error %s ---- %s" % (e, err))
                return False
    finally:
        conn.close()

# if __name__ == '__main__':
main()

---

It looks like https://docs.python.org/3/library/asyncio.html#module-asyncio is probably the thing that should be used in python3. Anyway, probably the next move is to disable secure in python 3 and print out a warning if someone enables it.

---

Depending on how good support for asyncio is on windows, this may also help with windows support (#13) for at least python 3.