Building on go-ipfs, c-ipfs: C/CGo extensions

[H-Plus-Time]

Would the maintainers be averse to doing this the (easier) way of building on C (I'm thinking kenCode's implementation) or CGo extensions?

The main con would probably be the loss of (some, perhaps not all) of the feedback on the spec, and the need to publish wheels. Pros would be implementation time, execution time (by a huge margin initially) and faster updates (in the case of C/Go internals).

[renatoathaydes]

Implementing IPFS purely in Python seems like a major waste of effort. I agree that a binding for Python would be much more adequate than a separate implementation entirely.

[mitra42]

One problem with a binding is that you lose all the open-source support from people using it that might dig in and fix things. One challenge with IPFS is that, being written in Go, its community of developers is really small (compared to Python and Javascript coders).

[ntninja]

As mentioned by me in another thread, I believe there would be some utility in having a Python client-only implementation of the IPFs protocol. However, client-only would, most importantly, not include things like:

• Server implementation of the HTTP Client API (or maybe just a limited subset of it for testing)
• A performant block storage (but hopefully a simple optional block cache at some point)
• A full DHT implementation (just enough for walking the DHT and finding data would be good enough)
• Any support for serving data to the network that isn't strictly required to make the library usable as such

Just like it is more convinient to use HTTP client libraries that are written in Python, I'd think it'd also be a lot more convinient to have an IPFS client library written in Python for Python users.

I'm not arguing that any of this is a huge priority through, just trying to argue that there probably is some point in having something like this if we don't set the wrong goals…

[mitra42]

I think one of the questions is how you think this will be used.

Lets take our example (archive.org) our servers are all in Python, I'm building gateways between our content and IPFS, but they are all written in Python using the http client API to talk to a separate IPFS server. This makes them innefficient and much less scalable, than if for example we could act as a IPFS server, and retrieve material direct from our disks to serve.

For example ... at the moment when a client (javascript on the browser) requests a search, I have to send it to the IPFS server, then hit ipfs.io so that the DHT propogates it, then give the IPFS client the ipfs hash. This is super inefficient and unscalable (though Kyle is doing a hack to make it to avoid double storage, its always going to make the client unacceptably slow).

Alternative solutions involve learning Go, and then rewriting internal (python) libraries in Go which realistically isn't going to happen. I'm sure we aren't the only server side people who are primarily Python.

[benmoran56]

I agree with @mitra42. In my opinion, a pure python implementation certainly has value. Not just the client library, but a full implementation eventually.

[mitra42]

Yes - and it might be that libp2p is not the most useful part to do first.

For example at the moment, the part I miss most is an IPLD constructor so that we could build IPLD files on the server, rather than having to feed the whole file to an IPFS (via HTTP) instance in order to get a IPLD file.