2021 - Summary of rust-libp2p user interviews

[mxinden]

I conducted user interviews with 4 major rust-libp2p users. My goal was to (a) understand how users use rust-libp2p today, (b) what users struggle with and (c) what users would need from rust-libp2p going forward. Below is a short summary.

Note: If you use rust-libp2p and would like to share your experience / suggestions in a call or some other way please send me an e-mail.

What is rust-libp2p used for today?

• Network layer of blockchain projects.

• Industry grade IOT mesh.

• File sharing networks.

Pain points

• Understanding the concepts of libp2p (e.g. NetworkBehaviour, Swarm, ProtocolsHandler, ...) is hard, especially for newcomers or people not working with rust-libp2p on a daily basis. This is especially true with the lack of getting started resources.

• The libp2p-swarm crate is difficult to use correctly. It imposes a complex architecture on applications and requires a user to have a clear understanding of its properties in order to correctly use it.

• rust-libp2ps use of of poll-style futures forces a steep learning curve onto users, as they have to both understand rust-libp2p and Rust's async programming model. One concrete example is implementing NetworkBehaviour::poll. Most implementation require a combination of local queues as well as synchronized channels. Making sure to be woken up once progress can be made before returning Poll::Pending can be a struggle.

• rust-libp2p does not provide a peer manager.

• There is little to no guidance on best practices. While rust-libp2p is very versatile, newcomers are missing directions. Instead they often have to read source code of other projects using rust-libp2p.

• There is no built-in instrumentation (e.g. Prometheus metrics) in rust-libp2p.

• Simple protocols like multiple requests and responses send on a single substream in a sequential manner are overly complicated to implement. More abstractions like libp2p-request-response would help.

• No support for packet based transport protocols, requiring hacks e.g. to implement libp2p-mdns.

Action items

While I think some of the pain points above are inherent to rust-libp2p's problem space itself I do see room for improvement. Many of these issues have been raised in the past. The list below should help us prioritize issues in the future.

• Current discussions on how to improve the libp2p-swarm architecture is happening in Review of NetworkBehaviour #2012.

• Simplifications of the rust-libp2p interface e.g. through async and await have been discussed in Reduce Generics - Hide implementation details. #1976.

• Discussions on peer management can be found on [Question] Peer management / disconnection #1090.

• Open metrics wrapper for libp2p-swarm is detailed in Provide Open Metrics (/Prometheus) Wrapper #2013.

• Another common abstraction implementation next to libp2p-request-response is proposed in protocols/streaming-response #1947.

• Support for datagram based transport protocols is proposed in Add traits that abstracts IO operations for datagram-based transport #1357.

With all the negative (though constructive) feedback above, let me finish off by emphasizing that overall people expressed their gratitude for libp2p in general and rust-libp2p in specific and are planning to build on top of rust-libp2p in the future.