PFQ: Functional Network Framework for Multi-Core Architectures

by do son · Published September 8, 2019 · Updated September 8, 2019

PFQ

PFQ is a functional framework designed for the Linux operating system built for efficient packets capture/transmission (10G, 40G and beyond), in-kernel functional processing, kernel-bypass and packets steering across groups of sockets/end-points.

It is highly optimized for multi-core architecture, as well as for network devices equipped with multiple hardware queues. Compliant with any NIC, it provides a script that generates accelerated network device drivers starting from the source code.

PFQ enables the development of high-performance network applications, and it is shipped with a custom version of libpcap that accelerate and parallelize legacy applications. Besides, a pure functional language designed for early stages in-kernel packet processing is included: pfq-lang.

Pfq-Lang is inspired by Haskell and is intended to define applications that run on top of network device drivers. Through pfq-lang it is possible to build efficient bridges, port mirrors, simple firewalls, network balancers and so forth.

The framework includes the source code of the PFQ kernel module, user-space libraries for C, C++11-14, Haskell language, an accelerated pcap library, an implementation of pfq-lang as eDSL for C++/Haskell, an experimental pfq-lang compiler and a set of diagnostic tools.

Features

Data-path with full lock-free architecture.
Preallocated pools of socket buffers.
Compliant with a plethora of network devices drivers.
Rx and Tx line-rate on 10-Gbit links (14,8 Mpps), tested with Intel ixgbe vanilla drivers.
Transparent support of kernel threads for asynchronous packets transmission.
Transmission with active timestamping.
Groups of sockets which enable concurrent monitoring of multiple multi-threaded applications.
Per-group packet steering through randomized hashing or deterministic classification.
Per-group Berkeley and VLAN filters.
User-space libraries for C, C++11-14 and Haskell language.
Functional engine for in-kernel packet processing with pfq-lang.
pfq-lang eDLS for C++11-14 and Haskell language.
pfq-lang compiler used to parse and compile pfq-lang programs.
Accelerated pcap library for legacy applications (line-speed tested with captop).
I/O user<->kernel memory-mapped communications allocated on top of HugePages.
pfqd daemon used to configure and parallelize (pcap) legacy applications.
pfq-omatic script that automatically accelerates vanilla drivers.