The goal of this document is to provide an overview of our plan to open source L3AF.
What is L3AF?
L3AF is an ecosystem to share, orchestrate, and run eBPF kernel programs and their associated userspace applications. This includes BPF programs for networking, observabilty, and tracing. Multiple eBPF programs are executed in a chain.
What will be open sourced?
Our goal is to open source all L3AF code. Our plan is to open source the networking-related L3AF code first.
The scope of the remainder of this document will be limited to the open sourcing of the network related portion of the L3AF ecosystem.
Review
This diagram is an overview of the network related portion of the L3AF ecosystem as of Q4 FY21.
Vision
Our vision is to provide a user-friendly eBPF program orchestration ecosystem that consists of:
- The L3AF orchestration tool (i.e., L3AFD and its control plane). This is written in Go. This software is the common platform used to orchestrate eBPF kernel functions (KFs).
- The eBPF kernel functions (written in C) and any associated userspace programs. These programs should be independent, modular, and composable.
- A kernel function marketplace where users can share eBPF programs and any associated userspace programs.
Key Deliverables
These are the essential deliverables for the project.
Alpha Release - (Milestone 1)
We believe these to be the minimum changes required before sharing our code to anyone outside of Walmart.
The plan is for the Alpha release to be available for select contributors within the Linux Foundation.
Control Plane Improvements
There are some control plane improvements that we would like to have in production; Santhosh Fernandes has already started on these improvements.
Summary
Currently there are some major limitations in our existing control plane:
- All KFs must be stopped whenever first non-root BPF program in the chain needs modified. This is because the root program does not read updated map information after it is initially loaded.
- KF and its userspace program must be restarted for configuration changes. This is because we pass configuration information as command-line arguments to the userspace programs (which load their associated KFs).
We are addressing these limitations by making the respective changes:
- Creating a "passthrough" BPF program as the first non-root BPF program in the chain
- Communicate configuration changes using shared maps
Estimate
2MM
Code Cleanup
Summary
Minor code cleanup including:
- License and copyright information in source code
- LICENSE and README files
- Removing any information specific to Walmart (secrets, URLs, etc)
- Code comments
- Minor refactoring (e.g., fix anything embarrassing
) - Trivial refactoring (e.g., fix typos, naming improvements, fix whitespace)
Estimate
1MM
Open Source Developer Guide Documentation
Summary
Provide documentation and minimal tooling necessary for open source contributors to get involved quickly.
This does not include full product documentation.
Estimate
0.5MM
Beta Release - (Milestone 2)
We believe these would be the minimum necessary changes for external users to begin using L3AF.
Config Distribution Changes
Summary
Currently L3AF uses the Torbit configuration distribution pipeline of Admind → MySQL → CDB Generator → CDB Push → CDB Pull → inotify → atomic reload.
We need to provide a configuration distribution pipeline that is open source and familiar to users. We also want to migrate to the new configuration distribution pipeline internally.
L3afd should be made so that different types of configuration management approaches are possible (perhaps similar to Envoy).
Estimate
3MM
API Changes
Summary
Currently the L3AF API is tied to AdminD (and its authentication, data models, CDBs, third-party dependencies, etc.). We need to implement our own API service. TODO: we wanna use this in Walmart, too!
Estimate
1MM
CLI/GUI
Summary
L3AF currently relies on the Swagger REST API documentation tool as its UI. Our current approach requires us to write raw JSON into the Swagger UI where the command is submitted.
We should provide a legitimate CLI and/or GUI for users.
Estimate
1MM for CLI
5MM for GUI
GA Release - (Milestone 3)
We believe these changes are required to fulfill our full vision open sourcing this project. This is the release that we really want to hype.
Kernel Function Marketplace Readiness
Summary
Create an ecosystem for the open source community to share KFs, which we will build and host for users to download.
Estimate
3MM
Move All KF Chaining Logic to L3AFD
(Linux Kernel >=5.10)
Summary
Currently the logic to chain multiple KFs exists in each KF's code. This has some downsides:
- KF developers must include boilerplate code for the chain to function properly
- Complex code is required in L3AFD to orchestrate the chaining
We can use new and existing 3rd party libraries and tools to move all KF chaining logic to L3AFD; this will decouple the chaining logic from the KFs. The new approach will require porting a 3rd party library (libxdp) to Go.
Estimate
4MM
Support Go Userspace Plugins
Summary
Currently we execute userspace programs as separate binaries and we mostly lose communication and control with the userspace program. We want to improve this situation by allowing userspace programs to be written as Go RPC plugins, which will allow for built-in logging (from L3AFD), bidirectional communication, establishing a common interface for userspace programs, and health checking.
Estimate
1MM
Other Deliverables
These are other important deliverables for the project.
Automated Build and Testing
Summary
Automate building and testing of the entire ecosystem.
Estimate
2MM