TSC Meeting Zoom link
Meeting Chat File
Attendees & Representation. Please add your name to the attendance table below.
LF Staff: LJ Illuzzi
Agenda
Minutes/Updates
Cross-platform signing proposal:
Details:
Matteo Croce from Microsoft joined today’s TSC call (excellent notes here, thanks to Daniel’s speedy fingers) to introduce and discuss his BPF patch to support cross-platform signing. Unfortunately, several key people were unable to make it to today’s call (that’s the holidays for you), so Matteo was able to introduce his patch but we weren’t able to get into a deep conversation about it.
A summary of today’s discussion: Matteo’s patch is cross-platform and will allow for signed BPF programs to be distributed remotely. Soon after Matteo’s patch, Alexei (maintainer of BPF) sent over a separate patch for signing BPF programs. It relies on an approve-list(*) as well as on Linux’s fs verity (meaning it’s not cross-platform).
The full discussion is in the recording of today’s call. Please give it a listen.
We were going to have an in-depth discussion of this be the topic of the next TSC call, but it seems Matteo’s patch is time-sensitive. Since the next call is January 5th, we’ll need people to have a look at the patch, the conversation in response to it, and then come up with an opinion based upon L3AF and its needs. That opinion should be expressed in the conversation on the patch.
- Cross platform signing
- DaveT: Was anybody able to review the patch.
- Brian: Went through the conversation
- DaveT: Topic will be discussed in the eBPF foundation BSC meeting. 1 Week from today L3AF will be presenting. Next meeting - design of signing needs to be cross-platform.
- Two proposals:
- Matteo's - cross-platform, very well aligned with L3AFd.
- Would be helpful if the L3AF community supported this proposal
- Other - approved list of binaries (Linux centric)
- Can load anything that is on the authorized list.
- Does not meet L3AF or eBPF for Windows needs.
- Would be fine if both were merged
- DaveT: Cisco's (Chris) opinion would be very helpful
- Weigh in on the Linux discussion group and on the BSC call.
- Karan could add a bullet point to presentation - collective opinion of the L3AF community.
- Brian: Add a point in your document about this?
- Matteo's original patch was a config option to add only signed programs.
- Alexi's other patch is moving forward
- John Fastabend (on Linux discussion) and Luca agreed that the features needed by MSFT could be implemented inside of libBPF and as an eBPF program
- This conversation ended on Dec. 9th (Before Matteo presented at L3AF)
- DaveT: Meeting with Matteo after this call
- Brian: L3AF could include the signing eBPF program as part of its eBPF program chain. (According to discussion on Linux group)
- Vicky: Invite Matteo to next weeks meeting.
- Have L3AF call next week to discuss signing before BSC meeting.
- Louis: Will not be at the L3AF call next week but will give the keys to an appropriate host.
- Brian: L3AF Kernel Marketplace
- https://github.com/l3af-project/l3af-arch/discussions/9
- DaveT suggests adding this as a PR for line-level comments (Brian will do)
- DaveT: Kernel functions only diss-allows eBPF programs that can be uploaded to NICs. Suggest a name change.
- Vicky: Suggest package manager as a concept for the name. Define broadly. Names have power.
- DaveT: The name implies scope.
- Brian: What should we name it?
- eBPF is difficult to say and will probably need an acronym.
- Vicky: eBPF Package Manager == EPM
- Karan: EPM / eBPF package manager does make a lot of sense, in terms of scope
- Brian: is the Kernel Function Marketplace part of the L3AF project?
- May make sense to migrate to its own project.
- In the future a platform agnostic place may be apropos for the EPM
- Vicky: L3AF could be its initial client. This could really help L3AF. Define it as something standardized that a package manager can use.
- This way the EPM would be a force to increase L3AF adoption and help us push towards standardization for both EPM and L3AF.
- DaveT: Benefits to both ways of doing this:
- Inside L3AF then it is closely located with all the other parts of L3AF. This could help widen the scope of L3AF.
- Outside L3AF then it can include things that do not work with the current version of L3AF.
- There isn't a BSC opinion yet. It is forming now.
- Distinguish between L3AFd and eBPF.
- Answer: What is the L3AF project?
- Today it is the L3AFd, but in the future we will expand scope.
- Vicky: EPM should be outside L3AF because there will be others working on it.
- DaveT: Is it part of one of these or both?
- Thing that LF sanctions - L3AFP (legal entity)
- L3AFp - Github repo
- DaveT: eBPF code signing portion in additional bullet point in the lifecycle management section.
- Brian: 2 different layers of signing
- Package contribs of compiled source code (signed). This is app layer packaging.
- Signing of eBPF programs.
- Doc only currently talks about package signing
- DaveT: Please put that in proposal.
- Some cases where signing should be done by author, others signed by the repository.
- Brian: Initial version Github repo may be sufficient
- Assumes that everyone will be okay pushing their code to a L3AF Project repo
- Revisions could be tested and reviewed by L3AF team
- DaveT: Requiring manual review? Good/Bad?
- Brian: Short term - no manual vetting
- We currently do not have automatic review
- DaveT: Requirement to have automated review.
- Vicky: Marketplace needs manual review for safety.
- DaveT: Manual review could be optional.
- Vicky: Is part of the review going to be for security.
- Automatic review - definitely. Manual reviews - maybe. (at the start)
- DaveT: Notion of private repo
- Jason: For startup we need manual review
- Brian: Hosting source code or packages
- Source code - versioning etc.
- Package or archive - what is needed to run the program along with docs
- Karan: Please review doc. We will discuss in next meeting.
- This is the area where we need support from the community.
- Brian: Will put up the pull request today.
- Louis: LEAF session is 8:15 ET will this work?
- Daniel: Will check with Poorna
- Need email for presenters.
- LFN induction - Need a separate meeting to discuss this
- Needs a lot of community input.
- General agreement.
*** Minutes from 12/15/2021 ***
PEN request
- Raga: Schedule call after this meeting
- Dave: What layer of the org is the PEN assigned to?
- L3AFD project, LF, L3AFP?
- Louis: Just put it under LFN. Ramnifications?
- Dave: Next PEN would probably have a sub-delegation under the original PEN
- This is how MSFT does this so that there is a single management point.
- Lous: Difference between L3AFd and L3AFP?
- Dave: Github organization with different GitHub repos under it.
- Louis: PEN that covers L3AF as a project and all current and future repos?
- Dave: Common use of PENS is for OID and PEN is inserted into the OID with arbitrary number of layers underneath.
- OIDs are used in x.509 certificats, etc.
- Lous: How are we going to use the PEN for L3AF? Want a PEN that covers all of L3AF, but not all of LF or LFN,
- Raga: As part of flow exporter we will add custom field support identified with PEN number.
- Dave: Inventing a new slot that fields are going to go in, can it be just an array of fields of integers?
- Raga: Will dig more and find out?
- Dave: It matters to how we will fill out the application.
- Raga: This is a requirement for the flow_exporter. We will not need another PEN number for L3AF.
- Lous: Will reconsult with legal and set up call with Raga. Review by email with rest of L3AF team (if this is doable).
- Dave: It's just a simple web form and cannot be pending
- Dave: Do we use LF, L3AFP or L3AFd for the name?
- Lous: Will discuss with legal.
- Dev testing forum
- Set date and time for MSFT presentation
- Cross platform signing
- Matteo: No way to do cross platform signing with eBPF programs
- Implementation that allows loading eBPF programs to kernel that takes care of relocation
- Created patch that does this. Creates eBPF prog and adds sig to it.
- Dave: talked about 3 peices in kernel function marketplace
- Orchestrator pulls stuff from marketplace
- Can you put signed programs in the marketplace
- We are discussing an option that allows remote distribution and is compatible with L3AF.
- The other approach does not play well with the L3AF vision that we have discussed.
- Vicky: Do we have representation as the L3AF at the kernel level where these decisions will be made?
- Dave: Need Karan, Chris, ect. If we had a collective decision it would carry more weight.
- This could be a call to create new contacts.
- Dave: This discussion is happening on the Linux kernel list
- MSFT would like cross-platform
- Move to the eBPF foundation (which is cross-platform)?
- Dave: Next BPF steering committee meeting would like L3AF to present.
- Dave: BSC does not officially have an answer if the meeting is open.
- Still have time to ask. Should be a yes answer (at least for this meeting)
- Matteo: Proposed to BPF ML & then another solution appeared from the BPF maintainer
- Very different solution: create an approve-list of programs that can load BPF programs
- Only allow programs loaded from progs on this approve-list
- Suspects this solution won't be cross-platform: verification requires Linux fs verity method
- Also allows L3AFd to install anything it wants if L3AFd is in the allow list, Could be a security flaw
- Matteo's approach allows individual signing and allows individual verfication, reputation, etc.
- Raga: Where is the signature exactly? Do you still have the verification step on signed programs? Use case please.
- Matteo: XDP. SOme BPF programs take actions on packets. These can be loaded and attached to network drivers.
- Malicious programs can mangle pacet traffic (very dnagerous). Must make sure that program is safe.
- Dave: Big value add: signing instead of verification step.
- Verification step can be CPU intensive. Signature check is cheap.
- Verification and signing together does not give this benefit. This is what the patch does.
- Raga: Does this work for UM progs also? Yes.
- Dave: Other approach with white list? How is this different from cap BPF?
- Matteo: Whitelist enforces Cap BPF.
- Dave: L3AFd pushes out both kernel function as well as a program that can use the kernel function.
- Matteo: Whitelist is a list programs that can be loaded
- Matteo: Sig verification is before verification check.
- Dave: Also reduces DOS style attacks.
- If sig check fails then verification does not run and waste cycles.
- Santhosh: Verifier runs only once at load time.
- Dave: Yes, but you can spin the loader.
- Dave: That is the intro.
- If we can get several orgs to support this then we can approach the BSC.
- Vicky: Once the video for the call is available we can take this to the mailing list.
- Lous: Cancelling next weeks call on the 22nd?
- Dave: Nope, on vacation.
- Vicky: Probably most people have made plans so Jan 5th would be better.
- Matteo: PR is urgent for MSFT because we want a signature system.
- It's too dangerous to load untrusted BPF programs
- Dave: Please post opinions on Linux Kernel mailing list sooner rather than later.
- Dave: Include signing into the BSC meeting on Jan 12th at 1PM PST.
- Lous: Please register for Dev and Test Forum
Action Items
- Schdule Dev & Testing Forum L3AF session (LJ/Daniel/Poorna)
- Schedule call with Raga to fill out the PEN application. (LJ/Raga)
- Ask BSC if the meeting can be open to the public (Dave)
- Vicky: Will post to mailing list so that people can discuss signing on list after watching video.
Future Agenda Items
***** Minutes from previous call *****