Skip to content

Commit 69a1b95

Browse files
committed
Initial support for Load/Attach Split
Includes support for Cluster-Scoped XDP, TXC, and Fentry Programs This commit introduces the foundation for the load/attach split, including: - Updates to the `BpfApplication` CRD to support a separate list of optional attach points for programs. This allows programs to be loaded before attachments are made and enables dynamic attachment updates. - An initial version of the `BpfApplicationState` CRD to manage per-node information for a single `BpfApplication`. - Proof of concept and initial implementation for cluster-scoped XDP, TCX, and Fentry programs, with working unit tests and samples. See TODO.md for more info. Signed-off-by: Andre Fredette <[email protected]>
1 parent d5ffe86 commit 69a1b95

File tree

138 files changed

+14202
-5033
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

138 files changed

+14202
-5033
lines changed

Makefile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -208,8 +208,8 @@ COMMON_FLAGS ?= ${VERIFY_FLAG} --go-header-file $(shell pwd)/hack/boilerplate.go
208208
.PHONY: manifests
209209
manifests: controller-gen ## Generate WebhookConfiguration, ClusterRole and CustomResourceDefinition objects.
210210
$(CONTROLLER_GEN) crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
211-
$(CONTROLLER_GEN) rbac:roleName=agent-role paths="./controllers/bpfman-agent/..." output:rbac:artifacts:config=config/rbac/bpfman-agent
212-
$(CONTROLLER_GEN) rbac:roleName=operator-role paths="./controllers/bpfman-operator" output:rbac:artifacts:config=config/rbac/bpfman-operator
211+
$(CONTROLLER_GEN) rbac:roleName=agent-role paths="./controllers/bpfman-agent/...;./controllers/app-agent/..." output:rbac:artifacts:config=config/rbac/bpfman-agent
212+
$(CONTROLLER_GEN) rbac:roleName=operator-role paths="./controllers/bpfman-operator;./controllers/app-operator" output:rbac:artifacts:config=config/rbac/bpfman-operator
213213

214214
.PHONY: generate
215215
generate: manifests generate-register generate-deepcopy generate-typed-clients generate-typed-listers generate-typed-informers ## Generate ALL auto-generated code.

PROJECT

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,4 +129,11 @@ resources:
129129
kind: BpfNsApplication
130130
path: github.com/bpfman/bpfman-operator/apis/v1alpha1
131131
version: v1alpha1
132+
- api:
133+
crdVersion: v1
134+
controller: true
135+
domain: bpfman.io
136+
kind: BpfApplicationState
137+
path: github.com/bpfman/bpfman-operator/apis/v1alpha1
138+
version: v1alpha1
132139
version: "3"

TODO.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
The code has support for XDP, TCX, and Fentry programs in a BpfApplication
2+
3+
The new code is mainly in these directories:
4+
5+
Updated & working APIs:
6+
7+
- apis/v1alpha1/fentryProgram_types.go
8+
- apis/v1alpha1/xdpProgram_types.go
9+
- apis/v1alpha1/tcxProgram_types.go
10+
- apis/v1alpha1/bpfApplication_types.go
11+
- apis/v1alpha1/bpfApplicationState_types.go
12+
13+
Note: the rest are partially updated.
14+
15+
New Agent:
16+
17+
- controllers/app-agent
18+
19+
New Operator:
20+
21+
- controllers/app-operator
22+
23+
Note: I left the old bpfman-agent and bpfman-operator code unchanged (except as needed due to CRD changed). It should work, but it's not being initialized when we run the operator.
24+
25+
Testing:
26+
27+
- Unit tests for the agent and the operator
28+
- The following working samples:
29+
- config/samples/bpfman.io_v1alpha1_bpfapplication.yaml
30+
- config/samples/bpfman.io_v1alpha1_fentry_bpfapplication.yaml
31+
32+
TODO:
33+
34+
- I want to redo the status/condition values. I’m currently using the existing framework and some values for status/conditions, but I intend to create a new set of conditions/status values that make more sense with the new design.
35+
- Review all comments and logs.
36+
- Maybe make more code common.
37+
- Support the rest of the program types (including namespace-scoped CRDs).
38+
- Integrate with the new bpfman code with load/attach split (of course)
39+
- Lots more testing.
40+
- Lots of other stuff (I'm sure).
Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
/*
2+
Copyright 2023 The bpfman Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package v1alpha1
18+
19+
import (
20+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
21+
metav1types "k8s.io/apimachinery/pkg/types"
22+
"sigs.k8s.io/controller-runtime/pkg/client"
23+
)
24+
25+
// BpfApplicationProgramState defines the desired state of BpfApplication
26+
// +union
27+
// +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'XDP' ? has(self.xdp) : !has(self.xdp)",message="xdp configuration is required when type is XDP, and forbidden otherwise"
28+
// // +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'TC' ? has(self.tc) : !has(self.tc)",message="tc configuration is required when type is TC, and forbidden otherwise"
29+
// +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'TCX' ? has(self.tcx) : !has(self.tcx)",message="tcx configuration is required when type is TCX, and forbidden otherwise"
30+
// +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'Fentry' ? has(self.fentry) : !has(self.fentry)",message="fentry configuration is required when type is Fentry, and forbidden otherwise"
31+
// // +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'Fexit' ? has(self.fexit) : !has(self.fexit)",message="fexit configuration is required when type is Fexit, and forbidden otherwise"
32+
// // +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'Kprobe' ? has(self.kprobe) : !has(self.kprobe)",message="kprobe configuration is required when type is Kprobe, and forbidden otherwise"
33+
// // +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'Kretprobe' ? has(self.kretprobe) : !has(self.kretprobe)",message="kretprobe configuration is required when type is Kretprobe, and forbidden otherwise"
34+
// // +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'Uprobe' ? has(self.uprobe) : !has(self.uprobe)",message="uprobe configuration is required when type is Uprobe, and forbidden otherwise"
35+
// // +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'Uretprobe' ? has(self.uretprobe) : !has(self.uretprobe)",message="uretprobe configuration is required when type is Uretprobe, and forbidden otherwise"
36+
// // +kubebuilder:validation:XValidation:rule="has(self.type) && self.type == 'Tracepoint' ? has(self.tracepoint) : !has(self.tracepoint)",message="tracepoint configuration is required when type is Tracepoint, and forbidden otherwise"
37+
type BpfApplicationProgramState struct {
38+
// ProgramAttachStatus records whether the program should be loaded and whether
39+
// the program is loaded.
40+
ProgramAttachStatus BpfProgramConditionType `json:"programattachstatus"`
41+
42+
// ProgramId is the id of the program in the kernel. Not set until the
43+
// program is loaded.
44+
// +optional
45+
ProgramId *uint32 `json:"program_id"`
46+
47+
// Type specifies the bpf program type
48+
// +unionDiscriminator
49+
// +kubebuilder:validation:Required
50+
// +kubebuilder:validation:Enum:="XDP";"TC";"TCX";"Fentry";"Fexit";"Kprobe";"Kretprobe";"Uprobe";"Uretprobe";"Tracepoint"
51+
Type EBPFProgType `json:"type,omitempty"`
52+
53+
// xdp defines the desired state of the application's XdpPrograms.
54+
// +unionMember
55+
// +optional
56+
XDP *XdpProgramInfoState `json:"xdp,omitempty"`
57+
58+
// // tc defines the desired state of the application's TcPrograms.
59+
// // +unionMember
60+
// // +optional
61+
// TC *TcProgramInfoState `json:"tc,omitempty"`
62+
63+
// tcx defines the desired state of the application's TcxPrograms.
64+
// +unionMember
65+
// +optional
66+
TCX *TcxProgramInfoState `json:"tcx,omitempty"`
67+
68+
// fentry defines the desired state of the application's FentryPrograms.
69+
// +unionMember
70+
// +optional
71+
Fentry *FentryProgramInfoState `json:"fentry,omitempty"`
72+
73+
// // fexit defines the desired state of the application's FexitPrograms.
74+
// // +unionMember
75+
// // +optional
76+
// Fexit *FexitProgramInfoState `json:"fexit,omitempty"`
77+
78+
// // kprobe defines the desired state of the application's KprobePrograms.
79+
// // +unionMember
80+
// // +optional
81+
// Kprobe *KprobeProgramInfoState `json:"kprobe,omitempty"`
82+
83+
// // kretprobe defines the desired state of the application's KretprobePrograms.
84+
// // +unionMember
85+
// // +optional
86+
// Kretprobe *KprobeProgramInfoState `json:"kretprobe,omitempty"`
87+
88+
// // uprobe defines the desired state of the application's UprobePrograms.
89+
// // +unionMember
90+
// // +optional
91+
// Uprobe *UprobeProgramInfoState `json:"uprobe,omitempty"`
92+
93+
// // uretprobe defines the desired state of the application's UretprobePrograms.
94+
// // +unionMember
95+
// // +optional
96+
// Uretprobe *UprobeProgramInfoState `json:"uretprobe,omitempty"`
97+
98+
// // tracepoint defines the desired state of the application's TracepointPrograms.
99+
// // +unionMember
100+
// // +optional
101+
// Tracepoint *TracepointProgramInfoState `json:"tracepoint,omitempty"`
102+
}
103+
104+
// BpfApplicationSpec defines the desired state of BpfApplication
105+
type BpfApplicationStateSpec struct {
106+
// The number of times the BpfApplicationState has been updated. Set to 1
107+
// when the object is created, then it is incremented prior to each update.
108+
// This allows us to verify that the API server has the updated object prior
109+
// to starting a new Reconcile operation.
110+
UpdateCount int64 `json:"updatecount"`
111+
// AppLoadStatus reflects the status of loading the bpf application on the
112+
// given node.
113+
AppLoadStatus BpfProgramConditionType `json:"apploadstatus"`
114+
// Programs is a list of bpf programs contained in the parent application.
115+
// It is a map from the bpf program name to BpfApplicationProgramState
116+
// elements.
117+
Programs map[string]BpfApplicationProgramState `json:"programs,omitempty"`
118+
}
119+
120+
// +genclient
121+
// +genclient:nonNamespaced
122+
// +kubebuilder:object:root=true
123+
// +kubebuilder:subresource:status
124+
// +kubebuilder:resource:scope=Cluster
125+
126+
// BpfApplicationState contains the per-node state of a BpfApplication.
127+
// ANF-TODO: I can't get the Node to display in the kubectl output.
128+
// // +kubebuilder:printcolumn:name="Node",type=string,JSONPath=".metadata.labels['kubernetes.io/hostname']"
129+
// +kubebuilder:printcolumn:name="Status",type=string,JSONPath=`.status.conditions[0].reason`
130+
// +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp"
131+
type BpfApplicationState struct {
132+
metav1.TypeMeta `json:",inline"`
133+
metav1.ObjectMeta `json:"metadata,omitempty"`
134+
135+
Spec BpfApplicationStateSpec `json:"spec,omitempty"`
136+
Status BpfAppStatus `json:"status,omitempty"`
137+
}
138+
139+
// +kubebuilder:object:root=true
140+
// BpfApplicationStateList contains a list of BpfApplicationState objects
141+
type BpfApplicationStateList struct {
142+
metav1.TypeMeta `json:",inline"`
143+
metav1.ListMeta `json:"metadata,omitempty"`
144+
Items []BpfApplicationState `json:"items"`
145+
}
146+
147+
func (an BpfApplicationState) GetName() string {
148+
return an.Name
149+
}
150+
151+
func (an BpfApplicationState) GetUID() metav1types.UID {
152+
return an.UID
153+
}
154+
155+
func (an BpfApplicationState) GetAnnotations() map[string]string {
156+
return an.Annotations
157+
}
158+
159+
func (an BpfApplicationState) GetLabels() map[string]string {
160+
return an.Labels
161+
}
162+
163+
func (an BpfApplicationState) GetStatus() *BpfAppStatus {
164+
return &an.Status
165+
}
166+
167+
func (an BpfApplicationState) GetClientObject() client.Object {
168+
return &an
169+
}
170+
171+
func (anl BpfApplicationStateList) GetItems() []BpfApplicationState {
172+
return anl.Items
173+
}

apis/v1alpha1/bpfApplication_types.go

Lines changed: 7 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -129,16 +129,11 @@ type BpfApplicationProgram struct {
129129
type BpfApplicationSpec struct {
130130
BpfAppCommon `json:",inline"`
131131

132-
// Programs is a list of bpf programs supported for a specific application.
133-
// It's possible that the application can selectively choose which program(s)
134-
// to run from this list.
135-
// +kubebuilder:validation:MinItems:=1
136-
Programs []BpfApplicationProgram `json:"programs,omitempty"`
137-
}
138-
139-
// BpfApplicationStatus defines the observed state of BpfApplication
140-
type BpfApplicationStatus struct {
141-
BpfProgramStatusCommon `json:",inline"`
132+
// Programs is the list of bpf programs in the BpfApplication that should be
133+
// loaded. The application can selectively choose which program(s) to run
134+
// from this list based on the optional attach points provided. The list is
135+
// implemented as a map from the bpf function name to BpfApplicationProgram.
136+
Programs map[string]BpfApplicationProgram `json:"programs,omitempty"`
142137
}
143138

144139
// +genclient
@@ -155,8 +150,8 @@ type BpfApplication struct {
155150
metav1.TypeMeta `json:",inline"`
156151
metav1.ObjectMeta `json:"metadata,omitempty"`
157152

158-
Spec BpfApplicationSpec `json:"spec,omitempty"`
159-
Status BpfApplicationStatus `json:"status,omitempty"`
153+
Spec BpfApplicationSpec `json:"spec,omitempty"`
154+
Status BpfAppStatus `json:"status,omitempty"`
160155
}
161156

162157
// +kubebuilder:object:root=true

apis/v1alpha1/bpfNsApplication_types.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ type BpfNsApplication struct {
8585
metav1.ObjectMeta `json:"metadata,omitempty"`
8686

8787
Spec BpfNsApplicationSpec `json:"spec,omitempty"`
88-
Status BpfApplicationStatus `json:"status,omitempty"`
88+
Status BpfAppStatus `json:"status,omitempty"`
8989
}
9090

9191
// +kubebuilder:object:root=true

apis/v1alpha1/fentryProgram_types.go

Lines changed: 32 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -37,9 +37,8 @@ type FentryProgram struct {
3737
metav1.TypeMeta `json:",inline"`
3838
metav1.ObjectMeta `json:"metadata,omitempty"`
3939

40-
Spec FentryProgramSpec `json:"spec"`
41-
// +optional
42-
Status FentryProgramStatus `json:"status,omitempty"`
40+
Spec FentryProgramSpec `json:"spec"`
41+
Status BpfAppStatus `json:"status,omitempty"`
4342
}
4443

4544
// FentryProgramSpec defines the desired state of FentryProgram
@@ -52,13 +51,23 @@ type FentryProgramSpec struct {
5251
// FentryProgramInfo defines the Fentry program details
5352
type FentryProgramInfo struct {
5453
BpfProgramCommon `json:",inline"`
55-
// Function to attach the fentry to.
56-
FunctionName string `json:"func_name"`
54+
FentryLoadInfo `json:",inline"`
55+
// Whether the program should be attached to the function.
56+
// This may be updated after the program has been loaded.
57+
// +optional
58+
FentryAttachInfo `json:",inline"`
5759
}
5860

59-
// FentryProgramStatus defines the observed state of FentryProgram
60-
type FentryProgramStatus struct {
61-
BpfProgramStatusCommon `json:",inline"`
61+
// FentryLoadInfo contains the program-specific load information for Fentry
62+
// programs
63+
type FentryLoadInfo struct {
64+
// FunctionName is the name of the function to attach the Fentry program to.
65+
FunctionName string `json:"function_name"`
66+
}
67+
68+
type FentryAttachInfo struct {
69+
// Whether the bpf program should be attached to the function.
70+
Attach bool `json:"attach"`
6271
}
6372

6473
// +kubebuilder:object:root=true
@@ -68,3 +77,18 @@ type FentryProgramList struct {
6877
metav1.ListMeta `json:"metadata,omitempty"`
6978
Items []FentryProgram `json:"items"`
7079
}
80+
81+
type FentryProgramInfoState struct {
82+
// The list of points to which the program should be attached. For Fentry
83+
// programs, there will be at most one attach point, but it's being
84+
// maintained as a list for consistency with most of the other program
85+
// types.
86+
// +optional
87+
AttachPoints []FentryAttachInfoState `json:"attach_points"`
88+
}
89+
90+
type FentryAttachInfoState struct {
91+
AttachInfoCommon `json:",inline"`
92+
// FunctionName is the name of the function to attach the Fentry program to.
93+
FunctionName string `json:"function_name"`
94+
}

apis/v1alpha1/fexitProgram_types.go

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -37,9 +37,8 @@ type FexitProgram struct {
3737
metav1.TypeMeta `json:",inline"`
3838
metav1.ObjectMeta `json:"metadata,omitempty"`
3939

40-
Spec FexitProgramSpec `json:"spec"`
41-
// +optional
42-
Status FexitProgramStatus `json:"status,omitempty"`
40+
Spec FexitProgramSpec `json:"spec"`
41+
Status BpfAppStatus `json:"status,omitempty"`
4342
}
4443

4544
// FexitProgramSpec defines the desired state of FexitProgram
@@ -52,13 +51,22 @@ type FexitProgramSpec struct {
5251
// FexitProgramInfo defines the Fexit program details
5352
type FexitProgramInfo struct {
5453
BpfProgramCommon `json:",inline"`
55-
// Function to attach the fexit to.
56-
FunctionName string `json:"func_name"`
54+
FexitLoadInfo `json:",inline"`
55+
// Whether the program should be attached to the function.
56+
// This may be updated after the program has been loaded.
57+
// +optional
58+
FexitAttachInfo `json:",inline"`
59+
}
60+
61+
// FexitLoadInfo contains the program-specific load information for Fexit
62+
// programs
63+
type FexitLoadInfo struct {
64+
// FunctionName is the name of the function to attach the Fexit program to.
65+
FunctionName string `json:"function_name"`
5766
}
5867

59-
// FexitProgramStatus defines the observed state of FexitProgram
60-
type FexitProgramStatus struct {
61-
BpfProgramStatusCommon `json:",inline"`
68+
type FexitAttachInfo struct {
69+
Attach bool `json:"attach"`
6270
}
6371

6472
// +kubebuilder:object:root=true

0 commit comments

Comments
 (0)