-
Notifications
You must be signed in to change notification settings - Fork 263
[feat] Prefix on NIC v6 windows support #4060
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
[feat] Prefix on NIC v6 windows support #4060
Conversation
* add iptables block to signed image * fix syntax * mariner version
* Prefix on NIC v6 support * update dedup comment in ipam * add log for a gatway error scenario * handle synchostversion sync on pon swiftv2 nic * handle gatewayip parse failure or nil scenario * remove unused code Address PR comments remove unnecessary logger update synhost skip test scenario fix linting error updated lint error fix linting error . update synhost skip test scenario fix linting error updated lint error fix linting error . * add test scenario * update test to handle gatway nil case * remove synchost skip and single ip skip . . * Add missing test scenarios * fix ipam_test conflicts * fix conflits small fixes fix lint errors lint fix . Delete examples/imds_nc_lookup.go Signed-off-by: NihaNallappagari <[email protected]> fix lint . * fix linting * add todo and remove multiple interface scenario * Skip add primary ip's to secondary config for swiftv2 pon scenarios fix lint error . * code review comment * Update gateway ipv6 to use default value, that auto detects and adds to neigh table * remove unwanted changes * address ipfamily comment * fix test failure * Handle nilgateway scenario * remove unnedded else block * comments changes --------- Co-authored-by: Kaushik Vuligonda <[email protected]> Co-authored-by: nn052161 <[email protected]>
…re-container-networking into ponv6WindowsSupport
875c0f0 to
4395f17
Compare
|
This pull request is stale because it has been open for 2 weeks with no activity. Remove stale label or comment or this will be closed in 7 days |
|
Pull request closed due to inactivity. |
|
This pull request is stale because it has been open for 2 weeks with no activity. Remove stale label or comment or this will be closed in 7 days |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds Windows support for the SwiftV2 PrefixOnNic scenario by implementing Windows registry configuration functionality. The changes enable CNS to detect SwiftV2 VNETBlock network containers and configure appropriate registry settings for HNS (Host Network Service) on Windows platforms.
Key Changes:
- Added Windows registry manipulation functions to configure PrefixOnNic settings
- Introduced
SwiftV2PrefixOnNicfield to track SwiftV2 VNETBlock containers - Modified IMDS NC retrieval to handle Windows SwiftV2 scenarios and trigger registry configuration
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| cns/restserver/internalapi_windows.go | Implements registry manipulation functions for configuring Windows HNS PrefixOnNic settings |
| cns/restserver/internalapi_test.go | Adds test coverage for Windows SwiftV2 PrefixOnNic scenario in SyncHostNCVersion |
| cns/restserver/internalapi_linux.go | Provides no-op implementation of setPrefixOnNICRegistry for Linux platform |
| cns/restserver/internalapi.go | Refactors getIMDSNCs to detect and configure Windows SwiftV2 scenarios, adds isPrefixonNicSwiftV2 helper |
| cns/kubecontroller/nodenetworkconfig/conversion_linux.go | Sets SwiftV2PrefixOnNic flag based on NC type for static NCs |
| cns/kubecontroller/nodenetworkconfig/conversion.go | Explicitly sets SwiftV2PrefixOnNic to false for dynamic NCs |
| cns/NetworkContainerContract.go | Adds SwiftV2PrefixOnNic field to CreateNetworkContainerRequest |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ca1cb7d to
19a8a62
Compare
39289be to
5e76f7b
Compare
5e76f7b to
aef689e
Compare
| return service.setRegistryValue(hnsRegistryPath, "EnableSNAT", isEnabled) | ||
| } | ||
|
|
||
| func (service *HTTPRestService) setPrefixOnNICRegistry(enablePrefixOnNic bool, infraNicMacAddress string) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why this is required for hns?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
HNS team need to know thinks like infra nic, delegated nic, swiftv2 prefix on nic scenario to probably add some routes specific to the scenario. HNS team might answer better, tagging @surajkrishnan14
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this is required for HNS to setup appropriate rules to forward packets correctly between the 2 interfaces and container in various scenarios, setup SNAT etc.
cns/restserver/internalapi.go
Outdated
|
|
||
| if ncID != "" { | ||
| ncs[ncID] = PrefixOnNicNCVersion // for prefix on nic version scenario nc version is 1 | ||
| } else if runtime.GOOS == "windows" && service.isPrefixonNicSwiftV2() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GOOS=windows not required since you are defining in windows specific file, linux specific calls would be dummy.. i think its better we expose all regkey apis through interface.. @behzad-mir / @rbtr to comment on this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good feedback. IMO any explicit check to GOOS or stubbing an interface in an *_<os>.go file indicates that the abstraction is in the wrong place.
There was a similar discussion here #3286 (comment) which you should read so that I don't have to repeat it all.
The tl;dr is that our abstraction should live at the last boundary of common code before we start splitting things up by OS. E.g. here: if getIMDSNCs is the final call possible before we have to do OS specific things, that is what lives in both the *_linux.go and the *_windows.go.
However - I haven't done a full review of this code, but at first glance getIMDSNCs is probably not the place to be doing OS specific things at all. That advertises that it just getting NCs.
And if we think about duplicating this func in OS-specific files, we end up copying a bunch of common code to both locations that doesn't care about the OS at all. This probably all just means that the current func is overloaded and poorly defined. Consider how to separate things in to the minimum necessary complete behaviors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed the os check and moved windowskey process call form getIMDSNCs . Thanks for the feedback
| return service.setRegistryValue(hnsRegistryPath, "EnableSNAT", isEnabled) | ||
| } | ||
|
|
||
| func (service *HTTPRestService) setPrefixOnNICRegistry(enablePrefixOnNic bool, infraNicMacAddress string) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since we are setting multiple regkeys under same function, can function name be setRegistryKeysForPrefixOnNic and then expose all registry key functions in interface, so that unit test can be written mocking it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Refactored as per suggestion
QxBytes
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could you also add to the pr description some background on prefix on nic (pon, particularly v6-- what PON is meant for/aims to accomplish), what this feature is allowing us to do that we couldn't do before, and what scenarios this affects (ex: standalone, multi/single tenant)
| dockerClient: dc, | ||
| wscli: wscli, | ||
| iptables: iptg, | ||
| windowsRegistry: newRegistryClient(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wondering if we should pass this in like how we do with iptablesGetter for example
|
|
||
| // newLinuxRegistryClient creates a no-op registry client for Linux. | ||
| func newLinuxRegistryClient() windowsRegistryClient { | ||
| return &linuxRegistryClient{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
newLinuxRegistryClient() seems to be called only from newRegistryClient()-- can we just have newRegistryClient() return &linuxRegistryClient directly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@NihaNallappagari I don't think you have understood my earlier feedback on OS abstraction/separations.
A func called newLinuxRegistryClient returning struct linuxRegistryClient as interface windowsRegistryClient is nonsensical. Stubbing Windows only behavior in a Linux struct so that the special Windows code compiles on Linux is bad. This abstraction is wrong.
If you need to make stubs like this, the place that you are calling these stubs is OS specific code already. THAT is what should be refactored in to Windows and Linux versions.
| if ncID != "" { | ||
| ncs[ncID] = PrefixOnNicNCVersion // for prefix on nic version scenario nc version is 1 | ||
| } else { | ||
| infraNicMacAddress = iface.MacAddress.String() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what does having an empty ncID mean and when does this happen? can this be added as a comment for the function? can we also clarify in the comment what mac address we are getting? will the infra nic mac address ever be empty if all the network interfaces have nc ids?
| NCVersion: version, | ||
| }, | ||
| }, | ||
| SwiftV2PrefixOnNic: true, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could you check conversion_test.go passes when OS=windows-- this request is the expected for one of the tests but in the test we set swiftv2 to false
| // Get IMDS NC versions for delegated NIC scenarios. | ||
| imdsNCVersions, infraNicMacAddress := service.getIMDSNCs(ctx) | ||
|
|
||
| // Process Windows registry keys with the retrieved MAC address (empty if not found). It is required for HNS team to configure cilium routes sepcific to windows nodes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: cilium routes "specific" to windows nodes and nit: isPrefixOnNicSwiftV2() (capital "O" for camel case or isPrefixOnNICSwiftV2() for capitalized acronyms)
|
|
||
| // Process Windows registry keys with the retrieved MAC address (empty if not found). It is required for HNS team to configure cilium routes sepcific to windows nodes. | ||
| isPrefixOnNic := service.isPrefixonNicSwiftV2() | ||
| service.processWindowsRegistryKeys(isPrefixOnNic, infraNicMacAddress) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you confirm that it's intentional that this runs even for non-prefix on nic scenarios?
| // GetIMDSNCs gets NC versions from IMDS and returns them as a map | ||
| func (service *HTTPRestService) GetIMDSNCs(ctx context.Context) (map[string]string, error) { | ||
| // GetIMDSNCs gets NC versions from IMDS and returns them as a map along with the MAC address | ||
| func (service *HTTPRestService) getIMDSNCs(ctx context.Context) (ncs map[string]string, infraNicMacAddress string) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since we're also getting the mac of the infra nic, maybe this should be something like getIMDSInfo now?
| } | ||
|
|
||
| // Check whether NC is SwiftV2 NIC associated NC and prefix on nic is enabled | ||
| func (service *HTTPRestService) isPrefixonNicSwiftV2() bool { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the comment it seems more like "check whether any NC is a swiftv2 nic with prefix on nic enabled" rather than any particular NC-- when this is enabled is there typically only 1 NC (or is it multitenant?)?
|
|
||
| // newWindowsRegistryClient creates a new Windows registry client. | ||
| func newWindowsRegistryClient() windowsRegistryClient { | ||
| return &windowsRegistry{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
similar comment to the one I made about the newLinuxRegistryClient -- can we just return the windows registry directly from the newRegistryClient
|
|
||
| // newLinuxRegistryClient creates a no-op registry client for Linux. | ||
| func newLinuxRegistryClient() windowsRegistryClient { | ||
| return &linuxRegistryClient{} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@NihaNallappagari I don't think you have understood my earlier feedback on OS abstraction/separations.
A func called newLinuxRegistryClient returning struct linuxRegistryClient as interface windowsRegistryClient is nonsensical. Stubbing Windows only behavior in a Linux struct so that the special Windows code compiles on Linux is bad. This abstraction is wrong.
If you need to make stubs like this, the place that you are calling these stubs is OS specific code already. THAT is what should be refactored in to Windows and Linux versions.
Reason for Change:
This PR includes the required changes for Windows to support prefix on NIC NIC Swift v2 scenario. The HNS team requires certain values to be added to the registry, and this PR sets those values under the appropriate registry key.
Issue Fixed:
Requirements:
Notes:
This PR is extension of Prefix on nix linux support