fix: paginate DescribeNetworkInterfaces with deep filters #375

sushrk · 2024-02-28T01:15:58Z

Issue #, if available:
Fixes #188
Description of changes:
Paginate DescribeNetworkInterfaces API call by specifying a deep filter like vpc-id and subnet-id as unpaginated API call causes the operation to fail when there is a large number of network interfaces.
https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-network-interfaces.html

Unpaginated calls were used in the periodic leaked ENI cleanup routine and during trunk initialization to fetch all branch ENIs on the node. These are now paginated to use vpc-id and subnet-id respectively, and also added jitter to avoid EC2 API throttling and increased call volume to address earlier issue with pagination.

Testing:
Security group for pods test succeeded:

Ran 19 of 23 Specs in 975.331 seconds
SUCCESS! -- 19 Passed | 0 Failed | 0 Pending | 4 Skipped
PASS

Ginkgo ran 1 suite in 16m18.325964387s
Test Suite Passed

Manual tests:

Node scale up/down
Pods using security groups scale up/down
Created 500+ ENIs in same VPC to confirm call volume has not increased drastically

ec2_describe_network_interfaces_pages_api_call_count 4
ec2_describe_network_interfaces_pages_api_err_count 0

Soak test with 100 nodes and scaling up pods using to security group from 0 to 1000 every 15mins. Did not observe a drastic increase in the API call volume.

Release notes:

Paginate DescribeNetworkInterfaces API call

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

pkg/aws/ec2/api/eni_cleanup.go

pkg/aws/ec2/api/wrapper.go

pkg/aws/ec2/api/eni_cleanup.go

main.go

pkg/aws/ec2/api/wrapper.go

pkg/aws/ec2/api/eni_cleanup.go

orsenthil · 2024-02-28T18:16:31Z

Some overall comments

    Created 500+ ENIs in same VPC to confirm call volume has not increased drastically

ec2_describe_network_interfaces_pages_api_call_count 4
ec2_describe_network_interfaces_pages_api_err_count 0

Do we have before and after call count.
We could measure is the pod create latency of the SGPP operation before and after the change in cluster with 1000 ENI and ensure that they there is no change.
Does the paginated version return any empty pages at all, even with a filter? Can we assert against it?
What are the drawbacks on including a required vpc-id as a parameter in the new version of VPC RC?
So the trunk Init routine, WaitForNetworkInterfaceStatusChange will continue to use DescribeNetworkInterfaces (unpaginated), but only the ENI cleanup will use the paginated. Is that correct? Do we see any problems with trunk init using the unpaginated, with large number of ENI cases?

sushrk · 2024-02-28T21:19:16Z

Some overall comments

    Created 500+ ENIs in same VPC to confirm call volume has not increased drastically

ec2_describe_network_interfaces_pages_api_call_count 4
ec2_describe_network_interfaces_pages_api_err_count 0

1. Do we have _before_ and _after_ call count.

2. We could measure is the pod create latency of the SGPP operation _before_ and _after_ the change in cluster with 1000 ENI and ensure that they there is no change.

3. Does the paginated version return any empty pages at all, even with a filter? Can we assert against it?

4. What are the drawbacks on including a required vpc-id as a parameter in the new version of VPC RC?

5. So the trunk Init routine, `WaitForNetworkInterfaceStatusChange` will continue to use DescribeNetworkInterfaces (unpaginated), but only the ENI cleanup will use the paginated. Is that correct? Do we see any problems with trunk init using the unpaginated, with large number of ENI cases?

Yes, we do a benchmark test for every release, the first graph is from v1.4.3 vs second graph from the test. This is running the integration test cases in a loop for 5 times. Did not observe any drastic increase in call volumes with pagination.

SGPP pod creation latency is comparable to previous releases when scaled up to 1000 pods, approx 4 mins.
The paginated version could return empty pages, this is not deterministic. But using deep filter seems to have alleviated the problem of increased call volume. Graph for soak test with 100 nodes and scaling pods using security pods from 0 to 1000 every 15 mins, ran overnight
There should be no drawbacks, we will update the controller manifest accordingly with the release to populate the VPC-ID. This is required for the cleanup routine to pass the vpc-id in DescribeNetworkInterfaces
All other calls to DescribeNetworkInterfaces pass in the network interface ID directly, like in WaitForNetworkInterfaceStatusChange, and is not an issue with respect to throttling/being blocked by EC2 due to large number of ENIs in the account.

haouc

lgtm

orsenthil · 2024-02-29T01:09:14Z

All other calls to DescribeNetworkInterfaces pass in the network interface ID, like in WaitForNetworkInterfaceStatusChange, and is not an issue with respect to throttling.

Got it. Thank you!.

orsenthil

LGTM

* fix: paginate DescribeNetworkInterfaces with deep filters * update metrics and address review comments * minor updates to address comments

* fix: paginate DescribeNetworkInterfaces with deep filters (#375) * fix: paginate DescribeNetworkInterfaces with deep filters * update metrics and address review comments * minor updates to address comments * Bump github.com/aws/aws-sdk-go from 1.49.13 to 1.50.29 (#380) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.49.13 to 1.50.29. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.49.13...v1.50.29) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump k8s.io/client-go from 0.29.1 to 0.29.2 (#377) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.1 to 0.29.2. - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.29.1...v0.29.2) --- updated-dependencies: - dependency-name: k8s.io/client-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/common from 0.46.0 to 0.49.0 (#378) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.46.0 to 0.49.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.46.0...v0.49.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Repo controlled build go version (#381) * update golang version (#383) --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jason Du <[email protected]>

…s#375)" This reverts commit b5699de.

…bnetID filter * Revert "fix: paginate DescribeNetworkInterfaces with deep filters (#375)" This reverts commit b5699de. * call DescribeNetworkInterfaces with vpcID or subnetID filter

* fix: paginate DescribeNetworkInterfaces with deep filters (aws#375) * fix: paginate DescribeNetworkInterfaces with deep filters * update metrics and address review comments * minor updates to address comments * Bump github.com/aws/aws-sdk-go from 1.49.13 to 1.50.29 (aws#380) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.49.13 to 1.50.29. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.49.13...v1.50.29) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump k8s.io/client-go from 0.29.1 to 0.29.2 (aws#377) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.1 to 0.29.2. - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.29.1...v0.29.2) --- updated-dependencies: - dependency-name: k8s.io/client-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/common from 0.46.0 to 0.49.0 (aws#378) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.46.0 to 0.49.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.46.0...v0.49.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Repo controlled build go version (aws#381) * update golang version (aws#383) --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jason Du <[email protected]>

* fix: paginate DescribeNetworkInterfaces with deep filters (#375) * fix: paginate DescribeNetworkInterfaces with deep filters * update metrics and address review comments * minor updates to address comments * Bump github.com/aws/aws-sdk-go from 1.49.13 to 1.50.29 (#380) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.49.13 to 1.50.29. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.49.13...v1.50.29) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump k8s.io/client-go from 0.29.1 to 0.29.2 (#377) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.1 to 0.29.2. - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.29.1...v0.29.2) --- updated-dependencies: - dependency-name: k8s.io/client-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/common from 0.46.0 to 0.49.0 (#378) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.46.0 to 0.49.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.46.0...v0.49.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Repo controlled build go version (#381) * update golang version (#383) * update protobuf to 1.33.0 (#387) * pin envtest version due to an upstream bug (#390) * Bump k8s.io/client-go from 0.29.2 to 0.29.3 (#392) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.2 to 0.29.3. - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.29.2...v0.29.3) --- updated-dependencies: - dependency-name: k8s.io/client-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/amazon-vpc-cni-k8s from 1.16.0 to 1.17.1 (#393) Bumps [github.com/aws/amazon-vpc-cni-k8s](https://github.com/aws/amazon-vpc-cni-k8s) from 1.16.0 to 1.17.1. - [Release notes](https://github.com/aws/amazon-vpc-cni-k8s/releases) - [Changelog](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/CHANGELOG.md) - [Commits](aws/amazon-vpc-cni-k8s@v1.16.0...v1.17.1) --- updated-dependencies: - dependency-name: github.com/aws/amazon-vpc-cni-k8s dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/common from 0.49.0 to 0.51.1 (#395) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.49.0 to 0.51.1. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.49.0...v0.51.1) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/aws-sdk-go from 1.50.29 to 1.51.12 (#397) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.50.29 to 1.51.12. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.50.29...v1.51.12) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * add github action to run gosec static analysis (#398) * add github action to run gosec static analysis * install gosec * update golang and dependency to fix CVE (#401) * revert pagination and call DescribeNetworkInterfaces with vpcID or subnetID filter * Revert "fix: paginate DescribeNetworkInterfaces with deep filters (#375)" This reverts commit b5699de. * call DescribeNetworkInterfaces with vpcID or subnetID filter * update EC2 supported instance types (#402) * remove global exclusion for G108,G114 and add nosec in code (#404) * Update controller_auth_proxy_patch.yaml (#405) Update the reference from gcr.io to registry.k8s.io > kube-rbac-proxy is moving to registry.k8s.io/kubebuilder/kube-rbac-proxy (from gcr.io/kubebuilder/kube-rbac-proxy) because GCR is being sunset. We need to update these references. * Fix log which causes panic (#407) * Fix log which causes panic * Consistent key name * consistent naming * run go mod tidy --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jason Du <[email protected]> Co-authored-by: Hao Zhou <[email protected]> Co-authored-by: Senthil Kumaran <[email protected]> Co-authored-by: Garvin Pang <[email protected]>

* fix: paginate DescribeNetworkInterfaces with deep filters (aws#375) * fix: paginate DescribeNetworkInterfaces with deep filters * update metrics and address review comments * minor updates to address comments * Bump github.com/aws/aws-sdk-go from 1.49.13 to 1.50.29 (aws#380) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.49.13 to 1.50.29. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.49.13...v1.50.29) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump k8s.io/client-go from 0.29.1 to 0.29.2 (aws#377) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.1 to 0.29.2. - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.29.1...v0.29.2) --- updated-dependencies: - dependency-name: k8s.io/client-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/common from 0.46.0 to 0.49.0 (aws#378) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.46.0 to 0.49.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.46.0...v0.49.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Repo controlled build go version (aws#381) * update golang version (aws#383) --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jason Du <[email protected]>

* fix: paginate DescribeNetworkInterfaces with deep filters (aws#375) * fix: paginate DescribeNetworkInterfaces with deep filters * update metrics and address review comments * minor updates to address comments * Bump github.com/aws/aws-sdk-go from 1.49.13 to 1.50.29 (aws#380) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.49.13 to 1.50.29. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.49.13...v1.50.29) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump k8s.io/client-go from 0.29.1 to 0.29.2 (aws#377) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.1 to 0.29.2. - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.29.1...v0.29.2) --- updated-dependencies: - dependency-name: k8s.io/client-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/common from 0.46.0 to 0.49.0 (aws#378) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.46.0 to 0.49.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.46.0...v0.49.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Repo controlled build go version (aws#381) * update golang version (aws#383) * update protobuf to 1.33.0 (aws#387) * pin envtest version due to an upstream bug (aws#390) * Bump k8s.io/client-go from 0.29.2 to 0.29.3 (aws#392) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.2 to 0.29.3. - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.29.2...v0.29.3) --- updated-dependencies: - dependency-name: k8s.io/client-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/amazon-vpc-cni-k8s from 1.16.0 to 1.17.1 (aws#393) Bumps [github.com/aws/amazon-vpc-cni-k8s](https://github.com/aws/amazon-vpc-cni-k8s) from 1.16.0 to 1.17.1. - [Release notes](https://github.com/aws/amazon-vpc-cni-k8s/releases) - [Changelog](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/CHANGELOG.md) - [Commits](aws/amazon-vpc-cni-k8s@v1.16.0...v1.17.1) --- updated-dependencies: - dependency-name: github.com/aws/amazon-vpc-cni-k8s dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/common from 0.49.0 to 0.51.1 (aws#395) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.49.0 to 0.51.1. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.49.0...v0.51.1) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/aws-sdk-go from 1.50.29 to 1.51.12 (aws#397) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.50.29 to 1.51.12. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.50.29...v1.51.12) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * add github action to run gosec static analysis (aws#398) * add github action to run gosec static analysis * install gosec * update golang and dependency to fix CVE (aws#401) * revert pagination and call DescribeNetworkInterfaces with vpcID or subnetID filter * Revert "fix: paginate DescribeNetworkInterfaces with deep filters (aws#375)" This reverts commit b5699de. * call DescribeNetworkInterfaces with vpcID or subnetID filter * update EC2 supported instance types (aws#402) * remove global exclusion for G108,G114 and add nosec in code (aws#404) * Update controller_auth_proxy_patch.yaml (aws#405) Update the reference from gcr.io to registry.k8s.io > kube-rbac-proxy is moving to registry.k8s.io/kubebuilder/kube-rbac-proxy (from gcr.io/kubebuilder/kube-rbac-proxy) because GCR is being sunset. We need to update these references. * Fix log which causes panic (aws#407) * Fix log which causes panic * Consistent key name * consistent naming * run go mod tidy --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jason Du <[email protected]> Co-authored-by: Hao Zhou <[email protected]> Co-authored-by: Senthil Kumaran <[email protected]> Co-authored-by: Garvin Pang <[email protected]>

* Call DisassociateTrunkInterface before deleting branch ENI (#372) * Call DisassociateTrunkInterface before deleting branch ENI * feat: Centralize leaked ENI cleanup (#374) * feat: centralized eni cleanup * Merge master into eni-cleanup (#385) * fix: paginate DescribeNetworkInterfaces with deep filters (#375) * fix: paginate DescribeNetworkInterfaces with deep filters * update metrics and address review comments * minor updates to address comments * Bump github.com/aws/aws-sdk-go from 1.49.13 to 1.50.29 (#380) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.49.13 to 1.50.29. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.49.13...v1.50.29) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump k8s.io/client-go from 0.29.1 to 0.29.2 (#377) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.1 to 0.29.2. - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.29.1...v0.29.2) --- updated-dependencies: - dependency-name: k8s.io/client-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/common from 0.46.0 to 0.49.0 (#378) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.46.0 to 0.49.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.46.0...v0.49.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Repo controlled build go version (#381) * update golang version (#383) --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jason Du <[email protected]> * fix:update cluster tag name in CNINode (#386) * fix:add node OS label in CNINode, retry get CNINode with backoff * update protobuf to 1.33.0 (#387) * add CNINode integration tests (#391) * use DescribeNetworkInterfaces with deep filters * add integration test to validate ec2 permissions * remove DisassociateAllBranchENIs as it is not useful (#400) * remove DisassociateAllBranchENIs as it is not useful * skip deletion success log for NotFound ENI * fix govulncheck * Merge master branch into eni-cleanup (#416) * fix: paginate DescribeNetworkInterfaces with deep filters (#375) * fix: paginate DescribeNetworkInterfaces with deep filters * update metrics and address review comments * minor updates to address comments * Bump github.com/aws/aws-sdk-go from 1.49.13 to 1.50.29 (#380) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.49.13 to 1.50.29. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.49.13...v1.50.29) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump k8s.io/client-go from 0.29.1 to 0.29.2 (#377) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.1 to 0.29.2. - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.29.1...v0.29.2) --- updated-dependencies: - dependency-name: k8s.io/client-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/common from 0.46.0 to 0.49.0 (#378) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.46.0 to 0.49.0. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.46.0...v0.49.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Repo controlled build go version (#381) * update golang version (#383) * update protobuf to 1.33.0 (#387) * pin envtest version due to an upstream bug (#390) * Bump k8s.io/client-go from 0.29.2 to 0.29.3 (#392) Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.29.2 to 0.29.3. - [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md) - [Commits](kubernetes/client-go@v0.29.2...v0.29.3) --- updated-dependencies: - dependency-name: k8s.io/client-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/amazon-vpc-cni-k8s from 1.16.0 to 1.17.1 (#393) Bumps [github.com/aws/amazon-vpc-cni-k8s](https://github.com/aws/amazon-vpc-cni-k8s) from 1.16.0 to 1.17.1. - [Release notes](https://github.com/aws/amazon-vpc-cni-k8s/releases) - [Changelog](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/CHANGELOG.md) - [Commits](aws/amazon-vpc-cni-k8s@v1.16.0...v1.17.1) --- updated-dependencies: - dependency-name: github.com/aws/amazon-vpc-cni-k8s dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/prometheus/common from 0.49.0 to 0.51.1 (#395) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.49.0 to 0.51.1. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.49.0...v0.51.1) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github.com/aws/aws-sdk-go from 1.50.29 to 1.51.12 (#397) Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.50.29 to 1.51.12. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](aws/aws-sdk-go@v1.50.29...v1.51.12) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * add github action to run gosec static analysis (#398) * add github action to run gosec static analysis * install gosec * update golang and dependency to fix CVE (#401) * revert pagination and call DescribeNetworkInterfaces with vpcID or subnetID filter * Revert "fix: paginate DescribeNetworkInterfaces with deep filters (#375)" This reverts commit b5699de. * call DescribeNetworkInterfaces with vpcID or subnetID filter * update EC2 supported instance types (#402) * remove global exclusion for G108,G114 and add nosec in code (#404) * Update controller_auth_proxy_patch.yaml (#405) Update the reference from gcr.io to registry.k8s.io > kube-rbac-proxy is moving to registry.k8s.io/kubebuilder/kube-rbac-proxy (from gcr.io/kubebuilder/kube-rbac-proxy) because GCR is being sunset. We need to update these references. * Fix log which causes panic (#407) * Fix log which causes panic * Consistent key name * consistent naming * run go mod tidy --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jason Du <[email protected]> Co-authored-by: Hao Zhou <[email protected]> Co-authored-by: Senthil Kumaran <[email protected]> Co-authored-by: Garvin Pang <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jason Du <[email protected]> Co-authored-by: Hao Zhou <[email protected]> Co-authored-by: Senthil Kumaran <[email protected]> Co-authored-by: Garvin Pang <[email protected]>

* fix: paginate DescribeNetworkInterfaces with deep filters * update metrics and address review comments * minor updates to address comments

…bnetID filter * Revert "fix: paginate DescribeNetworkInterfaces with deep filters (#375)" This reverts commit b5699de. * call DescribeNetworkInterfaces with vpcID or subnetID filter

fix: paginate DescribeNetworkInterfaces with deep filters

8d26dfe

sushrk requested a review from a team as a code owner February 28, 2024 01:15

haouc reviewed Feb 28, 2024

View reviewed changes