Skip to content

kconnect throttling describeCluster AWS API #751

@ashishapy

Description

@ashishapy

Describe the solution you'd like:
kconnect to optimize for high volume calls. kconnect to optimize how it makes calls to AWS API.

We made 100 concurrent call using kconnect & approx 20 calls failed with below error

Failed to connect to cluster Error: getting cluster: getting cluster config for arn:aws:eks:<account$>:cluster:<cluster-name>: describing cluster <cluster-name>: TooManyRequestException: Too Many Requests

Why do you want this feature:
For scalability & stability

Anything else you would like to add:

AWS Support has shared below documents to implement retries & exponential backoff. AWS confirmed that describeCluster is being invoked while generating KubeConfig from EKS API.

https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html
https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/

Proposals:

  • Implement retries & exponential backoff as per AWS recommendations.
  • Provide a way to generate static kubeConfig for each cluster only once & regenerate only sts token when trying to reconnect to same cluster using same user or different user.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions