Skip to content

Commit

Permalink
nginx-ingress: add rate limits options (#137)
Browse files Browse the repository at this point in the history
  • Loading branch information
@dobegor
dobegor authored Aug 20, 2021
1 parent bdedcf0 commit 9324af5
Show file tree
Hide file tree
Showing 8 changed files with 174 additions and 41 deletions.
43 changes: 22 additions & 21 deletions docs/nginx-ingress.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,27 +29,28 @@ CLI flags apply only at the global level i.e. applies to all paths and methods.
To override settings on the path or HTTP method level, you are required to use the x-kusk extension at that path in your API specification.ß

## Full Options Reference
| Name | CLI Option | OpenAPI Spec x-kusk label | Descriptions | Overwritable at path / method |
|:----------------------------:|:------------------------------:|:----------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-----------------------------:|
| OpenAPI or Swagger File | --in | N/A | Location of the OpenAPI or Swagger specification | ❌ |
| Namespace | --namespace | namespace | the namespace in which to create the generated resources (Required) | ❌ |
| Service Name | --service.name | service.name | the name of the service running in Kubernetes (Required) | ❌ |
| Service Namespace | --service.namespace | service.namespace | The namespace where the service named above resides (default value: default) | ❌ |
| Service Port | --service.port | service.port | Port the service is listening on (default value: 80) | ❌ |
| Path Base | --path.base | path.base | Prefix for your resource routes | ❌ |
| Path Trim Prefix | --path.trim_prefix | path.trim_prefix | Trim the specified prefix from URl before passing request onto service | ❌ |
| Path split | --path.split | path.split | Boolean; whether or not to force generator to generate a mapping for each path | ❌ |
| Ingress Host | --host | host | The value to set the host field to in the Ingress resource | ❌ |
| Nginx Ingress Rewrite Target | --nginx_ingress.rewrite_target | nginx_ingress.rewrite_target | Manually set the rewrite target for where traffic must be redirected | ❌ |
| Request Timeout | --timeouts.request_timeout | timeouts.request_timeout | Total request timeout (seconds) | ✅ |
| Idle Timeout | --timeouts.idle_timeout | timeouts.idle_timeout | Idle connection timeout (seconds) | ✅ |
| CORS Origins | N/A | cors.origins | Array of origins | ✅ |
| CORS Methods | N/A | cors.methods | Array of methods | ✅ |
| CORS Headers | N/A | cors.headers | Array of headers | ✅ |
| CORS ExposeHeaders | N/A | cors.expose_headers | Array of headers to expose | ✅ |
| CORS Credentials | N/A | cors.credentials | Boolean: enable credentials (default value: false) | ✅ |
| CORS Max Age | N/A | cors.max_age | Integer:how long the response to the preflight request can be cached for without sending another preflight request | ✅ |

| Name | CLI Option | OpenAPI Spec x-kusk label | Descriptions | Overwritable at path / method |
|------------------------------|--------------------------------|------------------------------|--------------------------------------------------------------------------------------------------------------------|-------------------------------|
| OpenAPI or Swagger File | --in | N/A | Location of the OpenAPI or Swagger specification | ❌ |
| Namespace | --namespace | namespace | the namespace in which to create the generated resources (Required) | ❌ |
| Service Name | --service.name | service.name | the name of the service running in Kubernetes (Required) | ❌ |
| Service Namespace | --service.namespace | service.namespace | The namespace where the service named above resides (default value: default) | ❌ |
| Service Port | --service.port | service.port | Port the service is listening on (default value: 80) | ❌ |
| Path Base | --path.base | path.base | Prefix for your resource routes | ❌ |
| Path Trim Prefix | --path.trim_prefix | path.trim_prefix | Trim the specified prefix from URl before passing request onto service | ❌ |
| Path split | --path.split | path.split | Boolean; whether or not to force generator to generate a mapping for each path | ❌ |
| Ingress Host | --host | host | The value to set the host field to in the Ingress resource | ❌ |
| Nginx Ingress Rewrite Target | --nginx_ingress.rewrite_target | nginx_ingress.rewrite_target | Manually set the rewrite target for where traffic must be redirected | ❌ |
| Rate limit (RPS) | --rate_limits.rps | rate_limits.rps | Request per second rate limit | ✅ |
| Rate limit (burst) | --rate_limits.burst | rate_limits.burst | Rate limit burst | ✅ |
| Request Timeout | --timeouts.request_timeout | timeouts.request_timeout | Total request timeout (seconds) | ✅ |
| Idle Timeout | --timeouts.idle_timeout | timeouts.idle_timeout | Idle connection timeout (seconds) | ✅ |
| CORS Origins | N/A | cors.origins | Array of origins | ✅ |
| CORS Methods | N/A | cors.methods | Array of methods | ✅ |
| CORS Headers | N/A | cors.headers | Array of headers | ✅ |
| CORS ExposeHeaders | N/A | cors.expose_headers | Array of headers to expose | ✅ |
| CORS Credentials | N/A | cors.credentials | Boolean: enable credentials (default value: false) | ✅ |
| CORS Max Age | N/A | cors.max_age | Integer:how long the response to the preflight request can be cached for without sending another preflight request | ✅ |
## Basic Usage
### CLI Flags
```shell
Expand Down
19 changes: 19 additions & 0 deletions generators/nginx_ingress/annotations.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ func (g *Generator) generateAnnotations(
path *options.PathOptions,
nginx *options.NGINXIngressOptions,
cors *options.CORSOptions,
rateLimits *options.RateLimitOptions,
timeoutOpts *options.TimeoutOptions,
) map[string]string {
annotations := map[string]string{}
Expand Down Expand Up @@ -71,6 +72,24 @@ func (g *Generator) generateAnnotations(
}
// End CORS

// Rate limits
if rps := rateLimits.RPS; rps != 0 {
annotations["nginx.ingress.kubernetes.io/limit-rps"] = fmt.Sprint(rps)

if burst := rateLimits.Burst; burst != 0 {
// https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#rate-limiting
// nginx-ingress uses a burst multiplier to configure burst for a rate limited path,
// i.e. burst = rps * burstMultiplier
var burstMultiplier = burst / rps
if burstMultiplier < 1 {
burstMultiplier = 1
}

annotations["nginx.ingress.kubernetes.io/limit-burst-multiplier"] = fmt.Sprint(burstMultiplier)
}
}
// End rate limits

// Timeouts
if requestTimeout := timeoutOpts.RequestTimeout; requestTimeout > 0 {
strTimeout := strconv.Itoa(int(requestTimeout) / 2)
Expand Down
38 changes: 32 additions & 6 deletions generators/nginx_ingress/nginx_ingress.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,18 @@ func (g *Generator) Flags() *pflag.FlagSet {
"an Ingress Host to listen on",
)

fs.Uint32(
"rate_limits.rps",
0,
"request per second rate limit",
)

fs.Uint32(
"rate_limits.burst",
0,
"request per second burst",
)

fs.Uint32(
"timeouts.request_timeout",
0,
Expand Down Expand Up @@ -109,10 +121,8 @@ func (g *Generator) Generate(opts *options.Options, spec *openapi3.T) (string, e

name := fmt.Sprintf("%s-%s", opts.Service.Name, ingressResourceNameFromPath(path))

var corsOpts options.CORSOptions

// take global CORS options
corsOpts = opts.CORS
corsOpts := opts.CORS

// if path-level CORS options are different, override with them
if pathSubOpts, ok := opts.PathSubOptions[path]; ok {
Expand All @@ -122,10 +132,19 @@ func (g *Generator) Generate(opts *options.Options, spec *openapi3.T) (string, e
}
}

var timeoutOpts options.TimeoutOptions
// take global rate limit options
rateLimitOpts := opts.RateLimits

// if path-level rate limit options are different, override with them
if pathSubOpts, ok := opts.PathSubOptions[path]; ok {
if !reflect.DeepEqual(options.RateLimitOptions{}, pathSubOpts.RateLimits) &&
!reflect.DeepEqual(rateLimitOpts, pathSubOpts.RateLimits) {
rateLimitOpts = pathSubOpts.RateLimits
}
}

// take global Timeout options
timeoutOpts = opts.Timeouts
timeoutOpts := opts.Timeouts

// if path-level Timeout options are different, override with them
if pathSubOpts, ok := opts.PathSubOptions[path]; ok {
Expand All @@ -141,6 +160,7 @@ func (g *Generator) Generate(opts *options.Options, spec *openapi3.T) (string, e
&opts.Path,
&opts.NGINXIngress,
&corsOpts,
&rateLimitOpts,
&timeoutOpts,
)

Expand Down Expand Up @@ -184,7 +204,7 @@ func (g *Generator) Generate(opts *options.Options, spec *openapi3.T) (string, e
opts.Namespace,
g.generatePath(&opts.Path, &opts.NGINXIngress),
pathTypePrefix,
g.generateAnnotations(&opts.Path, &opts.NGINXIngress, &opts.CORS, &opts.Timeouts),
g.generateAnnotations(&opts.Path, &opts.NGINXIngress, &opts.CORS, &opts.RateLimits, &opts.Timeouts),
&opts.Service,
opts.Host,
)
Expand Down Expand Up @@ -306,6 +326,12 @@ func (g *Generator) shouldSplit(opts *options.Options, spec *openapi3.T) bool {
return true
}

// a path has non-zero, different from global scope rate limits options
if !reflect.DeepEqual(options.RateLimitOptions{}, pathSubOptions.RateLimits) &&
!reflect.DeepEqual(opts.RateLimits, pathSubOptions.RateLimits) {
return true
}

// a path has non-zero, different from global scope timeouts options
if !reflect.DeepEqual(options.TimeoutOptions{}, pathSubOptions.Timeouts) &&
!reflect.DeepEqual(opts.Timeouts, pathSubOptions.Timeouts) {
Expand Down
72 changes: 72 additions & 0 deletions generators/nginx_ingress/nginx_ingress_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -441,6 +441,78 @@ spec:
pathType: Prefix
status:
loadBalancer: {}
`,
},
{
name: "rate limit options",
options: options.Options{
Namespace: "booksapp",
Service: options.ServiceOptions{
Namespace: "booksapp",
Name: "webapp",
Port: 7000,
},
Path: options.PathOptions{
Base: "/bookstore",
TrimPrefix: "/bookstore",
},
RateLimits: options.RateLimitOptions{
RPS: 100,
Burst: 400,
},
},
spec: `openapi: 3.0.1
x-kusk:
namespace: booksapp
rate_limits:
rps: 100
burst: 400
path:
base: /bookstore
trim_prefix: /bookstore
service:
name: webapp
namespace: booksapp
port: 7000
paths:
/:
get: {}
/books/{id}:
get:
parameters:
- name: id
in: path
required: true
schema:
type: integer
format: int64
`,
res: `---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/limit-burst-multiplier: "4"
nginx.ingress.kubernetes.io/limit-rps: "100"
nginx.ingress.kubernetes.io/rewrite-target: /$2
creationTimestamp: null
name: webapp-ingress
namespace: booksapp
spec:
ingressClassName: nginx
rules:
- http:
paths:
- backend:
service:
name: webapp
port:
number: 7000
path: /bookstore(/|$)(.*)
pathType: Prefix
status:
loadBalancer: {}
`,
},
}
Expand Down
10 changes: 7 additions & 3 deletions options/options.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,10 @@ import (
type SubOptions struct {
Disabled bool `yaml:"disabled,omitempty" json:"disabled,omitempty"`

Host string `yaml:"host,omitempty" json:"host,omitempty"`
CORS CORSOptions `yaml:"cors,omitempty" json:"cors,omitempty"`
Timeouts TimeoutOptions `yaml:"timeouts,omitempty" json:"timeouts,omitempty"`
Host string `yaml:"host,omitempty" json:"host,omitempty"`
CORS CORSOptions `yaml:"cors,omitempty" json:"cors,omitempty"`
RateLimits RateLimitOptions `yaml:"rate_limits,omitempty" json:"rate_limits,omitempty"`
Timeouts TimeoutOptions `yaml:"timeouts,omitempty" json:"timeouts,omitempty"`
}

type Options struct {
Expand Down Expand Up @@ -44,6 +45,8 @@ type Options struct {
// They are filled during extension parsing, the map key is method+path.
OperationSubOptions map[string]SubOptions `yaml:"-" json:"-"`

RateLimits RateLimitOptions `yaml:"rate_limits,omitempty" json:"rate_limits,omitempty"`

Timeouts TimeoutOptions `yaml:"timeouts,omitempty" json:"timeouts,omitempty"`
}

Expand Down Expand Up @@ -87,6 +90,7 @@ func (o *Options) FillDefaultsAndValidate() error {
&o.Cluster,
&o.CORS,
&o.NGINXIngress,
&o.RateLimits,
&o.Timeouts,
})

Expand Down
10 changes: 10 additions & 0 deletions options/ratelimit.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package options

type RateLimitOptions struct {
RPS uint32 `json:"rps,omitempty" yaml:"rps,omitempty"`
Burst uint32 `json:"burst,omitempty" yaml:"burst,omitempty"`
}

func (o *RateLimitOptions) Validate() error {
return nil
}
11 changes: 0 additions & 11 deletions options/service.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,6 @@ type ServiceOptions struct {
Port int32 `yaml:"port,omitempty" json:"port,omitempty"`
}

type TimeoutOptions struct {
// RequestTimeout is total request timeout
RequestTimeout uint32 `yaml:"request_timeout,omitempty" json:"request_timeout,omitempty"`
// IdleTimeout is timeout for idle connection
IdleTimeout uint32 `yaml:"idle_timeout,omitempty" json:"idle_timeout,omitempty"`
}

func (o *TimeoutOptions) Validate() error {
return nil
}

func (o *ServiceOptions) Validate() error {
return v.ValidateStruct(o,
v.Field(&o.Namespace, v.Required.Error("service.namespace is required")),
Expand Down
12 changes: 12 additions & 0 deletions options/timeout.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
package options

type TimeoutOptions struct {
// RequestTimeout is total request timeout
RequestTimeout uint32 `yaml:"request_timeout,omitempty" json:"request_timeout,omitempty"`
// IdleTimeout is timeout for idle connection
IdleTimeout uint32 `yaml:"idle_timeout,omitempty" json:"idle_timeout,omitempty"`
}

func (o *TimeoutOptions) Validate() error {
return nil
}

0 comments on commit 9324af5

Please sign in to comment.