Istio 基于 Envoy 实现服务限流

相关背景

  1. 美服出现部分应用接口被外部流量恶意请求，刷取登录接口的情况。
  2. 需要针对集群的所有应用进行限流，防止因为恶意请求或者大流量导致服务宕机。

预期效果：

  1. 能够实现全局或者应用维度的 IP 限流。
  2. 以每分钟300个请求的限流机制为例，向服务发送301个请求，限流器将拒绝第301个请求。同时，限流器会返回一个429 HTTP状态码（Too Many Requests），并在请求到达服务之前进行拒绝。

简单介绍

Istio 在 1.5 版本开始，已经弃用了 Mixer，因此也就不再支持基于 Mixer 的 Quota 方式进行限流。从 Istio 1.5 版本开始，推荐使用 Envoy 原生的限流功能，或者使用其他的第三方限流插件。
目前 Envoy 支持种限流：全局和本地。

  1. 全局限流依赖于外部 限流服务，envoy会通过 rpc 的方式进行调用限流服务，限流服务负责响应该请求是否被处理。
  2. 本地限流粒度是每个服务，用于限制每个服务的请求速率。基于每个Envoy进程（sidecar）配置，即每个注入了Envoy代理的Pod。相比全局限流来说，本地限流的配置更简单，不需要额外的组件。

全局限流

全局限流的配置涉及两个部分：Envoy 的 rate_limits过滤器 和 限流服务的配置。

  • rate_limits 过滤器中包含 actions 列表。Envoy 会尝试将每个请求与 rate_limits 过滤器中的每个 action 进行匹配。每个 action 会生成一个 descriptor 描述符。描述符是与 action 对应的一组描述符条目。每个描述符条目是一个键值对，通常表示为”descriptor-key-1”: “descriptor-value-1”、”descriptor-key-2”: “descriptor-value-2”等形式。

相关配置参数 ：https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter#config-http-filters-rate-limit

  • 限流服务的配置则能够匹配每个请求所产生的描述符条目。针对特定的一组描述符条目，限流服务的配置能够指定其对应的请求速率限制。限流服务通过与Redis缓存交互来决定是否对请求进行限流，并将限流决策响应给Envoy代理。

本地限流

  • 基于每个Envoy进程配置，即每个注入了Envoy代理的Pod。相比全局限流来说，本地限流的配置更简单，不需要额外的组件。本地限流的优先级高于全局限流，当同时使用本地速率限制器和全局速率限制器时，首先应用本地速率限制器进行限制，如果未达到本地速率限制，则应用全局速率限制器进行限制。示例场景如下：
  • 假设本地限流对特定客户端IP的请求限制为每分钟50个请求，全局请求限制数为每分钟60个请求。客户端发送超过50个请求，则本地限流将拒绝该请求，即使全局限流尚未达到限制。
  • 假设本地限流对特定客户端IP的请求限制为每分钟50个请求，全局请求限制数为每分钟40个请求。客户端发送超过40个请求，虽然未达到本地限流，但是已达到全局限流，因此该请求被拒绝。
  • 本地限流如果有多个副本，则每个副本都有各自的速率限制器，也就是说如果您在一个副本上被限流，在另一个副本上可能不会被限流。

详细配置

全局限流

创建限流服务

（可以自己自定义开发，满足rpc接口规范即可）
IDL：https://www.envoyproxy.io/docs/envoy/latest/configuration/other_features/rate_limit#config-rate-limit-service
外部限流服务：https://github.com/istio/istio/blob/release-1.21/samples/ratelimit/rate-limit-service.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
apiVersion: v1
kind: ConfigMap
metadata:
  name: ratelimit-config
data:
  config.yaml: |
    domain: ratelimit
    descriptors:
      
    #   可以按照路径进行匹配
    #  - key: PATH
    #    value: "/productpage"
    #    rate_limit:
    #      unit: minute
    #      requests_per_unit: 1
      
      - key: remote_address
        descriptors:
          - key: PATH
            rate_limit:
              unit: minute
              requests_per_unit: 100
---
apiVersion: v1
kind: Service
metadata:
  name: redis
  labels:
    app: redis
spec:
  ports:
    - name: redis
      port: 6379
  selector:
    app: redis
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
        - image: redis:alpine
          imagePullPolicy: Always
          name: redis
          ports:
            - name: redis
              containerPort: 6379
      restartPolicy: Always
      serviceAccountName: ""
---
apiVersion: v1
kind: Service
metadata:
  name: ratelimit
  labels:
    app: ratelimit
spec:
  ports:
    - name: http-port
      port: 8080
      targetPort: 8080
      protocol: TCP
    - name: grpc-port
      port: 8081
      targetPort: 8081
      protocol: TCP
    - name: http-debug
      port: 6070
      targetPort: 6070
      protocol: TCP
  selector:
    app: ratelimit
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ratelimit
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ratelimit
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: ratelimit
    spec:
      containers:
        - image: envoyproxy/ratelimit:9d8d70a8 # 2022/08/16
          imagePullPolicy: Always
          name: ratelimit
          command: ["/bin/ratelimit"]
          env:
            - name: LOG_LEVEL
              value: debug
            - name: REDIS_SOCKET_TYPE
              value: tcp
            - name: REDIS_URL
              value: redis:6379
            - name: USE_STATSD
              value: "false"
            - name: RUNTIME_ROOT
              value: /data
            - name: RUNTIME_SUBDIRECTORY
              value: ratelimit
            - name: RUNTIME_WATCH_ROOT
              value: "false"
            - name: RUNTIME_IGNOREDOTFILES
              value: "true"
            - name: HOST
              value: "::"
            - name: GRPC_HOST
              value: "::"
          ports:
            - containerPort: 8080
            - containerPort: 8081
            - containerPort: 6070
          volumeMounts:
            - name: config-volume
              mountPath: /data/ratelimit/config
      volumes:
        - name: config-volume
          configMap:
            name: ratelimit-config

创建 Envoy Filter

因为GET请求路径URL会携带参数，所以这里我们使用LUA去掉参数只取路径作为键。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: filter-ratelimit
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.router"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.lua
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
            inlineCode: |
              function envoy_on_request(request_handle)
                local path = request_handle:headers():get(":path")
                local method = request_handle:headers():get(":method")
                local params_start = string.find(path, "?")
                if params_start then
                  path = string.sub(path, 1, params_start - 1)
                end
                request_handle:headers():add("x-envoy-original-limit-rate-path", method .. path)
              end
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.router"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.ratelimit
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
            domain: ratelimit
            failure_mode_deny: true
            timeout: 10s
            rate_limit_service:
              grpc_service:
                envoy_grpc:
                  cluster_name: outbound|8081||ratelimit.default.svc.cluster.local
                  authority: ratelimit.default.svc.cluster.local
              transport_api_version: V3
---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: filter-ratelimit-svc
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: VIRTUAL_HOST
      match:
        context: GATEWAY
        routeConfiguration:
          vhost:
            name: "" # 对应 VirtualService 的 host（缺省全部服务）
            route:
              action: ANY
      patch:
        operation: MERGE
        # Applies the rate limit rules.
        value:
          rate_limits:
            - actions: # any actions in here
                - remote_address: {}
                - request_headers:
                    header_name: "x-envoy-original-limit-rate-path"
                    descriptor_key: "PATH"

此 patch 将 envoy.filters.http.ratelimitEnvoy 全局限流过滤器插入到 HTTP_FILTER 链中。
rate_limit_service 字段指定外部速率限制服务，在本例中为
outbound|8081||ratelimit.default.svc.cluster.local。

执行测试

限流功能测试

  1. 分别从两个ip地址发送请求90条（上面我们配置为每分钟每个ip访问相同路径最大100次）

IP地址1

IP地址2

均能正常访问

  1. 相同IP发送180条请求


出现限流返回 429 响应码，只有部分请求能够访问成功，超出限流阈值返回 429 too many requests

1
2
3
4
5
6
7
8
9
10
11
12
13
#!/bin/bash

url="https://api-test.xxxxxx.com/api/healthz"

for i in {1..95}
do
  {
    response=$(curl -s $url -o /dev/null -w "%{http_code}\n")
    echo "Response $i: $response."
  } &
done

wait

限流性能测试

在未加 Envoy Filter 之前我们对应用接口进行测试
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://support-api-test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   22.799 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54335680 bytes
HTML transferred:       5600000 bytes
Requests per second:    4386.07 [#/sec] (mean)
Time per request:       227.995 [ms] (mean)
Time per request:       0.228 [ms] (mean, across all concurrent requests)
Transfer rate:          2327.34 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   66   6.1     66    1071
Processing:    60  160  54.0    158     542
Waiting:       60  159  53.9    158     493
Total:        120  225  54.7    223    1390

Percentage of the requests served within a certain time (ms)
  50%    223
  66%    242
  75%    254
  80%    262
  90%    285
  95%    312
  98%    349
  99%    405
 100%   1390 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   24.585 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54329892 bytes
HTML transferred:       5600000 bytes
Requests per second:    4067.57 [#/sec] (mean)
Time per request:       245.847 [ms] (mean)
Time per request:       0.246 [ms] (mean, across all concurrent requests)
Transfer rate:          2158.11 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   66   8.1     66    1064
Processing:    60  179 322.6    150    5313
Waiting:       60  158  65.2    150     602
Total:        120  245 322.7    216    5384

Percentage of the requests served within a certain time (ms)
  50%    216
  66%    243
  75%    260
  80%    272
  90%    306
  95%    342
  98%    416
  99%    479
 100%   5384 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   23.324 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54344802 bytes
HTML transferred:       5600000 bytes
Requests per second:    4287.50 [#/sec] (mean)
Time per request:       233.236 [ms] (mean)
Time per request:       0.233 [ms] (mean, across all concurrent requests)
Transfer rate:          2275.42 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   66   9.3     66    1066
Processing:    61  166  53.3    167     572
Waiting:       61  166  53.1    167     564
Total:        120  232  54.4    233    1390

Percentage of the requests served within a certain time (ms)
  50%    233
  66%    252
  75%    265
  80%    273
  90%    297
  95%    318
  98%    350
  99%    386
 100%   1390 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$

可以看到 P99 大概在 400ms 左右

加上全局的 Envoy Filter 限流对应用接口进行测试（基于IP）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        0 bytes

Concurrency Level:      1000
Time taken for tests:   53.607 seconds
Complete requests:      100000
Failed requests:        100
   (Connect: 0, Receive: 0, Length: 100, Exceptions: 0)
Non-2xx responses:      99900
Total transferred:      15738501 bytes
HTML transferred:       5600 bytes
Requests per second:    1865.44 [#/sec] (mean)
Time per request:       536.068 [ms] (mean)
Time per request:       0.536 [ms] (mean, across all concurrent requests)
Transfer rate:          286.71 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   4.0     66     157
Processing:   181  469  74.9    462     818
Waiting:      181  469  74.9    462     818
Total:        241  534  75.2    528     880

Percentage of the requests served within a certain time (ms)
  50%    528
  66%    559
  75%    581
  80%    595
  90%    634
  95%    666
  98%    707
  99%    732
 100%    880 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        0 bytes

Concurrency Level:      1000
Time taken for tests:   53.869 seconds
Complete requests:      100000
Failed requests:        100
   (Connect: 0, Receive: 0, Length: 100, Exceptions: 0)
Non-2xx responses:      99900
Total transferred:      15738506 bytes
HTML transferred:       5600 bytes
Requests per second:    1856.36 [#/sec] (mean)
Time per request:       538.689 [ms] (mean)
Time per request:       0.539 [ms] (mean, across all concurrent requests)
Transfer rate:          285.32 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   6.8     66    1070
Processing:    84  472  80.4    462     959
Waiting:       84  472  80.4    462     959
Total:        143  537  80.8    528    1606

Percentage of the requests served within a certain time (ms)
  50%    528
  66%    559
  75%    582
  80%    598
  90%    641
  95%    681
  98%    728
  99%    770
 100%   1606 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        0 bytes

Concurrency Level:      1000
Time taken for tests:   51.153 seconds
Complete requests:      100000
Failed requests:        100
   (Connect: 0, Receive: 0, Length: 100, Exceptions: 0)
Non-2xx responses:      99900
Total transferred:      15738559 bytes
HTML transferred:       5600 bytes
Requests per second:    1954.93 [#/sec] (mean)
Time per request:       511.526 [ms] (mean)
Time per request:       0.512 [ms] (mean, across all concurrent requests)
Transfer rate:          300.47 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   6.8     66    1066
Processing:   158  445  57.9    443     757
Waiting:      158  445  57.9    443     757
Total:        218  510  58.6    508    1593

Percentage of the requests served within a certain time (ms)
  50%    508
  66%    529
  75%    544
  80%    554
  90%    582
  95%    609
  98%    642
  99%    669
 100%   1593 (longest request)

可以看到 P99 大概在 700ms 左右，增加了大概 300ms 延迟。（RPC调用限流服务->redis存储限流桶数据）

加上全局的 Envoy Filter 限流对应用接口进行测试（基于IP+ratelimit5个pod）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   30.070 seconds
Complete requests:      100000
Failed requests:        99900
   (Connect: 0, Receive: 0, Length: 99900, Exceptions: 0)
Non-2xx responses:      99900
Total transferred:      15738564 bytes
HTML transferred:       5600 bytes
Requests per second:    3325.61 [#/sec] (mean)
Time per request:       300.696 [ms] (mean)
Time per request:       0.301 [ms] (mean, across all concurrent requests)
Transfer rate:          511.14 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   6.0     66    1061
Processing:    67  234  72.1    222     754
Waiting:       67  234  72.1    222     754
Total:        127  299  72.6    287    1278

Percentage of the requests served within a certain time (ms)
  50%    287
  66%    314
  75%    334
  80%    348
  90%    390
  95%    435
  98%    500
  99%    542
 100%   1278 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        0 bytes

Concurrency Level:      1000
Time taken for tests:   31.984 seconds
Complete requests:      100000
Failed requests:        100
   (Connect: 0, Receive: 0, Length: 100, Exceptions: 0)
Non-2xx responses:      99900
Total transferred:      15738658 bytes
HTML transferred:       5600 bytes
Requests per second:    3126.52 [#/sec] (mean)
Time per request:       319.844 [ms] (mean)
Time per request:       0.320 [ms] (mean, across all concurrent requests)
Transfer rate:          480.54 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   6.0     66    1064
Processing:    65  253  80.1    238     810
Waiting:       65  253  80.0    238     810
Total:        125  318  80.5    304    1452

Percentage of the requests served within a certain time (ms)
  50%    304
  66%    335
  75%    360
  80%    376
  90%    424
  95%    472
  98%    530
  99%    569
 100%   1452 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        0 bytes

Concurrency Level:      1000
Time taken for tests:   32.559 seconds
Complete requests:      100000
Failed requests:        100
   (Connect: 0, Receive: 0, Length: 100, Exceptions: 0)
Non-2xx responses:      99900
Total transferred:      15738549 bytes
HTML transferred:       5600 bytes
Requests per second:    3071.38 [#/sec] (mean)
Time per request:       325.587 [ms] (mean)
Time per request:       0.326 [ms] (mean, across all concurrent requests)
Transfer rate:          472.06 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   9.8     66    1065
Processing:    71  259  78.6    246     781
Waiting:       71  259  78.6    246     781
Total:        131  324  79.5    311    1479

Percentage of the requests served within a certain time (ms)
  50%    311
  66%    343
  75%    366
  80%    382
  90%    429
  95%    475
  98%    531
  99%    567
 100%   1479 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$

可以看到 P99 大概在 500ms 左右，相对于不配置全局限流，增加了大概 100ms 延迟。相对于单个限流服务，降低大概 200ms 延迟。

加上全局的 Envoy Filter 限流对应用接口进行测试（基于IP+请求方法+请求路径（LUA））

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   70.648 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54260423 bytes
HTML transferred:       5600000 bytes
Requests per second:    1415.47 [#/sec] (mean)
Time per request:       706.481 [ms] (mean)
Time per request:       0.706 [ms] (mean, across all concurrent requests)
Transfer rate:          750.04 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   9.1     66    1071
Processing:    74  639  94.6    632    1394
Waiting:       74  639  94.6    632    1393
Total:        142  704  95.2    697    1758

Percentage of the requests served within a certain time (ms)
  50%    697
  66%    726
  75%    747
  80%    760
  90%    798
  95%    840
  98%    936
  99%   1046
 100%   1758 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   76.901 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54247955 bytes
HTML transferred:       5600000 bytes
Requests per second:    1300.37 [#/sec] (mean)
Time per request:       769.011 [ms] (mean)
Time per request:       0.769 [ms] (mean, across all concurrent requests)
Transfer rate:          688.89 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   7.2     66    1065
Processing:   225  702  76.7    693    1179
Waiting:      225  702  76.7    693    1179
Total:        292  766  77.1    758    1834

Percentage of the requests served within a certain time (ms)
  50%    758
  66%    784
  75%    803
  80%    816
  90%    857
  95%    899
  98%    974
  99%   1012
 100%   1834 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   76.800 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54257732 bytes
HTML transferred:       5600000 bytes
Requests per second:    1302.08 [#/sec] (mean)
Time per request:       768.002 [ms] (mean)
Time per request:       0.768 [ms] (mean, across all concurrent requests)
Transfer rate:          689.92 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   9.1     66    1066
Processing:   194  701  93.3    693    1470
Waiting:      194  701  93.2    693    1470
Total:        256  766  93.8    757    1858

Percentage of the requests served within a certain time (ms)
  50%    757
  66%    790
  75%    813
  80%    829
  90%    871
  95%    915
  98%    992
  99%   1071
 100%   1858 (longest request)

在启用全局限流后 P99 大概在 1000 ms，因为使用了 LUA 对请求进行切割了 请求路径+拼接请求方法，增加调用延迟。

  1. 限流服务负载

  1. istiod负载（Istiod 本身不处理服务间的请求，因此它不会直接影响服务的并发量，无需关心）

  1. Istio 负载

加上全局的 Envoy Filter 限流对应用接口进行测试（基于IP+请求方法+请求路径（LUA）+ 5个rate limit Pod）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   47.142 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54359865 bytes
HTML transferred:       5600000 bytes
Requests per second:    2121.27 [#/sec] (mean)
Time per request:       471.416 [ms] (mean)
Time per request:       0.471 [ms] (mean, across all concurrent requests)
Transfer rate:          1126.09 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   7.5     66    1066
Processing:    74  405 164.6    384    1214
Waiting:       72  405 164.5    384    1213
Total:        134  470 164.9    449    1462

Percentage of the requests served within a certain time (ms)
  50%    449
  66%    529
  75%    577
  80%    607
  90%    678
  95%    739
  98%    825
  99%    954
 100%   1462 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   46.939 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54356541 bytes
HTML transferred:       5600000 bytes
Requests per second:    2130.41 [#/sec] (mean)
Time per request:       469.394 [ms] (mean)
Time per request:       0.469 [ms] (mean, across all concurrent requests)
Transfer rate:          1130.87 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   6.1     66    1058
Processing:    70  403 166.6    377    1439
Waiting:       70  403 166.5    377    1439
Total:        130  468 166.8    442    1511

Percentage of the requests served within a certain time (ms)
  50%    442
  66%    516
  75%    565
  80%    599
  90%    694
  95%    775
  98%    881
  99%    960
 100%   1511 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   47.835 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54354362 bytes
HTML transferred:       5600000 bytes
Requests per second:    2090.52 [#/sec] (mean)
Time per request:       478.351 [ms] (mean)
Time per request:       0.478 [ms] (mean, across all concurrent requests)
Transfer rate:          1109.65 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   9.3     66    1080
Processing:    68  411 182.4    378    1805
Waiting:       68  411 182.4    378    1805
Total:        128  476 182.7    443    1865

Percentage of the requests served within a certain time (ms)
  50%    443
  66%    527
  75%    582
  80%    620
  90%    727
  95%    821
  98%    930
  99%   1004
 100%   1865 (longest request)

P99没有很明显的延迟提升，和之前差不多，说明延迟的原因应该不是这里造成的。

本地限流

本地限流是在每个服务的 Envoy Sidecar 代理中执行的。当一个请求到达 Sidecar 代理时，代理会根据配置的本地限流规则来决定是否允许这个请求通过。这种方式的优点是决策过程在本地完成，不需要额外的网络调用，因此延迟较低。但是，由于每个 Sidecar 代理都独立执行限流决策，所以这种方式可能无法处理跨服务的流量模式

配置 Envoy Filter

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: filter-local-ratelimit-svc
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      # 指定应用
      app: api
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.local_ratelimit
          typed_config:
            "@type": type.googleapis.com/udpa.type.v1.TypedStruct
            type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
            value:
              # 配置允许每分钟 10 个请求
              stat_prefix: http_local_rate_limiter
              token_bucket:
                max_tokens: 10
                tokens_per_fill: 10
                fill_interval: 60s
              filter_enabled:
                runtime_key: local_rate_limit_enabled
                default_value:
                  numerator: 100
                  denominator: HUNDRED
              filter_enforced:
                runtime_key: local_rate_limit_enforced
                default_value:
                  numerator: 100
                  denominator: HUNDRED
              # 添加 x-local-rate-limit 响应头到被阻塞的请求。
              response_headers_to_add:
                - append: false
                  header:
                    key: x-local-rate-limit
                    value: 'true'

执行测试

限流功能测试

上面我们配置了一分钟只能接受10次请求，同样通过脚本进行请求，超出配置的请求次数会拒绝掉请求同时返回 429 响应码。

本地限流（不配置全局限流）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   23.582 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54347244 bytes
HTML transferred:       5600000 bytes
Requests per second:    4240.46 [#/sec] (mean)
Time per request:       235.824 [ms] (mean)
Time per request:       0.236 [ms] (mean, across all concurrent requests)
Transfer rate:          2250.56 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65  10.4     66    1064
Processing:    60  170  55.8    168     485
Waiting:       60  169  55.6    168     457
Total:        120  235  57.0    233    1296

Percentage of the requests served within a certain time (ms)
  50%    233
  66%    254
  75%    267
  80%    275
  90%    299
  95%    323
  98%    360
  99%    437
 100%   1296 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   23.665 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54348148 bytes
HTML transferred:       5600000 bytes
Requests per second:    4225.62 [#/sec] (mean)
Time per request:       236.651 [ms] (mean)
Time per request:       0.237 [ms] (mean, across all concurrent requests)
Transfer rate:          2242.72 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   6.2     66    1066
Processing:    60  170  53.5    171     761
Waiting:       60  170  53.4    170     761
Total:        120  236  54.1    236    1197

Percentage of the requests served within a certain time (ms)
  50%    236
  66%    256
  75%    269
  80%    278
  90%    301
  95%    327
  98%    358
  99%    378
 100%   1197 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   22.425 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54335284 bytes
HTML transferred:       5600000 bytes
Requests per second:    4459.23 [#/sec] (mean)
Time per request:       224.254 [ms] (mean)
Time per request:       0.224 [ms] (mean, across all concurrent requests)
Transfer rate:          2366.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   7.0     66    1066
Processing:    60  158  48.5    154     596
Waiting:       60  157  48.4    154     569
Total:        120  223  49.4    219    1222

Percentage of the requests served within a certain time (ms)
  50%    219
  66%    240
  75%    253
  80%    261
  90%    285
  95%    310
  98%    339
  99%    359
 100%   1222 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$

在使用本地限流时，P99 大概只有 400毫秒左右，和不加 Envoy filter 变化不大。

本地限流 + 全局限流

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   75.996 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54266486 bytes
HTML transferred:       5600000 bytes
Requests per second:    1315.86 [#/sec] (mean)
Time per request:       759.960 [ms] (mean)
Time per request:       0.760 [ms] (mean, across all concurrent requests)
Transfer rate:          697.33 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   66  27.9     66    2081
Processing:    94  691 115.2    686    1742
Waiting:       94  691 115.2    686    1742
Total:        154  757 118.1    752    2605

Percentage of the requests served within a certain time (ms)
  50%    752
  66%    787
  75%    811
  80%    828
  90%    879
  95%    921
  98%    974
  99%   1039
 100%   2605 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   70.649 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54253627 bytes
HTML transferred:       5600000 bytes
Requests per second:    1415.45 [#/sec] (mean)
Time per request:       706.487 [ms] (mean)
Time per request:       0.706 [ms] (mean, across all concurrent requests)
Transfer rate:          749.94 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   7.5     66    1063
Processing:   167  639  81.3    632    1325
Waiting:      167  639  81.3    632    1325
Total:        228  704  81.9    697    1909

Percentage of the requests served within a certain time (ms)
  50%    697
  66%    724
  75%    744
  80%    758
  90%    801
  95%    848
  98%    906
  99%    945
 100%   1909 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$ ab -c 1000 -n 100000  http://test.xxxxx.com/api/healthz
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking test.xxxxx.com (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests


Server Software:        
Server Hostname:        test.xxxxx.com
Server Port:            80

Document Path:          /api/healthz
Document Length:        56 bytes

Concurrency Level:      1000
Time taken for tests:   70.834 seconds
Complete requests:      100000
Failed requests:        0
Total transferred:      54252076 bytes
HTML transferred:       5600000 bytes
Requests per second:    1411.74 [#/sec] (mean)
Time per request:       708.345 [ms] (mean)
Time per request:       0.708 [ms] (mean, across all concurrent requests)
Transfer rate:          747.95 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       59   65   7.5     66    1065
Processing:   139  641  77.7    633    1053
Waiting:      139  641  77.6    633    1053
Total:        199  706  78.2    698    1708

Percentage of the requests served within a certain time (ms)
  50%    698
  66%    729
  75%    751
  80%    765
  90%    807
  95%    844
  98%    891
  99%    918
 100%   1708 (longest request)
lindaoqiang@NeuronMQServerClusterB:~$

从测试报告中看出本地限流配置因为是在sidecar中进行配置，无需RPC、redis等调用，其延迟是最低的，但是本地限流无法像全局限流那个使用 ip+请求方法+请求路径的方式 维度来进行限流。
本地限流可以与全局限流结合使用，以提供不同层面的限流功能。

相关资料

  1. https://istio.io/latest/docs/tasks/policy-enforcement/rate-limit/
  2. Envoy proxy rate limit：https://github.com/envoyproxy/ratelimit
  3. https://www.alibabacloud.com/help/zh/asm/user-guide/current-limiting-protection/
  4. https://github.com/istio/istio/blob/release-1.21/samples/ratelimit
云原生组 > 服务网格
#Envoy
Istio 基于 Envoy 实现服务限流
https://mikeygithub.github.io/2024/04/12/yuque/Istio 基于 Envoy 实现服务限流/
作者
Mikey
发布于
2024年4月12日
许可协议