使用 Envoy 对 gRPC 服务进行负载均衡

Kubernetes Service 的负载均衡代理不支持 gRPC，需要通过其他手段来实现。因集群的其他限制，最终选用 Envoy Proxy。

Envoy 可以作为 Sidecar 和服务在一块，不过考虑到集群内有多个 gRPC 服务，故将其作为单独的服务进行部署，这样子，所有 gRPC 请求都指向 Envoy 即可。

截至当前，Envoy 最新稳定版是 v1.17.0。对应的配置文件格式和示例 yaml 如下，其中包含了三种路由匹配示例：gRPC，路由前缀，路由正则。

apiVersion: v1
kind: ConfigMap
metadata:
  name: envoy
  namespace: gateway
data:
  envoy.yaml: |
    static_resources:
      listeners:
      - address:
          socket_address:
            address: 0.0.0.0
            port_value: 10000
        filter_chains:
        - filters:
          - name: envoy.filters.network.http_connection_manager
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              stat_prefix: ingress_http
              route_config:
                name: local_route
                virtual_hosts:
                - name: example
                  domains:
                  - "*"
                  routes:
                  - match:
                      prefix: /example.v1
                      grpc: {}
                    route:
                      cluster: example-grpc
                  - match:
                      prefix: /example/prefix
                    route:
                      cluster: example-http
                      prefix_rewrite: /
                  - match:
                      prefix: /example/regex
                    route:
                      cluster: example-http
                      regex_rewrite:
                        pattern:
                          google_re2: {}
                          regex: ^/example/regex(/.*)$
                        substitution: \1
              http_filters:
              - name: envoy.filters.http.router
                typed_config: {}
      clusters:
      - name: example-grpc
        connect_timeout: 0.5s
        type: strict_dns
        http2_protocol_options: {}
        lb_policy: round_robin
        load_assignment:
          cluster_name: example-grpc
          endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: example.test.svc
                    port_value: 50051
      - name: example-grpc
        connect_timeout: 0.5s
        type: strict_dns
        lb_policy: round_robin
        load_assignment:
          cluster_name: example-http
          endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: example.test.svc
                    port_value: 8080
    admin:
      access_log_path: "/dev/stdout"
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 9090
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: envoy
  namespace: gateway
  labels:
    app: envoy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: envoy
  template:
    metadata:
      labels:
        app: envoy
    spec:
      containers:
      - name: envoy
        image: envoyproxy/envoy-alpine:v1.17.0
        ports:
        - name: http
          containerPort: 10000
        volumeMounts:
        - name: config
          mountPath: /etc/envoy
      volumes:
      - name: config
        configMap:
          name: envoy
---
apiVersion: v1
kind: Service
metadata:
  name: envoy
  namespace: gateway
  labels:
    app: envoy
spec:
  ports:
    - name: envoy
      protocol: TCP
      port: 10000
      targetPort: 10000
  selector:
    app: envoy
  type: ClusterIP

需要注意的是，Envoy 的 admin 接口配置在上述中绑定的是 0.0.0.0，该值需要根据集群实际情况进行配置；因为这里能确保安全，所以可以这样绑定。

如果需要修改 Envoy 的 ConfigMap，改完需要手动重启：

kubectl -n gateway rollout restart deploy envoy