本文示例说明

• 由于业务情况，采用了6.6.2版本，7版本见ELK文章
• 采用的HostPath方式挂载的目录，建议安装阿里云插件，直接挂载阿里云的云盘
• Service采用注册到阿里云Loadbanlacer，externalTrafficPolicy策略为Cluster

Cluster策略原理：会把集群中所有的Node加到SLB的后端服务器组，转发到的目录节点中如果没有es服务，k8s内部转发（慎用，SLB有后端节点配额限制）
Local策略原理：只会把集群中含有es服务的Node添加到SLB的后端服务器组

service+statefulset的yaml 文件

apiVersion: v1
kind: Service
metadata:
  labels:
    app: elasticsearch
  name: elasticsearch
  namespace: cms-es
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: lb-2ze2w05fw1134irugz9u5
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
spec:
  type: LoadBalancer
  #externalTrafficPolicy: Local
  externalTrafficPolicy: Cluster
  ports:
    - port: 9200
      targetPort: 9200
      protocol: TCP
  selector:
    app: elasticsearch
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: es
  namespace: cms-es
spec:
  serviceName: elasticsearch
  replicas: 3
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      labels: 
        app: elasticsearch
      annotations:
        configmap/checksum: "96ff75e8d9becc33de899087f05053ebec0fb00fffdb6435049bcabea95d6325"
    spec:
      nodeSelector:
        cms-es: cms-es
      initContainers:
      - name: increase-vm-max-map
        image: busybox
        command: ["sysctl", "-w", "vm.max_map_count=262144"]
        securityContext:
          privileged: true
      - name: increase-fd-ulimit
        image: busybox
        command: ["sh", "-c", "ulimit -n 65536"]
        securityContext:
          privileged: true
      containers:
      - name: elasticsearch
        #image: docker.elastic.co/elasticsearch/elasticsearch:6.6.2
        image: registry-vpc.cn-beijing.aliyuncs.com/dataoke-prod/elasticsearch:6.6.2
        ports:
        - name: rest
          containerPort: 9200
        - name: inter
          containerPort: 9300
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sleep","10"]
       # livenessProbe:
       #   httpGet:
       #     path: /
       #     port: 9200
       #   initialDelaySeconds: 240
       #   periodSeconds: 5
       # readinessProbe:
       #   httpGet:
       #     path: /
       #     port: 9200
        readinessProbe:
          tcpSocket:
            port: 9200
          initialDelaySeconds: 30
          periodSeconds: 5
        resources:
          limits:
            cpu: 8192m
            #memory: 30000Mi
          requests:
            cpu: 7000m
            memory: 16384Mi
        volumeMounts:
        - name: data
          mountPath: /usr/share/elasticsearch/data
        env:
        - name: cluster.name
          value: product-cluster
        - name: node.name
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: discovery.zen.ping.unicast.hosts
          value: "es-0.elasticsearch,es-1.elasticsearch,es-2.elasticsearch"
        - name: discovery.zen.minimum_master_nodes
          value: "2"
        - name: ES_JAVA_OPTS
          value: "-Xms16384m -Xmx16384m"
        - name: network.host
          value: "0.0.0.0"
        - name: reindex.remote.whitelist
          value: "192.168.12.200:9200"
#        - name: index.refresh_interval
#          value: "1"
#        - name: path.data
#          value: "/var/lib/container/esdata"
      tolerations:
      - effect: NoSchedule
        key: cmses
        operator: Exists
      volumes:
      - name: data
        hostPath:
          path: "/esdata"
#  volumeClaimTemplates:
#  - metadata:
#      name: data
#      labels:
#        app: elasticsearch
#    spec:
#      accessModes: [ "ReadWriteOnce" ]
#      storageClassName: nfs-1
#      resources:
#        requests:
#          storage: 1024Gi
#---
#
#apiVersion: extensions/v1beta1
#kind: Ingress
#metadata:
#  name: es
#  namespace: logging
##  annotations:
##    nginx.ingress.kubernetes.io/whitelist-source-range: '172.26.0.0/16'
#spec:
#  rules:
#    - host: k8ses.haojiequ.com
#      http:
#        paths:
#        - path: /
#          backend:
#            serviceName: elasticsearch
#            servicePort: 9200

基础条件

node上创建es存储目录

mkdir /esdata && mkdir 777 /esdata

节点打标签、打污点,禁止Pod随意分配上ES节点上

kubectl create ns cms-es
kubectl label Node_name cms-es:cms-es
kubectl taint Node_name cmses:cmses:NoSchedule

执行yaml文件

kubectl create -f es.yaml

重做es镜像，添加ik分词器

cat <<'EOF' > Dockerfile
FROM docker.elastic.co/elasticsearch/elasticsearch:6.6.2
ADD ik.tar.gz /usr/share/elasticsearch/plugins/
EOF

docker build -t registry-vpc.cn-beijing.aliyuncs.com/dataoke-prod/elasticsearch:6.6.2 .
docker push registry-vpc.cn-beijing.aliyuncs.com/dataoke-prod/elasticsearch:6.6.2

高可用测试

jmter压测时随机执行kubectl delete pod es-0 -n cms-es删除pod，零中断

最后

• 探测时不要使用URL探测，ElasticSearch 7 on k8s戳这里