本文示例说明
- 由于业务情况,采用了6.6.2版本,7版本见ELK文章
- 采用的HostPath方式挂载的目录,建议安装阿里云插件,直接挂载阿里云的云盘
- Service采用注册到阿里云Loadbanlacer,externalTrafficPolicy策略为Cluster
Cluster策略原理:会把集群中所有的Node加到SLB的后端服务器组,转发到的目录节点中如果没有es服务,k8s内部转发(慎用,SLB有后端节点配额限制)
Local策略原理:只会把集群中含有es服务的Node添加到SLB的后端服务器组
service+statefulset的yaml 文件
apiVersion: v1
kind: Service
metadata:
labels:
app: elasticsearch
name: elasticsearch
namespace: cms-es
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: lb-2ze2w05fw1134irugz9u5
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
spec:
type: LoadBalancer
#externalTrafficPolicy: Local
externalTrafficPolicy: Cluster
ports:
- port: 9200
targetPort: 9200
protocol: TCP
selector:
app: elasticsearch
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es
namespace: cms-es
spec:
serviceName: elasticsearch
replicas: 3
selector:
matchLabels:
app: elasticsearch
template:
metadata:
labels:
app: elasticsearch
annotations:
configmap/checksum: "96ff75e8d9becc33de899087f05053ebec0fb00fffdb6435049bcabea95d6325"
spec:
nodeSelector:
cms-es: cms-es
initContainers:
- name: increase-vm-max-map
image: busybox
command: ["sysctl", "-w", "vm.max_map_count=262144"]
securityContext:
privileged: true
- name: increase-fd-ulimit
image: busybox
command: ["sh", "-c", "ulimit -n 65536"]
securityContext:
privileged: true
containers:
- name: elasticsearch
#image: docker.elastic.co/elasticsearch/elasticsearch:6.6.2
image: registry-vpc.cn-beijing.aliyuncs.com/dataoke-prod/elasticsearch:6.6.2
ports:
- name: rest
containerPort: 9200
- name: inter
containerPort: 9300
lifecycle:
preStop:
exec:
command: ["/bin/sleep","10"]
# livenessProbe:
# httpGet:
# path: /
# port: 9200
# initialDelaySeconds: 240
# periodSeconds: 5
# readinessProbe:
# httpGet:
# path: /
# port: 9200
readinessProbe:
tcpSocket:
port: 9200
initialDelaySeconds: 30
periodSeconds: 5
resources:
limits:
cpu: 8192m
#memory: 30000Mi
requests:
cpu: 7000m
memory: 16384Mi
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
env:
- name: cluster.name
value: product-cluster
- name: node.name
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: discovery.zen.ping.unicast.hosts
value: "es-0.elasticsearch,es-1.elasticsearch,es-2.elasticsearch"
- name: discovery.zen.minimum_master_nodes
value: "2"
- name: ES_JAVA_OPTS
value: "-Xms16384m -Xmx16384m"
- name: network.host
value: "0.0.0.0"
- name: reindex.remote.whitelist
value: "192.168.12.200:9200"
# - name: index.refresh_interval
# value: "1"
# - name: path.data
# value: "/var/lib/container/esdata"
tolerations:
- effect: NoSchedule
key: cmses
operator: Exists
volumes:
- name: data
hostPath:
path: "/esdata"
# volumeClaimTemplates:
# - metadata:
# name: data
# labels:
# app: elasticsearch
# spec:
# accessModes: [ "ReadWriteOnce" ]
# storageClassName: nfs-1
# resources:
# requests:
# storage: 1024Gi
#---
#
#apiVersion: extensions/v1beta1
#kind: Ingress
#metadata:
# name: es
# namespace: logging
## annotations:
## nginx.ingress.kubernetes.io/whitelist-source-range: '172.26.0.0/16'
#spec:
# rules:
# - host: k8ses.haojiequ.com
# http:
# paths:
# - path: /
# backend:
# serviceName: elasticsearch
# servicePort: 9200
基础条件
node上创建es存储目录
mkdir /esdata && mkdir 777 /esdata
节点打标签、打污点,禁止Pod随意分配上ES节点上
kubectl create ns cms-es
kubectl label Node_name cms-es:cms-es
kubectl taint Node_name cmses:cmses:NoSchedule
执行yaml文件
kubectl create -f es.yaml
重做es镜像,添加ik分词器
cat <<'EOF' > Dockerfile
FROM docker.elastic.co/elasticsearch/elasticsearch:6.6.2
ADD ik.tar.gz /usr/share/elasticsearch/plugins/
EOF
docker build -t registry-vpc.cn-beijing.aliyuncs.com/dataoke-prod/elasticsearch:6.6.2 .
docker push registry-vpc.cn-beijing.aliyuncs.com/dataoke-prod/elasticsearch:6.6.2
高可用测试
jmter压测时随机执行kubectl delete pod es-0 -n cms-es删除pod,零中断
最后
- 探测时不要使用URL探测,ElasticSearch 7 on k8s戳这里
文章评论