elasticsearch8高级运维 (elasticsearch yum安装教程)

1、centos环境下安装

  • 安装包*载下**
  • *载下** elasticsearch8 安装包 elasticsearch-8.1.0-linux-x86_64.tar.gz ,并*载下**好 jdk-17.0.2 ,解压后目录为 /software/elasticsearch-8.1.0 /software/jdk-17.0.2
  • ES 环境变量配置
export ES_HOME=/software/elasticsearch-8.1.0
export ES_JAVA_HOME=/software/jdk-17.0.2

  • 创建 ES 专用账号(需要使用此账号来管理 ES
useradd elastic
chown elastic /software/elasticsearch-8.1.0 -R

  • 使用 root 账号更改打开的文件数限制及最大虚拟内存限制
查看当前用户文件数限制
ulimit -n 

vi /etc/security/limits.conf 文件最后添加两行

elastic soft nofile 65536
elastic hard nofile 65536

elastic 用户的文件数限制调整为65536

vi /etc/sysctl.conf 文件最后添加一行

vm.max_map_count=655360

使用 sysctl -p 使系统配置生效

  • 屏蔽自动*载下** geopip
vi config/elasticsearch.yml  
修改两项host
network.host: 192.168.91.133
discovery.seed_hosts: ["192.168.91.133"]

并在文件最后添加一行
ingest.geoip.downloader.enabled: false

  • 切换到 elastic 账号启动 ES (加上 -d 参数表示后台启动)
su elastic
cd /software/elasticsearch-8.1.0
./bin/elasticsearch

首次启动后控制台会打印出一些预设置的log信息,以及会在 elasticsearch.yml 配置文件的的最后面增加 SECURITY 相关配置,可以不用理会,因为需要使用时可用脚本工具去修改默认生成的一些配置信息;

2、修改默认配置

第一次启动完成后会自动生成一些信息,接下来修改一下默认密码

  • 修改默认密码
bin/elasticsearch-reset-password --username elastic -i
elastic8888

  • 配置 ssl false (首次启动后才会自动生成 xpack 相关配置项,改为 false 后重启)
xpack.security.http.ssl:
  enabled: false

  • 重新启动后安装结果
curl -u elastic:elastic8888 http://192.168.91.133:9200
{
  "name" : "localhost.localdomain",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "W4dJfYBGTOyJr2S1w2W8Hg",
  "version" : {
    "number" : "8.1.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "3700f7679f7d95e36da0b43762189bab189bc53a",
    "build_date" : "2022-03-03T14:20:00.690422633Z",
    "build_snapshot" : false,
    "lucene_version" : "9.0.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

此时是未开启 ssl 的情况下可以正常访问,同时也可以开启 ssl 完成请求

  • 开启 ssl 访问

打开 config/elasticsearch.yml 中可以看到配置项 xpack.security.http.ssl.keystore.path: certs/http.p12 ,此 http.p12 证书为默认生成的,使用脚本工具重新生成一份替换它,生成命令如下

./bin/elasticsearch-certutil http

Generate a CSR? [y/N] N
Use an existing CA? [y/N] N
Do you wish to change any of these options? [y/N] y

CA Name [CN=Elasticsearch HTTP CA] www.minxyz.com

CA password:  [<ENTER> for none] 123456
Repeat password to confirm: 123456
For how long should your certificate be valid? [5y] [ENTER]
Generate a certificate per node? [y/N]N

Enter all the hostnames that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.

www.minxyz.com     

You entered the following hostnames.

 - www.minxyz.com

Is this correct [Y/n]Y

Key Name: www.minxyz.com
Subject DN: CN=www, DC=minxyz, DC=com
Key Size: 2048

Do you wish to change any of these options? [y/N]N

If you wish to use a blank password, simply press <enter> at the prompt below.
Provide a password for the "http.p12" file:  [<ENTER> for none] 123456
Repeat password to confirm: 123456

What filename should be used for the output zip file? [/software/elasticsearch-8.1.0/elasticsearch-ssl-http.zip] [ENTER]

Zip file written to /software/elasticsearch-8.1.0/elasticsearch-ssl-http.zip


解压 elasticsearch-ssl-http.zip 后,在目录 elasticsearch 中可以看到新生成的文件 http.p12 ,将其替换掉 config/cert/http.p12 ,由于生成此证书时密码为 123456 ,所以需要重新设置 keystore 的密码(需注意不是在配置文件上添加密码配置项),命令如下

./bin/elasticsearch-keystore add "xpack.security.http.ssl.keystore.secure_password"

重新打开 ssl 配置

xpack.security.http.ssl:
  enabled: true

配置 hosts 192.168.91.133 www.minxyz.com (此域名为任意定义的),重启 elasticsearch ,此时需要使用 https 来访问 ES ,可以先增加 -k 参数允许不使用证书连接到 SSL 站点(若浏览器可以强制访问不安全的地址)

curl -k -u elastic:elastic8888 https://www.minxyz.com:9200
{
  "name" : "localhost.localdomain",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "W4dJfYBGTOyJr2S1w2W8Hg",
  "version" : {
    "number" : "8.1.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "3700f7679f7d95e36da0b43762189bab189bc53a",
    "build_date" : "2022-03-03T14:20:00.690422633Z",
    "build_snapshot" : false,
    "lucene_version" : "9.0.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

若需要安全访问,需要增加证书配置,通过以下命令生成 http.pem 后,使用 http.pem 证书完成 https 请求访问

openssl pkcs12 -nodes -in http.p12 -out http.pem
curl --cert ./http.pem --cacert ./http.pem -u elastic:elastic8888 https://www.minxyz.com:9200

在浏览器上需要安全访问,可将 http.p12 证书安装到根证书目录下之后访问

elasticsearch8java,elasticsearchlinux鐜鎼缓

elasticsearch8java,elasticsearchlinux鐜鎼缓

3、ES常用操作

由于直接使用 curl 操作相对较麻烦,临时扩展定义一个 ecurl 别名,如下(后面即可用 ecurl 简化操作)

alias ecurl='curl --cert ./http.pem --cacert ./http.pem -u elastic:elastic8888'

  • 查看 es 版本信息
ecurl https://www.minxyz.com:9200

  • 查看所有索引
ecurl https://www.minxyz.com:9200/_cat/indices?v

  • 新增索引 新增完成后查看索引的状态是 yellow ,默认有一个副本
ecurl -X PUT https://www.minxyz.com:9200/user-index

  • 修改副本数 非集群状态下修改完副本数后即变为 green
ecurl -X PUT -H "content-type:application/json;charset=utf-8" https://www.minxyz.com:9200/user-index/_settings -d "{\"number_of_replicas\": 0}"

  • 查看索引的 mapping
ecurl https://www.minxyz.com:9200/user-index/_mapping?pretty

  • 新增索引对应的 mapping (其中 user-index 为索引名)
ecurl -X POST --cert ./http.pem --cacert ./http.pem -u elastic:elastic8888 -H "content-type:application/json;charset=utf-8" https://www.minxyz.com:9200/user-index/_mapping -d \
"{                                                \
     \"properties\": {                            \
         \"name\": {                              \
             \"type\": \"keyword\"                \
         },                                       \
         \"type\": {                              \
             \"type\": \"integer\"                \
         },                                       \
         \"content\": {                           \
             \"type\": \"text\"                   \
         }                                        \
     }                                            \
 }"


  • 向索引里面新增测试数据
ecurl -X POST -H "content-type:application/json;charset=utf-8" https://www.minxyz.com:9200/user-index/_create/1 -d "{\"name\":\"测试姓名\",\"type\":100,\"content\":\"测试永远滴神啊。。。\"}"

  • 根据文档 id 查找文档
ecurl https://www.minxyz.com:9200/user-index/_doc/1?pretty
或
ecurl https://www.minxyz.com:9200/user-index/_search?q=_id:1

  • 检索所有文档
ecurl -X POST -H "Content-Type:application/json;charset=UTF-8" "https://www.minxyz.com:9200/user-index/_search?pretty" -d  "{\"query\":{\"match_all\":{}}}"

  • 按查询条件查询
ecurl -X POST -H "content-type:application/json;charset=utf-8" https://www.minxyz.com:9200/user-index/_search?pretty -d \
"{                                     \
    \"query\": {                       \
        \"match\": {                   \
            \"content\": \"永远滴神\"  \
        }                              \
    }                                \
}"

  • 根据文档 id 删除文档
ecurl -X DELETE https://www.minxyz.com:9200/user-index/_doc/1?pretty

  • 查看全部索引配置或某个索引配置
ecurl https://www.minxyz.com:9200/_settings/_all?pretty
ecurl https://www.minxyz.com:9200/user-index/_settings?pretty

  • 创建、查看与删除索引别名
创建索引别名
ecurl -X POST -H "content-type:application/json;charset=utf-8" https://www.minxyz.com:9200/_aliases -d'
{                                      
    "actions": [{                      
            "add": {                   
                "index": "test-create-3", 
                "alias": "article-index-master"   
            }                          
        }                              
    ]                                  
}
'

查看索引别名
ecurl https://www.minxyz.com:9200/_aliases?pretty

删除索引别名
ecurl -X POST https://www.minxyz.com:9200/_aliases -H 'Content-Type: application/json' -d'
{
    "actions": [
        {
            "remove": {
                "index": "test-create-3",
                "aliases": "article-index-master"
            }
        }
    ]
}
'

  • 删除索引
ecurl -X DELETE https://www.minxyz.com:9200/user-index

  • 查看分片信息
ecurl https://www.minxyz.com:9200/_cat/shards?v

  • 查看集群健康状态
ecurl https://www.minxyz.com:9200/_cluster/health?pretty=true

  • 查看副本信息
ecurl -X GET "https://www.minxyz.com:9200/_cluster/allocation/explain?pretty" -H 'Content-Type: application/json' -d'
{
  "index": "user-index",
  "shard": 0,
  "primary": false,
  "current_node": "localhost.localdomain"  
}
'

  • 查看分片详情
ecurl https://www.minxyz.com:9200/_cat/allocation?v

4、中文分词

安装中文分词插件 sudo bin/elasticsearch-plugin install analysis-smartcn ,安装成功后重启 es

  • 分词分析
ecurl -X POST  -H 'Content-Type: application/json' https://www.minxyz.com:9200/_analyze?pretty -d'
{
	"analyzer":"smartcn",
	"text":"测试永远滴神啊。。。"
}'

  • 自定义中文词库 安装 IK 中文分词器
./elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v8.1.0/elasticsearch-analysis-ik-8.1.0.zip

比如要将“永远滴神”定义为一个词,需要在配置 IKAnalyzer.cfg.xml 中指定 <entry key="ext_dict">customer.dic</entry> ,然后创建文件 customer.dic ,并添加一行“永远滴神”,重启后再次访问即可看到分词结果

 ecurl -X POST  -H 'Content-Type: application/json' https://www.minxyz.com:9200/_analyze?pretty -d'
> {
> "analyzer":"ik_smart",
> "text":"测试永远滴神啊。。。"
> }'

{
  "tokens" : [
    {
      "token" : "测试",
      "start_offset" : 0,
      "end_offset" : 2,
      "type" : "CN_WORD",
      "position" : 0
    },
    {
      "token" : "永远滴神",
      "start_offset" : 2,
      "end_offset" : 6,
      "type" : "CN_WORD",
      "position" : 1
    },
    {
      "token" : "啊",
      "start_offset" : 6,
      "end_offset" : 7,
      "type" : "CN_CHAR",
      "position" : 2
    }
  ]
}