万字长文:详解 Spring Boot 中操作 ElasticSearch
发布日期:2021-06-29 21:47:09 浏览次数:2 分类:技术文章

本文共 72175 字,大约阅读时间需要 240 分钟。

点击上方蓝色“程序猿DD”,选择“设为星标”

回复“资源”获取独家整理的学习资料!

作者 | 超级小豆丁

来源 | http://www.mydlq.club/article/64/

一、ElasticSearch 简介

1、简介

ElasticSearch 是一个基于 Lucene 的搜索服务器。它提供了一个分布式多员工能力的全文搜索引擎,基于 RESTful web 接口。Elasticsearch 是用 Java 语言开发的,并作为 Apache 许可条款下的开放源码发布,是一种流行的企业级搜索引擎。

ElasticSearch 用于云计算中,能够达到实时搜索,稳定,可靠,快速,安装使用方便。

2、特性

  • 分布式的文档存储引擎

  • 分布式的搜索引擎和分析引擎

  • 分布式,支持PB级数据

3、使用场景

  • 搜索领域:如百度、谷歌,全文检索等。

  • 门户网站:访问统计、文章点赞、留言评论等。

  • 广告推广:记录员工行为数据、消费趋势、员工群体进行定制推广等。

  • 信息采集:记录应用的埋点数据、访问日志数据等,方便大数据进行分析。

二、ElasticSearch 基础概念

1、ElaticSearch 和 DB 的关系

在 Elasticsearch 中,文档归属于一种类型 type,而这些类型存在于索引 index 中,我们可以列一些简单的不同点,来类比传统关系型数据库:

  • Relational DB -> Databases -> Tables -> Rows -> Columns

  • Elasticsearch -> Indices -> Types -> Documents -> Fields

Elasticsearch 集群可以包含多个索引 indices,每一个索引可以包含多个类型 types,每一个类型包含多个文档 documents,然后每个文档包含多个字段 Fields。而在 DB 中可以有多个数据库 Databases,每个库中可以有多张表 Tables,没个表中又包含多行Rows,每行包含多列Columns。

2、索引

索引基本概念(indices):

索引是含义相同属性的文档集合,是 ElasticSearch 的一个逻辑存储,可以理解为关系型数据库中的数据库,ElasticSearch 可以把索引数据存放到一台服务器上,也可以 sharding 后存到多台服务器上,每个索引有一个或多个分片,每个分片可以有多个副本。

索引类型(index_type):

索引可以定义一个或多个类型,文档必须属于一个类型。在 ElasticSearch 中,一个索引对象可以存储多个不同用途的对象,通过索引类型可以区分单个索引中的不同对象,可以理解为关系型数据库中的表。每个索引类型可以有不同的结构,但是不同的索引类型不能为相同的属性设置不同的类型。

3、文档

文档(document):

文档是可以被索引的基本数据单位。存储在 ElasticSearch 中的主要实体叫文档 document,可以理解为关系型数据库中表的一行记录。每个文档由多个字段构成,ElasticSearch 是一个非结构化的数据库,每个文档可以有不同的字段,并且有一个唯一的标识符。

4、映射

映射(mapping):

ElasticSearch 的 Mapping 非常类似于静态语言中的数据类型:声明一个变量为 int 类型的变量,以后这个变量都只能存储 int 类型的数据。同样的,一个 number 类型的 mapping 字段只能存储 number 类型的数据。

同语言的数据类型相比,Mapping 还有一些其他的含义,Mapping 不仅告诉 ElasticSearch 一个 Field 中是什么类型的值, 它还告诉 ElasticSearch 如何索引数据以及数据是否能被搜索到。

ElaticSearch 默认是动态创建索引和索引类型的 Mapping 的。这就相当于无需定义 Solr 中的 Schema,无需指定各个字段的索引规则就可以索引文件,很方便。但有时方便就代表着不灵活。比如,ElasticSearch 默认一个字段是要做分词的,但我们有时要搜索匹配整个字段却不行。如有统计工作要记录每个城市出现的次数。对于 name 字段,若记录 new york 文本,ElasticSearch 可能会把它拆分成 new 和 york 这两个词,分别计算这个两个单词的次数,而不是我们期望的 new york。

三、SpringBoot 项目引入 ElasticSearch 依赖

下面介绍下 SpringBoot 如何通过 elasticsearch-rest-high-level-client 工具操作 ElasticSearch,这里需要说一下,为什么没有使用 Spring 家族封装的 spring-data-elasticsearch。

主要原因是灵活性和更新速度,Spring 将 ElasticSearch 过度封装,让开发者很难跟 ES 的 DSL 查询语句进行关联。再者就是更新速度,ES 的更新速度是非常快,但是 spring-data-elasticsearch 更新速度比较缓慢。

由于上面两点,所以选择了官方推出的 Java 客户端 elasticsearch-rest-high-level-client,它的代码写法跟 DSL 语句很相似,懂 ES 查询的使用其上手很快。

示例项目地址:https://github.com/my-dlq/blog-example/tree/master/springboot/springboot-elasticsearch-example

1、Maven 引入相关依赖

  • lombok:lombok 工具依赖。

  • fastjson:用于将 JSON 转换对象的依赖。

  • spring-boot-starter-web:SpringBoot 的 Web 依赖。

  • elasticsearch:ElasticSearch:依赖,需要和 ES 版本保持一致。

  • elasticsearch-rest-high-level-client:用于操作 ES 的 Java 客户端。

    
4.0.0
    
        
org.springframework.boot
        
spring-boot-starter-parent
        
2.2.4.RELEASE
        
 
    
    
club.mydlq
    
springboot-elasticsearch-example
    
0.0.1-SNAPSHOT
    
springboot-elasticsearch-example
    
Demo project for Spring Boot ElasticSearch
    
        
1.8
    
    
        
        
            
org.springframework.boot
            
spring-boot-starter-web
        
        
        
            
org.projectlombok
            
lombok
            
true
        
        
        
            
com.alibaba
            
fastjson
            
1.2.61
        
        
        
            
org.elasticsearch.client
            
elasticsearch-rest-high-level-client
            
6.5.4
        
        
            
org.elasticsearch
            
elasticsearch
            
6.5.4
        
    
    
        
            
                
org.springframework.boot
                
spring-boot-maven-plugin
            
        
    

2、ElasticSearch 连接配置

(1)、application.yml 配置文件

为了方便更改连接 ES 的连接配置,所以我们将配置信息放置于 application.yaml 中:

#baseserver:  port: 8080#springspring:  application:    name: springboot-elasticsearch-example#elasticsearchelasticsearch:  schema: http  address: 127.0.0.1:9200  connectTimeout: 5000  socketTimeout: 5000  connectionRequestTimeout: 5000  maxConnectNum: 100  maxConnectPerRoute: 100

(2)、java 连接配置类

这里需要写一个 Java 配置类读取 application 中的配置信息:

import org.apache.http.HttpHost;import org.elasticsearch.client.RestClient;import org.elasticsearch.client.RestClientBuilder;import org.elasticsearch.client.RestHighLevelClient;import org.springframework.beans.factory.annotation.Value;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;import java.util.ArrayList;import java.util.List;/** * ElasticSearch 配置 */@Configurationpublic class ElasticSearchConfig {    /** 协议 */    @Value("${elasticsearch.schema:http}")    private String schema;    /** 集群地址,如果有多个用“,”隔开 */    @Value("${elasticsearch.address}")    private String address;    /** 连接超时时间 */    @Value("${elasticsearch.connectTimeout:5000}")    private int connectTimeout;    /** Socket 连接超时时间 */    @Value("${elasticsearch.socketTimeout:10000}")    private int socketTimeout;    /** 获取连接的超时时间 */    @Value("${elasticsearch.connectionRequestTimeout:5000}")    private int connectionRequestTimeout;    /** 最大连接数 */    @Value("${elasticsearch.maxConnectNum:100}")    private int maxConnectNum;    /** 最大路由连接数 */    @Value("${elasticsearch.maxConnectPerRoute:100}")    private int maxConnectPerRoute;    @Bean    public RestHighLevelClient restHighLevelClient() {        // 拆分地址        List
 hostLists = new ArrayList<>();        String[] hostList = address.split(",");        for (String addr : hostList) {            String host = addr.split(":")[0];            String port = addr.split(":")[1];            hostLists.add(new HttpHost(host, Integer.parseInt(port), schema));        }        // 转换成 HttpHost 数组        HttpHost[] httpHost = hostLists.toArray(new HttpHost[]{});        // 构建连接对象        RestClientBuilder builder = RestClient.builder(httpHost);        // 异步连接延时配置        builder.setRequestConfigCallback(requestConfigBuilder -> {            requestConfigBuilder.setConnectTimeout(connectTimeout);            requestConfigBuilder.setSocketTimeout(socketTimeout);            requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeout);            return requestConfigBuilder;        });        // 异步连接数配置        builder.setHttpClientConfigCallback(httpClientBuilder -> {            httpClientBuilder.setMaxConnTotal(maxConnectNum);            httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);            return httpClientBuilder;        });        return new RestHighLevelClient(builder);    }}

四、索引操作示例

1、Restful 操作示例

创建索引

创建名为 mydlq-user 的索引与对应 Mapping。

PUT /mydlq-user{  "mappings": {    "doc": {      "dynamic": true,      "properties": {        "name": {          "type": "text",          "fields": {            "keyword": {              "type": "keyword"            }          }        },        "address": {          "type": "text",          "fields": {            "keyword": {              "type": "keyword"            }          }        },        "remark": {          "type": "text",          "fields": {            "keyword": {              "type": "keyword"            }          }        },        "age": {          "type": "integer"        },        "salary": {          "type": "float"        },        "birthDate": {          "type": "date",          "format": "yyyy-MM-dd"        },        "createTime": {          "type": "date"        }      }    }  }}

删除索引

删除 mydlq-user 索引。

DELETE /mydlq-user

2、Java 代码示例

import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;import org.elasticsearch.action.admin.indices.create.CreateIndexResponse;import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;import org.elasticsearch.action.support.master.AcknowledgedResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.common.settings.Settings;import org.elasticsearch.common.xcontent.XContentBuilder;import org.elasticsearch.common.xcontent.XContentFactory;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class IndexService2 {    @Autowired    private RestHighLevelClient restHighLevelClient;    /**     * 创建索引     */    public void createIndex() {        try {            // 创建 Mapping            XContentBuilder mapping = XContentFactory.jsonBuilder()                .startObject()                    .field("dynamic", true)                    .startObject("properties")                        .startObject("name")                            .field("type","text")                            .startObject("fields")                                .startObject("keyword")                                    .field("type","keyword")                                .endObject()                            .endObject()                        .endObject()                        .startObject("address")                            .field("type","text")                            .startObject("fields")                                .startObject("keyword")                                    .field("type","keyword")                                .endObject()                            .endObject()                        .endObject()                        .startObject("remark")                            .field("type","text")                            .startObject("fields")                                .startObject("keyword")                                    .field("type","keyword")                                .endObject()                            .endObject()                        .endObject()                        .startObject("age")                            .field("type","integer")                        .endObject()                        .startObject("salary")                            .field("type","float")                        .endObject()                        .startObject("birthDate")                            .field("type","date")                            .field("format", "yyyy-MM-dd")                        .endObject()                        .startObject("createTime")                            .field("type","date")                        .endObject()                    .endObject()                .endObject();            // 创建索引配置信息,配置            Settings settings = Settings.builder()                    .put("index.number_of_shards", 1)                    .put("index.number_of_replicas", 0)                    .build();            // 新建创建索引请求对象,然后设置索引类型(ES 7.0 将不存在索引类型)和 mapping 与 index 配置            CreateIndexRequest request = new CreateIndexRequest("mydlq-user", settings);            request.mapping("doc", mapping);            // RestHighLevelClient 执行创建索引            CreateIndexResponse createIndexResponse = restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);            // 判断是否创建成功            boolean isCreated = createIndexResponse.isAcknowledged();            log.info("是否创建成功:{}", isCreated);        } catch (IOException e) {            log.error("", e);        }    }    /**     * 删除索引     */    public void deleteIndex() {        try {            // 新建删除索引请求对象            DeleteIndexRequest request = new DeleteIndexRequest("mydlq-user");            // 执行删除索引            AcknowledgedResponse acknowledgedResponse = restHighLevelClient.indices().delete(request, RequestOptions.DEFAULT);            // 判断是否删除成功            boolean siDeleted = acknowledgedResponse.isAcknowledged();            log.info("是否删除成功:{}", siDeleted);        } catch (IOException e) {            log.error("", e);        }    }}

五、文档操作示例

1、Restful 操作示例

增加文档信息

在索引 mydlq-user 中增加一条文档信息。

POST /mydlq-user/doc{    "address": "北京市",    "age": 29,    "birthDate": "1990-01-10",    "createTime": 1579530727699,    "name": "张三",    "remark": "来自北京市的张先生",    "salary": 100}

获取文档信息

获取 mydlq-user 的索引 id=1 的文档信息。

GET /mydlq-user/doc/1

更新文档信息

更新之前创建的 id=1 的文档信息。

PUT /mydlq-user/doc/1{    "address": "北京市海淀区",    "age": 29,    "birthDate": "1990-01-10",    "createTime": 1579530727699,    "name": "张三",    "remark": "来自北京市的张先生",    "salary": 100}

删除文档信息

删除之前创建的 id=1 的文档信息。

DELETE /mydlq-user/doc/1

2、Java 代码示例

import club.mydlq.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.delete.DeleteRequest;import org.elasticsearch.action.delete.DeleteResponse;import org.elasticsearch.action.get.GetRequest;import org.elasticsearch.action.get.GetResponse;import org.elasticsearch.action.index.IndexRequest;import org.elasticsearch.action.index.IndexResponse;import org.elasticsearch.action.update.UpdateRequest;import org.elasticsearch.action.update.UpdateResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.common.xcontent.XContentType;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;import java.util.Date;@Slf4j@Servicepublic class IndexService {    @Autowired    private RestHighLevelClient restHighLevelClient;    /**     * 增加文档信息     */    public void addDocument() {        try {            // 创建索引请求对象            IndexRequest indexRequest = new IndexRequest("mydlq-user", "doc", "1");            // 创建员工信息            UserInfo userInfo = new UserInfo();            userInfo.setName("张三");            userInfo.setAge(29);            userInfo.setSalary(100.00f);            userInfo.setAddress("北京市");            userInfo.setRemark("来自北京市的张先生");            userInfo.setCreateTime(new Date());            userInfo.setBirthDate("1990-01-10");            // 将对象转换为 byte 数组            byte[] json = JSON.toJSONBytes(userInfo);            // 设置文档内容            indexRequest.source(json, XContentType.JSON);            // 执行增加文档            IndexResponse response = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);            log.info("创建状态:{}", response.status());        } catch (Exception e) {            log.error("", e);        }    }    /**     * 获取文档信息     */    public void getDocument() {        try {            // 获取请求对象            GetRequest getRequest = new GetRequest("mydlq-user", "doc", "1");            // 获取文档信息            GetResponse getResponse = restHighLevelClient.get(getRequest, RequestOptions.DEFAULT);            // 将 JSON 转换成对象            if (getResponse.isExists()) {                UserInfo userInfo = JSON.parseObject(getResponse.getSourceAsBytes(), UserInfo.class);                log.info("员工信息:{}", userInfo);            }        } catch (IOException e) {            log.error("", e);        }    }    /**     * 更新文档信息     */    public void updateDocument() {        try {            // 创建索引请求对象            UpdateRequest updateRequest = new UpdateRequest("mydlq-user", "doc", "1");            // 设置员工更新信息            UserInfo userInfo = new UserInfo();            userInfo.setSalary(200.00f);            userInfo.setAddress("北京市海淀区");            // 将对象转换为 byte 数组            byte[] json = JSON.toJSONBytes(userInfo);            // 设置更新文档内容            updateRequest.doc(json, XContentType.JSON);            // 执行更新文档            UpdateResponse response = restHighLevelClient.update(updateRequest, RequestOptions.DEFAULT);            log.info("创建状态:{}", response.status());        } catch (Exception e) {            log.error("", e);        }    }    /**     * 删除文档信息     */    public void deleteDocument() {        try {            // 创建删除请求对象            DeleteRequest deleteRequest = new DeleteRequest("mydlq-user", "doc", "1");            // 执行删除文档            DeleteResponse response = restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);            log.info("删除状态:{}", response.status());        } catch (IOException e) {            log.error("", e);        }    }}

六、插入初始化数据

执行查询示例前,先往索引中插入一批数据:

1、单条插入

POST mydlq-user/_doc

{"name":"零零","address":"北京市丰台区","remark":"低层员工","age":29,"salary":3000,"birthDate":"1990-11-11","createTime":"2019-11-11T08:18:00.000Z"}

2、批量插入

POST _bulk

{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"刘一","address":"北京市丰台区","remark":"低层员工","age":30,"salary":3000,"birthDate":"1989-11-11","createTime":"2019-03-15T08:18:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"陈二","address":"北京市昌平区","remark":"中层员工","age":27,"salary":7900,"birthDate":"1992-01-25","createTime":"2019-11-08T11:15:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"张三","address":"北京市房山区","remark":"中层员工","age":28,"salary":8800,"birthDate":"1991-10-05","createTime":"2019-07-22T13:22:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"李四","address":"北京市大兴区","remark":"高层员工","age":26,"salary":9000,"birthDate":"1993-08-18","createTime":"2019-10-17T15:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"王五","address":"北京市密云区","remark":"低层员工","age":31,"salary":4800,"birthDate":"1988-07-20","createTime":"2019-05-29T09:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"赵六","address":"北京市通州区","remark":"中层员工","age":32,"salary":6500,"birthDate":"1987-06-02","createTime":"2019-12-10T18:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"孙七","address":"北京市朝阳区","remark":"中层员工","age":33,"salary":7000,"birthDate":"1986-04-15","createTime":"2019-06-06T13:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"周八","address":"北京市西城区","remark":"低层员工","age":32,"salary":5000,"birthDate":"1987-09-26","createTime":"2019-01-26T14:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"吴九","address":"北京市海淀区","remark":"高层员工","age":30,"salary":11000,"birthDate":"1989-11-25","createTime":"2019-09-07T13:34:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"郑十","address":"北京市东城区","remark":"低层员工","age":29,"salary":5000,"birthDate":"1990-12-25","createTime":"2019-03-06T12:08:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"萧十一","address":"北京市平谷区","remark":"低层员工","age":29,"salary":3300,"birthDate":"1990-11-11","createTime":"2019-03-10T08:17:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"曹十二","address":"北京市怀柔区","remark":"中层员工","age":27,"salary":6800,"birthDate":"1992-01-25","createTime":"2019-12-03T11:09:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"吴十三","address":"北京市延庆区","remark":"中层员工","age":25,"salary":7000,"birthDate":"1994-10-05","createTime":"2019-07-27T14:22:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"冯十四","address":"北京市密云区","remark":"低层员工","age":25,"salary":3000,"birthDate":"1994-08-18","createTime":"2019-04-22T15:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"蒋十五","address":"北京市通州区","remark":"低层员工","age":31,"salary":2800,"birthDate":"1988-07-20","createTime":"2019-06-13T10:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"苗十六","address":"北京市门头沟区","remark":"高层员工","age":32,"salary":11500,"birthDate":"1987-06-02","createTime":"2019-11-11T18:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"鲁十七","address":"北京市石景山区","remark":"高员工","age":33,"salary":9500,"birthDate":"1986-04-15","createTime":"2019-06-06T14:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"沈十八","address":"北京市朝阳区","remark":"中层员工","age":31,"salary":8300,"birthDate":"1988-09-26","createTime":"2019-09-25T14:00:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"吕十九","address":"北京市西城区","remark":"低层员工","age":31,"salary":4500,"birthDate":"1988-11-25","createTime":"2019-09-22T13:34:00.000Z"}{"index":{"_index":"mydlq-user","_type":"doc"}}{"name":"丁二十","address":"北京市东城区","remark":"低层员工","age":33,"salary":2100,"birthDate":"1986-12-25","createTime":"2019-03-07T12:08:00.000Z"}

3、查询数据

插入完成后再查询数据,查看之前插入的数据是否存在:

GET mydlq-user/_search

执行后得到下面记录:

{  "took": 2,  "timed_out": false,  "_shards": {    "total": 1,    "successful": 1,    "skipped": 0,    "failed": 0  },  "hits": {    "total": 20,    "max_score": 1,    "hits": [      {        "_index": "mydlq-user",        "_type": "_doc",        "_id": "BeN0BW8B7BNodGwRFTRj",        "_score": 1,        "_source": {          "name": "刘一",          "address": "北京市丰台区",          "remark": "低层员工",          "age": 30,          "salary": 3000,          "birthDate": "1989-11-11",          "createTime": "2019-03-15T08:18:00.000Z"        }      },      {        "_index": "mydlq-user",        "_type": "_doc",        "_id": "BuN0BW8B7BNodGwRFTRj",        "_score": 1,        "_source": {          "name": "陈二",          "address": "北京市昌平区",          "remark": "中层员工",          "age": 27,          "salary": 7900,          "birthDate": "1992-01-25",          "createTime": "2019-11-08T11:15:00.000Z"        }      },      {        "_index": "mydlq-user",        "_type": "_doc",        "_id": "B-N0BW8B7BNodGwRFTRj",        "_score": 1,        "_source": {          "name": "张三",          "address": "北京市房山区",          "remark": "中层员工",          "age": 28,          "salary": 8800,          "birthDate": "1991-10-05",          "createTime": "2019-07-22T13:22:00.000Z"        }      },      {        "_index": "mydlq-user",        "_type": "_doc",        "_id": "CON0BW8B7BNodGwRFTRj",        "_score": 1,        "_source": {          "name": "李四",          "address": "北京市大兴区",          "remark": "高层员工",          "age": 26,          "salary": 9000,          "birthDate": "1993-08-18",          "createTime": "2019-10-17T15:00:00.000Z"        }      },      {        "_index": "mydlq-user",        "_type": "_doc",        "_id": "CeN0BW8B7BNodGwRFTRj",        "_score": 1,        "_source": {          "name": "王五",          "address": "北京市密云区",          "remark": "低层员工",          "age": 31,          "salary": 4800,          "birthDate": "1988-07-20",          "createTime": "2019-05-29T09:00:00.000Z"        }      },      {        "_index": "mydlq-user",        "_type": "_doc",        "_id": "CuN0BW8B7BNodGwRFTRj",        "_score": 1,        "_source": {          "name": "赵六",          "address": "北京市通州区",          "remark": "中层员工",          "age": 32,          "salary": 6500,          "birthDate": "1987-06-02",          "createTime": "2019-12-10T18:00:00.000Z"        }      },      {        "_index": "mydlq-user",        "_type": "_doc",        "_id": "C-N0BW8B7BNodGwRFTRj",        "_score": 1,        "_source": {          "name": "孙七",          "address": "北京市朝阳区",          "remark": "中层员工",          "age": 33,          "salary": 7000,          "birthDate": "1986-04-15",          "createTime": "2019-06-06T13:00:00.000Z"        }      },      {        "_index": "mydlq-user",        "_type": "_doc",        "_id": "DON0BW8B7BNodGwRFTRj",        "_score": 1,        "_source": {          "name": "周八",          "address": "北京市西城区",          "remark": "低层员工",          "age": 32,          "salary": 5000,          "birthDate": "1987-09-26",          "createTime": "2019-01-26T14:00:00.000Z"        }      },      {        "_index": "mydlq-user",        "_type": "_doc",        "_id": "DeN0BW8B7BNodGwRFTRj",        "_score": 1,        "_source": {          "name": "吴九",          "address": "北京市海淀区",          "remark": "高层员工",          "age": 30,          "salary": 11000,          "birthDate": "1989-11-25",          "createTime": "2019-09-07T13:34:00.000Z"        }      },      {        "_index": "mydlq-user",        "_type": "_doc",        "_id": "DuN0BW8B7BNodGwRFTRj",        "_score": 1,        "_source": {          "name": "郑十",          "address": "北京市东城区",          "remark": "低层员工",          "age": 29,          "salary": 5000,          "birthDate": "1990-12-25",          "createTime": "2019-03-06T12:08:00.000Z"        }      }    ]  }}

七、查询操作示例

1、精确查询(term)

(1)、Restful 操作示例

精确查询

精确查询,查询地址为 北京市通州区 的人员信息:

查询条件不会进行分词,但是查询内容可能会分词,导致查询不到。之前在创建索引时设置 Mapping 中 address 字段存在 keyword 字段是专门用于不分词查询的子字段。

GET mydlq-user/_search{  "query": {    "term": {      "address.keyword": {        "value": "北京市通州区"      }    }  }}

精确查询-多内容查询

精确查询,查询地址为 北京市丰台区、北京市昌平区 或 北京市大兴区 的人员信息:

GET mydlq-user/_search{  "query": {    "terms": {      "address.keyword": [        "北京市丰台区",        "北京市昌平区",        "北京市大兴区"      ]    }  }}

(2)、Java 代码示例

import club.mydlq.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class TermQueryService {    @Autowired    private RestHighLevelClient restHighLevelClient;    /**     * 精确查询(查询条件不会进行分词,但是查询内容可能会分词,导致查询不到)     */    public void termQuery() {        try {            // 构建查询条件(注意:termQuery 支持多种格式查询,如 boolean、int、double、string 等,这里使用的是 string 的查询)            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.query(QueryBuilders.termQuery("address.keyword", "北京市通州区"));            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        } catch (IOException e) {            log.error("", e);        }    }    /**     * 多个内容在一个字段中进行查询     */    public void termsQuery() {        try {            // 构建查询条件(注意:termsQuery 支持多种格式查询,如 boolean、int、double、string 等,这里使用的是 string 的查询)            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.query(QueryBuilders.termsQuery("address.keyword", "北京市丰台区", "北京市昌平区", "北京市大兴区"));            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        } catch (IOException e) {            log.error("", e);        }    }}

2、匹配查询(match)

(1)、Restful 操作示例

匹配查询全部数据与分页

匹配查询符合条件的所有数据,并且设置以 salary 字段升序排序,并设置分页:

GET mydlq-user/_search{  "query": {    "match_all": {}  },  "from": 0,  "size": 10,  "sort": [    {      "salary": {        "order": "asc"      }    }  ]}

匹配查询数据

匹配查询地址为 通州区 的数据:

GET mydlq-user/_search{  "query": {    "match": {      "address": "通州区"    }  }}

词语匹配查询

词语匹配进行查询,匹配 address 中为 北京市通州区 的员工信息:

GET mydlq-user/_search{  "query": {    "match_phrase": {      "address": "北京市通州区"    }  }}

内容多字段查询

查询在字段 address、remark 中存在 北京 内容的员工信息:

GET mydlq-user/_search{  "query": {    "multi_match": {      "query": "北京",      "fields": ["address","remark"]    }  }}

(2)、Java 代码示例

import club.mydlq.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.MatchAllQueryBuilder;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.elasticsearch.search.sort.SortOrder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class MatchQueryService {    @Autowired    private RestHighLevelClient restHighLevelClient;    /**     * 匹配查询符合条件的所有数据,并设置分页     */    public Object matchAllQuery() {        try {            // 构建查询条件            MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();            // 创建查询源构造器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.query(matchAllQueryBuilder);            // 设置分页            searchSourceBuilder.from(0);            searchSourceBuilder.size(3);            // 设置排序            searchSourceBuilder.sort("salary", SortOrder.ASC);            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        } catch (IOException e) {            log.error("", e);        }    }    /**     * 匹配查询数据     */    public Object matchQuery() {        try {            // 构建查询条件            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.query(QueryBuilders.matchQuery("address", "*通州区"));            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        } catch (IOException e) {            log.error("", e);        }    }    /**     * 词语匹配查询     */    public Object matchPhraseQuery() {        try {            // 构建查询条件            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.query(QueryBuilders.matchPhraseQuery("address", "北京市通州区"));            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        } catch (IOException e) {            log.error("", e);        }    }    /**     * 内容在多字段中进行查询     */    public Object matchMultiQuery() {        try {            // 构建查询条件            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.query(QueryBuilders.multiMatchQuery("北京市", "address", "remark"));            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        } catch (IOException e) {            log.error("", e);        }    }}

3、模糊查询(fuzzy)

(1)、Restful 操作示例

模糊查询所有以 三 结尾的姓名

GET mydlq-user/_search{  "query": {    "fuzzy": {      "name": "三"    }  }}

(2)、Java 代码示例

import club.mydlq.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.common.unit.Fuzziness;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class FuzzyQueryService {    @Autowired    private RestHighLevelClient restHighLevelClient;    /**     * 模糊查询所有以 “三” 结尾的姓名     */    public Object fuzzyQuery() {        try {            // 构建查询条件            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.query(QueryBuilders.fuzzyQuery("name", "三").fuzziness(Fuzziness.AUTO));            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        } catch (IOException e) {            log.error("", e);        }    }}

4、范围查询(range)

(1)、Restful 操作示例

查询岁数 ≥ 30 岁的员工数据:

GET /mydlq-user/_search{  "query": {    "range": {      "age": {        "gte": 30      }    }  }}

查询生日距离现在 30 年间的员工数据:

GET mydlq-user/_search{  "query": {    "range": {      "birthDate": {        "gte": "now-30y"      }    }  }}

(2)、Java 代码示例

import club.mydlq.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class RangeQueryService {    @Autowired    private RestHighLevelClient restHighLevelClient;    /**     * 查询岁数 ≥ 30 岁的员工数据     */    public void rangeQuery() {        try {            // 构建查询条件            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.query(QueryBuilders.rangeQuery("age").gte(30));            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        } catch (IOException e) {            log.error("", e);        }    }    /**     * 查询距离现在 30 年间的员工数据     * [年(y)、月(M)、星期(w)、天(d)、小时(h)、分钟(m)、秒(s)]     * 例如:     * now-1h 查询一小时内范围     * now-1d 查询一天内时间范围     * now-1y 查询最近一年内的时间范围     */    public void dateRangeQuery() {        try {            // 构建查询条件            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            // includeLower(是否包含下边界)、includeUpper(是否包含上边界)            searchSourceBuilder.query(QueryBuilders.rangeQuery("birthDate")                    .gte("now-30y").includeLower(true).includeUpper(true));            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        } catch (IOException e) {            log.error("", e);        }    }}

5、通配符查询(wildcard)

(1)、Restful 操作示例

查询所有以 “三” 结尾的姓名:

GET mydlq-user/_search{  "query": {    "wildcard": {      "name.keyword": {        "value": "*三"      }    }  }}

(2)、Java 代码示例

import club.mydlq.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class WildcardQueryService {    @Autowired    private RestHighLevelClient restHighLevelClient;    /**     * 查询所有以 “三” 结尾的姓名     *     * *:表示多个字符(0个或多个字符)     * ?:表示单个字符     */    public Object wildcardQuery() {        try {            // 构建查询条件            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.query(QueryBuilders.wildcardQuery("name.keyword", "*三"));            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        } catch (IOException e) {            log.error("", e);        }    }}

6、布尔查询(bool)

(1)、Restful 操作示例

查询出生在 1990-1995 年期间,且地址在 北京市昌平区、北京市大兴区、北京市房山区 的员工信息:

GET /mydlq-user/_search{  "query": {    "bool": {      "filter": {        "range": {          "birthDate": {            "format": "yyyy",             "gte": 1990,            "lte": 1995          }        }      },      "must": [        {          "terms": {            "address.keyword": [              "北京市昌平区",              "北京市大兴区",              "北京市房山区"            ]          }        }      ]    }  }}

(2)、Java 代码示例

import club.mydlq.elasticsearch.model.entity.UserInfo;import com.alibaba.fastjson.JSON;import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.index.query.BoolQueryBuilder;import org.elasticsearch.index.query.QueryBuilders;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.SearchHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class BoolQueryService {    @Autowired    private RestHighLevelClient restHighLevelClient;    public Object boolQuery() {        try {            // 创建 Bool 查询构建器            BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();            // 构建查询条件            boolQueryBuilder.must(QueryBuilders.termsQuery("address.keyword", "北京市昌平区", "北京市大兴区", "北京市房山区"))                    .filter().add(QueryBuilders.rangeQuery("birthDate").format("yyyy").gte("1990").lte("1995"));            // 构建查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.query(boolQueryBuilder);            // 创建查询请求对象,将查询对象配置到其中            SearchRequest searchRequest = new SearchRequest("mydlq-user");            searchRequest.source(searchSourceBuilder);            // 执行查询,然后处理响应结果            SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);            // 根据状态和数据条数验证是否返回了数据            if (RestStatus.OK.equals(searchResponse.status()) && searchResponse.getHits().totalHits > 0) {                SearchHits hits = searchResponse.getHits();                for (SearchHit hit : hits) {                    // 将 JSON 转换成对象                    UserInfo userInfo = JSON.parseObject(hit.getSourceAsString(), UserInfo.class);                    // 输出查询信息                    log.info(userInfo.toString());                }            }        }catch (IOException e){            log.error("",e);        }    }}

八、聚合查询操作示例

1、Metric 聚合分析

(1)、Restful 操作示例

统计员工总数、工资最高值、工资最低值、工资平均工资、工资总和:

GET /mydlq-user/_search{  "size": 0,  "aggs": {    "salary_stats": {      "stats": {        "field": "salary"      }    }  }}

统计员工工资最低值:

GET /mydlq-user/_search{  "size": 0,  "aggs": {    "salary_min": {      "min": {        "field": "salary"      }    }  }}

统计员工工资最高值:

GET /mydlq-user/_search{  "size": 0,  "aggs": {    "salary_max": {      "max": {        "field": "salary"      }    }  }}

统计员工工资平均值:

GET /mydlq-user/_search{  "size": 0,  "aggs": {    "salary_avg": {      "avg": {        "field": "salary"      }    }  }}

统计员工工资总值:

GET /mydlq-user/_search{  "size": 0,  "aggs": {    "salary_sum": {      "sum": {        "field": "salary"      }    }  }}

统计员工总数:

GET /mydlq-user/_search{  "size": 0,  "aggs": {    "employee_count": {      "value_count": {        "field": "salary"      }    }  }}

统计员工工资百分位:

GET /mydlq-user/_search{  "size": 0,  "aggs": {    "salary_percentiles": {      "percentiles": {        "field": "salary"      }    }  }}
(2)、Java 代码示例
import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.aggregations.AggregationBuilder;import org.elasticsearch.search.aggregations.AggregationBuilders;import org.elasticsearch.search.aggregations.Aggregations;import org.elasticsearch.search.aggregations.metrics.avg.ParsedAvg;import org.elasticsearch.search.aggregations.metrics.max.ParsedMax;import org.elasticsearch.search.aggregations.metrics.min.ParsedMin;import org.elasticsearch.search.aggregations.metrics.percentiles.ParsedPercentiles;import org.elasticsearch.search.aggregations.metrics.percentiles.Percentile;import org.elasticsearch.search.aggregations.metrics.stats.ParsedStats;import org.elasticsearch.search.aggregations.metrics.sum.ParsedSum;import org.elasticsearch.search.aggregations.metrics.sum.SumAggregationBuilder;import org.elasticsearch.search.aggregations.metrics.valuecount.ParsedValueCount;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;@Slf4j@Servicepublic class AggrMetricService {    @Autowired    private RestHighLevelClient restHighLevelClient;    /**     * stats 统计员工总数、员工工资最高值、员工工资最低值、员工平均工资、员工工资总和     */    public Object aggregationStats() {        String responseResult = "";        try {            // 设置聚合条件            AggregationBuilder aggr = AggregationBuilders.stats("salary_stats").field("salary");            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.aggregation(aggr);            // 设置查询结果不返回,只返回聚合结果            searchSourceBuilder.size(0);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status()) || aggregations != null) {                // 转换为 Stats 对象                ParsedStats aggregation = aggregations.get("salary_stats");                log.info("-------------------------------------------");                log.info("聚合信息:");                log.info("count:{}", aggregation.getCount());                log.info("avg:{}", aggregation.getAvg());                log.info("max:{}", aggregation.getMax());                log.info("min:{}", aggregation.getMin());                log.info("sum:{}", aggregation.getSum());                log.info("-------------------------------------------");            }            // 根据具体业务逻辑返回不同结果,这里为了方便直接将返回响应对象Json串            responseResult = response.toString();        } catch (IOException e) {            log.error("", e);        }        return responseResult;    }    /**     * min 统计员工工资最低值     */    public Object aggregationMin() {        String responseResult = "";        try {            // 设置聚合条件            AggregationBuilder aggr = AggregationBuilders.min("salary_min").field("salary");            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.aggregation(aggr);            searchSourceBuilder.size(0);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status()) || aggregations != null) {                // 转换为 Min 对象                ParsedMin aggregation = aggregations.get("salary_min");                log.info("-------------------------------------------");                log.info("聚合信息:");                log.info("min:{}", aggregation.getValue());                log.info("-------------------------------------------");            }            // 根据具体业务逻辑返回不同结果,这里为了方便直接将返回响应对象Json串            responseResult = response.toString();        } catch (IOException e) {            log.error("", e);        }        return responseResult;    }    /**     * max 统计员工工资最高值     */    public Object aggregationMax() {        String responseResult = "";        try {            // 设置聚合条件            AggregationBuilder aggr = AggregationBuilders.max("salary_max").field("salary");            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.aggregation(aggr);            searchSourceBuilder.size(0);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status()) || aggregations != null) {                // 转换为 Max 对象                ParsedMax aggregation = aggregations.get("salary_max");                log.info("-------------------------------------------");                log.info("聚合信息:");                log.info("max:{}", aggregation.getValue());                log.info("-------------------------------------------");            }            // 根据具体业务逻辑返回不同结果,这里为了方便直接将返回响应对象Json串            responseResult = response.toString();        } catch (IOException e) {            log.error("", e);        }        return responseResult;    }    /**     * avg 统计员工工资平均值     */    public Object aggregationAvg() {        String responseResult = "";        try {            // 设置聚合条件            AggregationBuilder aggr = AggregationBuilders.avg("salary_avg").field("salary");            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.aggregation(aggr);            searchSourceBuilder.size(0);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status()) || aggregations != null) {                // 转换为 Avg 对象                ParsedAvg aggregation = aggregations.get("salary_avg");                log.info("-------------------------------------------");                log.info("聚合信息:");                log.info("avg:{}", aggregation.getValue());                log.info("-------------------------------------------");            }            // 根据具体业务逻辑返回不同结果,这里为了方便直接将返回响应对象Json串            responseResult = response.toString();        } catch (IOException e) {            log.error("", e);        }        return responseResult;    }    /**     * sum 统计员工工资总值     */    public Object aggregationSum() {        String responseResult = "";        try {            // 设置聚合条件            SumAggregationBuilder aggr = AggregationBuilders.sum("salary_sum").field("salary");            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.aggregation(aggr);            searchSourceBuilder.size(0);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status()) || aggregations != null) {                // 转换为 Sum 对象                ParsedSum aggregation = aggregations.get("salary_sum");                log.info("-------------------------------------------");                log.info("聚合信息:");                log.info("sum:{}", String.valueOf((aggregation.getValue())));                log.info("-------------------------------------------");            }            // 根据具体业务逻辑返回不同结果,这里为了方便直接将返回响应对象Json串            responseResult = response.toString();        } catch (IOException e) {            log.error("", e);        }        return responseResult;    }    /**     * count 统计员工总数     */    public Object aggregationCount() {        String responseResult = "";        try {            // 设置聚合条件            AggregationBuilder aggr = AggregationBuilders.count("employee_count").field("salary");            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.aggregation(aggr);            searchSourceBuilder.size(0);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status()) || aggregations != null) {                // 转换为 ValueCount 对象                ParsedValueCount aggregation = aggregations.get("employee_count");                log.info("-------------------------------------------");                log.info("聚合信息:");                log.info("count:{}", aggregation.getValue());                log.info("-------------------------------------------");            }            // 根据具体业务逻辑返回不同结果,这里为了方便直接将返回响应对象Json串            responseResult = response.toString();        } catch (IOException e) {            log.error("", e);        }        return responseResult;    }    /**     * percentiles 统计员工工资百分位     */    public Object aggregationPercentiles() {        String responseResult = "";        try {            // 设置聚合条件            AggregationBuilder aggr = AggregationBuilders.percentiles("salary_percentiles").field("salary");            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.aggregation(aggr);            searchSourceBuilder.size(0);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status()) || aggregations != null) {                // 转换为 Percentiles 对象                ParsedPercentiles aggregation = aggregations.get("salary_percentiles");                log.info("-------------------------------------------");                log.info("聚合信息:");                for (Percentile percentile : aggregation) {                    log.info("百分位:{}:{}", percentile.getPercent(), percentile.getValue());                }                log.info("-------------------------------------------");            }            // 根据具体业务逻辑返回不同结果,这里为了方便直接将返回响应对象Json串            responseResult = response.toString();        } catch (IOException e) {            log.error("", e);        }        return responseResult;    }}

2、Bucket 聚合分析

(1)、Restful 操作示例

按岁数进行聚合分桶,统计各个岁数员工的人数:

GET mydlq-user/_search{  "size": 0,  "aggs": {    "age_bucket": {      "terms": {        "field": "age",        "size": "10"      }    }  }}

按工资范围进行聚合分桶,统计工资在 3000-5000、5000-9000 和 9000 以上的员工信息:

GET mydlq-user/_search{  "aggs": {    "salary_range_bucket": {      "range": {        "field": "salary",        "ranges": [          {            "key": "低级员工",             "to": 3000          },{            "key": "中级员工",            "from": 5000,            "to": 9000          },{            "key": "高级员工",            "from": 9000          }        ]      }    }  }}

按照时间范围进行分桶,统计 1985-1990 年和 1990-1995 年出生的员工信息:

GET mydlq-user/_search{  "size": 10,  "aggs": {    "date_range_bucket": {      "date_range": {        "field": "birthDate",        "format": "yyyy",         "ranges": [          {            "key": "出生日期1985-1990的员工",             "from": "1985",            "to": "1990"          },{            "key": "出生日期1990-1995的员工",             "from": "1990",            "to": "1995"          }        ]      }    }  }}

按工资多少进行聚合分桶,设置统计的最小值为 0,最大值为 12000,区段间隔为 3000:

GET mydlq-user/_search{  "size": 0,  "aggs": {    "salary_histogram": {      "histogram": {        "field": "salary",        "extended_bounds": {          "min": 0,          "max": 12000        },         "interval": 3000      }    }  }}

按出生日期进行分桶:

GET mydlq-user/_search{  "size": 0,  "aggs": {    "birthday_histogram": {      "date_histogram": {        "format": "yyyy",         "field": "birthDate",        "interval": "year"      }    }  }}
(2)、Java 代码示例
import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.aggregations.AggregationBuilder;import org.elasticsearch.search.aggregations.AggregationBuilders;import org.elasticsearch.search.aggregations.Aggregations;import org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramInterval;import org.elasticsearch.search.aggregations.bucket.histogram.Histogram;import org.elasticsearch.search.aggregations.bucket.range.Range;import org.elasticsearch.search.aggregations.bucket.terms.Terms;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;import java.util.List;@Slf4j@Servicepublic class AggrBucketService {    @Autowired    private RestHighLevelClient restHighLevelClient;    /**     * 按岁数进行聚合分桶     */    public Object aggrBucketTerms() {        try {            AggregationBuilder aggr = AggregationBuilders.terms("age_bucket").field("age");            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.size(10);            searchSourceBuilder.aggregation(aggr);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status())) {                // 分桶                Terms byCompanyAggregation = aggregations.get("age_bucket");                List
 buckets = byCompanyAggregation.getBuckets();                // 输出各个桶的内容                log.info("-------------------------------------------");                log.info("聚合信息:");                for (Terms.Bucket bucket : buckets) {                    log.info("桶名:{} | 总数:{}", bucket.getKeyAsString(), bucket.getDocCount());                }                log.info("-------------------------------------------");            }        } catch (IOException e) {            log.error("", e);        }    }    /**     * 按工资范围进行聚合分桶     */    public Object aggrBucketRange() {        try {            AggregationBuilder aggr = AggregationBuilders.range("salary_range_bucket")                    .field("salary")                    .addUnboundedTo("低级员工", 3000)                    .addRange("中级员工", 5000, 9000)                    .addUnboundedFrom("高级员工", 9000);            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.size(0);            searchSourceBuilder.aggregation(aggr);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status())) {                // 分桶                Range byCompanyAggregation = aggregations.get("salary_range_bucket");                List
 buckets = byCompanyAggregation.getBuckets();                // 输出各个桶的内容                log.info("-------------------------------------------");                log.info("聚合信息:");                for (Range.Bucket bucket : buckets) {                    log.info("桶名:{} | 总数:{}", bucket.getKeyAsString(), bucket.getDocCount());                }                log.info("-------------------------------------------");            }        } catch (IOException e) {            log.error("", e);        }    }    /**     * 按照时间范围进行分桶     */    public Object aggrBucketDateRange() {        try {            AggregationBuilder aggr = AggregationBuilders.dateRange("date_range_bucket")                    .field("birthDate")                    .format("yyyy")                    .addRange("1985-1990", "1985", "1990")                    .addRange("1990-1995", "1990", "1995");            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.size(0);            searchSourceBuilder.aggregation(aggr);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status())) {                // 分桶                Range byCompanyAggregation = aggregations.get("date_range_bucket");                List
 buckets = byCompanyAggregation.getBuckets();                // 输出各个桶的内容                log.info("-------------------------------------------");                log.info("聚合信息:");                for (Range.Bucket bucket : buckets) {                    log.info("桶名:{} | 总数:{}", bucket.getKeyAsString(), bucket.getDocCount());                }                log.info("-------------------------------------------");            }        } catch (IOException e) {            log.error("", e);        }    }    /**     * 按工资多少进行聚合分桶     */    public Object aggrBucketHistogram() {        try {            AggregationBuilder aggr = AggregationBuilders.histogram("salary_histogram")                    .field("salary")                    .extendedBounds(0, 12000)                    .interval(3000);            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.size(0);            searchSourceBuilder.aggregation(aggr);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status())) {                // 分桶                Histogram byCompanyAggregation = aggregations.get("salary_histogram");                List
 buckets = byCompanyAggregation.getBuckets();                // 输出各个桶的内容                log.info("-------------------------------------------");                log.info("聚合信息:");                for (Histogram.Bucket bucket : buckets) {                    log.info("桶名:{} | 总数:{}", bucket.getKeyAsString(), bucket.getDocCount());                }                log.info("-------------------------------------------");            }        } catch (IOException e) {            log.error("", e);        }    }    /**     * 按出生日期进行分桶     */    public Object aggrBucketDateHistogram() {        try {            AggregationBuilder aggr = AggregationBuilders.dateHistogram("birthday_histogram")                    .field("birthDate")                    .interval(1)                    .dateHistogramInterval(DateHistogramInterval.YEAR)                    .format("yyyy");            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.size(0);            searchSourceBuilder.aggregation(aggr);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status())) {                // 分桶                Histogram byCompanyAggregation = aggregations.get("birthday_histogram");                List
 buckets = byCompanyAggregation.getBuckets();                // 输出各个桶的内容                log.info("-------------------------------------------");                log.info("聚合信息:");                for (Histogram.Bucket bucket : buckets) {                    log.info("桶名:{} | 总数:{}", bucket.getKeyAsString(), bucket.getDocCount());                }                log.info("-------------------------------------------");            }        } catch (IOException e) {            log.error("", e);        }    }}

3、Metric 与 Bucket 聚合分析

(1)、Restful 操作示例

按照员工岁数分桶、然后统计每个岁数员工工资最高值:

GET mydlq-user/_search

{  "size": 0,  "aggs": {    "salary_bucket": {      "terms": {        "field": "age",        "size": "10"      },      "aggs": {        "salary_max_user": {          "top_hits": {            "size": 1,            "sort": [              {                "salary": {                  "order": "desc"                }              }            ]          }        }      }    }  }}

(2)、Java 代码示例

import lombok.extern.slf4j.Slf4j;import org.elasticsearch.action.search.SearchRequest;import org.elasticsearch.action.search.SearchResponse;import org.elasticsearch.client.RequestOptions;import org.elasticsearch.client.RestHighLevelClient;import org.elasticsearch.rest.RestStatus;import org.elasticsearch.search.SearchHit;import org.elasticsearch.search.aggregations.AggregationBuilder;import org.elasticsearch.search.aggregations.AggregationBuilders;import org.elasticsearch.search.aggregations.Aggregations;import org.elasticsearch.search.aggregations.bucket.terms.Terms;import org.elasticsearch.search.aggregations.metrics.tophits.ParsedTopHits;import org.elasticsearch.search.builder.SearchSourceBuilder;import org.elasticsearch.search.sort.SortOrder;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;import java.util.List;@Slf4j@Servicepublic class AggrBucketMetricService {    @Autowired    private RestHighLevelClient restHighLevelClient;    /**     * topHits 按岁数分桶、然后统计每个员工工资最高值     */    public Object aggregationTopHits() {        try {            AggregationBuilder testTop = AggregationBuilders.topHits("salary_max_user")                    .size(1)                    .sort("salary", SortOrder.DESC);            AggregationBuilder salaryBucket = AggregationBuilders.terms("salary_bucket")                    .field("age")                    .size(10);            salaryBucket.subAggregation(testTop);            // 查询源构建器            SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();            searchSourceBuilder.size(0);            searchSourceBuilder.aggregation(salaryBucket);            // 创建查询请求对象,将查询条件配置到其中            SearchRequest request = new SearchRequest("mydlq-user");            request.source(searchSourceBuilder);            // 执行请求            SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);            // 获取响应中的聚合信息            Aggregations aggregations = response.getAggregations();            // 输出内容            if (RestStatus.OK.equals(response.status())) {                // 分桶                Terms byCompanyAggregation = aggregations.get("salary_bucket");                List
 buckets = byCompanyAggregation.getBuckets();                // 输出各个桶的内容                log.info("-------------------------------------------");                log.info("聚合信息:");                for (Terms.Bucket bucket : buckets) {                    log.info("桶名:{}", bucket.getKeyAsString());                    ParsedTopHits topHits = bucket.getAggregations().get("salary_max_user");                    for (SearchHit hit:topHits.getHits()){                        log.info(hit.getSourceAsString());                    }                }                log.info("-------------------------------------------");            }        } catch (IOException e) {            log.error("", e);        }    }}

扫一扫,关注我

一起学习,一起进步

转载地址:https://didispace-wx.blog.csdn.net/article/details/105479546 如侵犯您的版权,请留言回复原文章的地址,我们会给您删除此文章,给您带来不便请您谅解!

上一篇:Windows中现在有独立的Linux文件夹系统
下一篇:IntelliJ Idea 2020.1 正式发布,官方支持中文了

发表评论

最新留言

路过按个爪印,很不错,赞一个!
[***.219.124.196]2024年04月07日 16时47分20秒