땅지원
땅지원's Personal blog
땅지원
전체 방문자
오늘
어제
  • 전체 (353)
    • Frontend (2)
      • React (2)
    • Backend (90)
      • Java (16)
      • Python (19)
      • Spring (23)
      • Database (21)
      • Troubleshooting (8)
    • DevOps (27)
      • ELK (13)
    • CS (40)
    • OS (2)
      • Linux (2)
    • Algorithm (95)
      • concept (18)
      • Algorithm Problem (77)
    • 인공지능 (25)
      • 인공지능 (12)
      • 연구노트 (13)
    • 수업정리 (35)
      • 임베디드 시스템 (10)
      • 데이터통신 (17)
      • Linux (8)
    • 한국정보통신학회 (5)
      • 학술대회 (4)
      • 논문지 (1)
    • 수상기록 (8)
      • 수상기록 (6)
      • 특허 (2)
    • 삼성 청년 SW 아카데미 (6)
    • 42seoul (12)
    • Toy project (3)
    • 땅's 낙서장 (2)

블로그 메뉴

  • 홈
  • 태그
  • 방명록

공지사항

  • 20.11.6 BB21플러스 온라인 학술대회
  • 20.10.30 한국정보통신학회 온라인 학술대회

인기 글

태그

  • E
  • D
  • ㅗ
  • I
  • 이것이 리눅스다 with Rocky Linux9

최근 댓글

최근 글

티스토리

hELLO · Designed By 정상우.
땅지원

땅지원's Personal blog

[Elastic Stack] 검색어 자동완성 구현(Spring Data Elasticsearch)
DevOps/ELK

[Elastic Stack] 검색어 자동완성 구현(Spring Data Elasticsearch)

2023. 5. 4. 11:12

이번엔 평소에 해보고싶었던 ElasticSearch를 이용하여 검색어 자동완성 구현을 해보려고 한다.

 

웹 개발을 한다면 검색 엔진이 필수인데 이런 기능을 개발한다니 너무 재밌었다

 

기존에 이런 기능을 구현한 적이 있었는데 SQL의 Like 써가면서 했는데 데이터가 많아지다보니 아무리봐도 이건 아닌거같아서 기능을 찾던 도중 ElasticSearch를 찾았고 SpringBoot로 개발하다보니 Spring Data ElasticSearch가 있다는걸 알아서 바로 적용해봤다.

 

ElasticSearch 설치

일단 로컬이나 ec2에 ElasticSearch를 설치해야한다. 난 ec2를 사용하고 있으니 편리한 Docker를 이용해서 설치했당

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.9.1
docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --name elasticsearch7 docker.elastic.co/elasticsearch/elasticsearch:7.9.1

Kibana 설치

Elasticsearch 데이터를 편리하게 보기위해서 설치

docker pull docker.elastic.co/kibana/kibana:7.9.1
docker run -d --link elasticsearch7:elasticsearch -p 5601:5601 --name kibana7 docker.elastic.co/kibana/kibana:7.9.1

 

Config & Util

@Configuration
@ComponentScan(basePackages = { "com.moham.coursemores.common.elasticsearch" })
public class ElasticSearchConfig extends AbstractElasticsearchConfiguration {

    @Bean
    @Override
    public RestHighLevelClient elasticsearchClient() {
        final ClientConfiguration config = ClientConfiguration.builder()
                .connectedTo("엘라스틱서치 url주소")
                .build();
        return RestClients.create(config).rest();
    }
}
package com.moham.coursemores.common.util;

public final class Indices {
    public static final String COURSE_INDEX = "course";
    public static final String COURSELOCATION_INDEX = "courselocation";
    public static final String HASHTAG_INDEX = "hashtag";

}
package com.moham.coursemores.common.util;

import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.index.query.Operator;

import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.index.query.*;

public class SearchUtil {
    private SearchUtil() {}

    public static SearchRequest buildSearchRequest(String indexName, String value){
        try {
            SearchSourceBuilder builder = new SearchSourceBuilder()
                    .postFilter(getQueryBuilder(value));

            SearchRequest request = new SearchRequest(indexName);
            request.source(builder);

            return request;
        } catch (Exception e) {
            e.printStackTrace();
            return null;
        }
    }

    private static QueryBuilder getQueryBuilder(String value){
        if (value == null)
            return null;

        return QueryBuilders.wildcardQuery("value", "*" + value + "*");
    }
}

QueryBuilder.wildcareQuery를 이용해서 value가 들어간 모든 데이터를 읽어드린다

 

 

 

Service

package com.moham.coursemores.service.impl;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.moham.coursemores.common.util.Indices;
import com.moham.coursemores.common.util.SearchUtil;
import com.moham.coursemores.domain.document.CourseDocument;
import com.moham.coursemores.domain.document.CourseLocationDocument;
import com.moham.coursemores.domain.document.HashtagDocument;
import com.moham.coursemores.dto.elasticsearch.IndexDataReqDTO;
import com.moham.coursemores.dto.elasticsearch.IndexDataResDTO;
import com.moham.coursemores.service.ElasticSearchService;
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Map;
import lombok.RequiredArgsConstructor;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.action.DocWriteResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;

import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.rest.RestStatus;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.xcontent.XContentType;
import org.springframework.stereotype.Service;
import org.elasticsearch.action.search.SearchRequest;

import java.util.List;

@Service
@RequiredArgsConstructor
public class ElasticSearchServiceImpl implements ElasticSearchService {
    private static final ObjectMapper MAPPER = new ObjectMapper();
    private final RestHighLevelClient client;

    public Boolean index(CourseDocument courseDocument) {
        try {
            String value = MAPPER.writeValueAsString(courseDocument);

            IndexRequest request = new IndexRequest(Indices.COURSE_INDEX);

            request.id(courseDocument.getId());
            request.source(value, XContentType.JSON);

            IndexResponse response = client.index(request, RequestOptions.DEFAULT);

            return response != null && response.status().equals(RestStatus.OK);
        } catch (final Exception e) {
            e.printStackTrace();
            return false;
        }
    }

    public Boolean indexLoction(CourseLocationDocument courseLocationDocument) {
        try {
            String value = MAPPER.writeValueAsString(courseLocationDocument);

            IndexRequest request = new IndexRequest(Indices.COURSELOCATION_INDEX);
            request.id(courseLocationDocument.getId());
            request.source(value, XContentType.JSON);

            IndexResponse response = client.index(request, RequestOptions.DEFAULT);

            return response != null && response.status().equals(RestStatus.OK);
        } catch (final Exception e) {
            e.printStackTrace();
            return false;
        }
    }

    public Boolean indexHashtag(HashtagDocument hashtagDocument) {
        try {
            String value = MAPPER.writeValueAsString(hashtagDocument);

            IndexRequest request = new IndexRequest(Indices.HASHTAG_INDEX);
            request.id(hashtagDocument.getId());
            request.source(value, XContentType.JSON);

            IndexResponse response = client.index(request, RequestOptions.DEFAULT);

            return response != null && response.status().equals(RestStatus.OK);
        } catch (final Exception e) {
            e.printStackTrace();
            return false;
        }
    }

    public IndexDataResDTO search(String value) throws IOException {
        SearchRequest requestCourse = SearchUtil.buildSearchRequest(Indices.COURSE_INDEX, value);
        SearchRequest requestCourseLocation = SearchUtil.buildSearchRequest(Indices.COURSELOCATION_INDEX, value);
        SearchRequest requestHashtag = SearchUtil.buildSearchRequest(Indices.HASHTAG_INDEX, value);

        SearchResponse responseCourse = client.search(requestCourse, RequestOptions.DEFAULT);
        SearchResponse responseCourseLocation = client.search(requestCourseLocation, RequestOptions.DEFAULT);
        SearchResponse responseHashtag = client.search(requestHashtag, RequestOptions.DEFAULT);

        SearchHit[] searchHitsCourse = responseCourse.getHits().getHits();
        SearchHit[] searchHitsCourseLocation = responseCourseLocation.getHits().getHits();
        SearchHit[] searchHitsHashtag = responseHashtag.getHits().getHits();



        List<String> courses = new ArrayList<>();

        Map<String, Object> sourceAsMap;
        for (SearchHit hit : searchHitsCourse) {
            sourceAsMap = hit.getSourceAsMap();
            courses.add((String) sourceAsMap.get("value"));
        }

        List<String> courseLocations = new ArrayList<>();
        for (SearchHit hit : searchHitsCourseLocation) {
            sourceAsMap = hit.getSourceAsMap();
            courseLocations.add((String) sourceAsMap.get("value"));
        }

        List<String> hashtags = new ArrayList<>();
        for (SearchHit hit : searchHitsHashtag) {
            sourceAsMap = hit.getSourceAsMap();
            hashtags.add((String) sourceAsMap.get("value"));
        }


        IndexDataResDTO indexDataResDTO = IndexDataResDTO.builder()
                .courses(courses)
                .courseLocations(courseLocations)
                .hashtags(hashtags)
                .build();
        return indexDataResDTO;

    }

    public void updateCourseDocument(String index, String id, String value) throws IOException {
        UpdateRequest request = null;

        if ("course".equals(index)) {
            request = new UpdateRequest(Indices.COURSE_INDEX, id);
        } else if ("courselocation".equals(index)) {
            request = new UpdateRequest(Indices.COURSELOCATION_INDEX, id);
        } else if ("hashtag".equals(index)) {
            request = new UpdateRequest(Indices.HASHTAG_INDEX, id);
        }

        Map<String, Object> jsonMap = new HashMap<>();
        jsonMap.put("value",value);
        request.doc(jsonMap);

        UpdateResponse response = client.update(request, RequestOptions.DEFAULT);
    }

    public void deleteCoureDocument(String index,String id) throws IOException {
        DeleteRequest request = null;

        if ("course".equals(index)) {
            request = new DeleteRequest(Indices.COURSE_INDEX, id);
        } else if ("courselocation".equals(index)) {
            request = new DeleteRequest(Indices.COURSELOCATION_INDEX, id);
        } else if ("hashtag".equals(index)) {
            request = new DeleteRequest(Indices.HASHTAG_INDEX, id);
        }

        DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);

        if (response.getResult() == DocWriteResponse.Result.NOT_FOUND) {
            throw new ElasticsearchException("Document not found with id " + id + " in index course");
        }

    }
}

 

 

Controller

@PostMapping()
public void insert(@RequestBody IndexDataReqDTO indexDataReqDTO) {
    String id = indexDataReqDTO.getId();
    String index = indexDataReqDTO.getIndex();
    String value = indexDataReqDTO.getValue();

    logger.info(">> request : index={}, id={}, value={}", id, index, value);

    if ("course".equals(index)) {
        CourseDocument courseDocument = CourseDocument.builder().id(id).value(value).build();
        courseSearchService.index(courseDocument);
    } else if ("courselocation".equals(index)) {
        CourseLocationDocument courseLocationDocument = CourseLocationDocument.builder().id(id).value(value).build();
        courseSearchService.indexLoction(courseLocationDocument);
    } else if ("hashtag".equals(index)) {
        HashtagDocument hashtagDocument = HashtagDocument.builder().id(id).value(value).build();
        courseSearchService.indexHashtag(hashtagDocument);
    }

}
@PostMapping("search")
public ResponseEntity<IndexDataResDTO> search(@RequestBody Map<String, String> map) throws IOException {

    logger.info(">> request : value={}", map.get("value"));

    IndexDataResDTO result = courseSearchService.search(map.get("value"));

    logger.info("<< response : result={}",result);

    return new ResponseEntity<>(result, HttpStatus.OK);
}

@PutMapping()
public ResponseEntity<Map<String, Object>> updateIndexData(@RequestBody IndexDataReqDTO indexDataReqDTO) throws IOException {
    String id = indexDataReqDTO.getId();
    String index = indexDataReqDTO.getIndex();
    String value = indexDataReqDTO.getValue();

    logger.info(">> request : index={}, id={}, value={}", id, index, value);
    courseSearchService.updateCourseDocument(index, id, value);

    return new ResponseEntity<>(HttpStatus.OK);
}

@DeleteMapping()
public ResponseEntity<Map<String, Object>> deleteIndexData(@RequestBody IndexSimpleDataReqDTO indexSimpleDataReqDTO) throws IOException {
    String index = indexSimpleDataReqDTO.getIndex();
    String id = indexSimpleDataReqDTO.getId();

    logger.info(">> request : index={}, id={}", id, index);
    courseSearchService.deleteCoureDocument(index, id);

    return new ResponseEntity<>(HttpStatus.OK);
}

 

최종적으로 value에 "Test"를 입력하면 각 index의 데이터에서 "Test"가 들어가는 모든 데이터를 return 한다

{
    "courses": [
        "Test course 1",
        "Testcourse3",
        "Testcourse4"
    ],
    "courseLocations": [
        "Test courselocation 2",
        "Test courselocation 1",
        "Testcourselocation4",
        "Testcourselocation5"
    ],
    "hashtags": [
        "Test hashtag 1",
        "Test hashtag 2",
        "Testhashtag5"
    ]
}

 

 

 

 

'DevOps > ELK' 카테고리의 다른 글

[Elastic Stack] Elasticsearch Command line tools  (2) 2024.03.19
[Elastic Stack] Elasticsearch.yml 파일 세팅  (0) 2024.03.18
[Elastic Stack] Ingest Pipeline + processor  (0) 2024.03.14
[Elastic Stack] Fleet and Elastic Agent  (0) 2024.03.12
[Elastic Stack] Grok Pattern에 대해  (0) 2024.03.08
    'DevOps/ELK' 카테고리의 다른 글
    • [Elastic Stack] Elasticsearch.yml 파일 세팅
    • [Elastic Stack] Ingest Pipeline + processor
    • [Elastic Stack] Fleet and Elastic Agent
    • [Elastic Stack] Grok Pattern에 대해
    땅지원
    땅지원
    신입 개발자의 우당탕탕 기술 블로그

    티스토리툴바