[Elastic Stack] Elasticsearch.yml 파일 세팅

Elasticsearch 다운로드 페이지에서 Elasticsearch를 다운로드하고 설치할 수 있으며 설치 후, elasticsearch.yml 파일을 열어 Elasticsearch를 구성해야 한다.

 

Default location : elasticsearch x.xx.x/config/elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: elastic-sangil
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: elknode01
#
# Add custom attributes to the node:
#
node.roles: ["data_hot","data_content","remote_cluster_client","transform","ml"]
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /data1/elastic/data,/data2/elastic/data
#
# Path to log files:
#
path.logs: /data1/elastic/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 0.0.0.0
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
transport.port: 9300
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#/h
discovery.seed_hosts: ["elkmaster","elknode02","elknode03"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
cluster.initial_master_nodes: ["elkmaster"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# --------------------------------- Readiness ----------------------------------
#
# Enable an unauthenticated TCP readiness endpoint on localhost
#
#readiness.port: 9399
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 07-07-2023 07:02:06
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true
#on for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Discover existing nodes in the cluster
#discovery.seed_hosts: ["127.0.0.1:9300"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication


# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0
#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

 

클러스터(cluster) 이름

cluster.name: elastic-sangil

클러스터명을 설정할 수 있다. Elasticsearch의 노드들은 클러스터명이 같으면 같은 클러스로 묶이고 클러스터명이 다르면 동일한 물리적 장비나 바인딩이 가능한 네트워크상에 있더라도 서로 다른 클러스터로 바인딩 된다.

Default cluster는 "elasticsearch"이며 충돌을 방지하기 위해 클러스터명은 반드시 고유한 이름으로 설정

 

노드(node) 이름

node.name: elknode01

 

노드(node) 역할

node.roles: ["data_hot","data_content","remote_cluster_client","transform","ml"]

master : 마스터 노드 설정

data_content : 일반적으로 쿼리 성능에 최적화되어 있다. 복잡한 검색 및 집계를 처리하고 결과를 신속하게 반환할 수 있도록 IO 처리량보다 처리 능력을 우선시한다.

data_hot : 시계열 데이터를 저장, 자주 사용되는 데이터 / 읽기 쓰기 모두 빠름

data_warm : hot data만큼 정기적으로 업데이트되진 않지만 여전히 쿼리되는 인덱스 저장

data_cold : 자주 사용되지 않는 데이터(읽기 전용 인덱스)를 저장 / 성능이 낮은 하드웨어 사용

ml : 머신러닝노드, 머신러닝 기능을 사용하기 위해 필요한 노드 role

transform : 트랜스폼 api request와 트랜스폼을 담당하는 노드

remote_cluster_client  : 클러스터 간 검색이나 복제 등을 가능하게 해주는 노드 role로 ml과 transform과 같이 쓰인다.

 

데이터/로그 경로

path.data: /data1/elastic/data,/data2/elastic/data

path.logs: /data1/elastic/log

 

네트워크 관련 설정

network.host: 0.0.0.0

transport.port: 9300
http.port: 9200

 

클러스터 관련 설정

# 클러스터를 구성하는 노드 구성 리스트
discovery.seed_hosts: ["elkmaster","elknode02","elknode03"]

cluster.initial_master_nodes: ["elkmaster"]

 

보안 관련 설정

xpack.security.enabled: true

xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12