Austin Songer

Install and Configure Elasticsearch, Logstash, Kibana on Docker

Austin Songer -

This tutorial how to install ELK stack on Docker Containers

Install Docker on Debian-Based Distributions

apt update
apt install apt-transport-https ca-certificates curl software-properties-common -y
echo 'deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable' >> /etc/apt/sources.list.d/docker.list
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

apt update
apt install docker-ce -y
curl -L https://github.com/docker/compose/releases/download/1.20.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

Pull Elasticstack Image & Prepare

git clone https://github.com/elastic/stack-docker /user/share/elastic
sysctl -w vm.max_map_count=262144

Set the PWD Environment Variable

echo 'PWD=/usr/share/elastic/' >> /usr/share/elastic/.env

Create Elasticstack containers

docker-compose -f .\setup.yml up

Save the password given at the end

NOTE: The password will only be given this once

Run the containers

docker-compose up -d

Access Kibana/Elastic

infconfig eth0

http://<IP Address>:5601 for Kibana (with your eth0 IP address)

  • Username: kibana
  • Password: Given earlier

http://<IP Address>:5601 for Elastic (with your eth0 IP address)

  • Username: elastic
  • Password: Given earlier

Enable HTTPS

Configure the values in the config files

# APM
/user/share/elastic/config/apm-server/apm-server.yml

# Auditbeat
/user/share/elastic/config/auditbeat/auditbeat.yml

# Filebeat
/user/share/elastic/config/filebeat/filebeat.yml

# Heartbeat
/user/share/elastic/config/heartbeat/heartbeat.yml

# Metricbeat
/user/share/elastic/config/metricbeat/metricbeat.yml

# Packetbeat
/user/share/elastic/config/packetbeat/packetbeat.yml

setup.kibana:
host: "https://localhost:5601"
protocol: "https"
ssl.enabled: true

Configure the values in /user/share/elastic/config/kibana/kibana.yml

server.ssl.enabled: true
server.ssl.certificate: /usr/share/kibana/config/certs/kibana/kibana.crt
server.ssl.key: /usr/share/kibana/config/certs/kibana/kibana.key

Then restart the stack

docker-compose restart
  • You can then monitor status via docker container ls

NOTE: It can takes a minute or so after containers are "healthy", and more time for Kibana to successfully connect to the elasticsearch service

Glossary

  • Docker is a set of platform as a service products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels.
  • Elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.
  • Logstash is a free and open server-side data processing pipeline that ingests data from a multitude of sources, transforms it, and then sends it to your favorite "stash."
  • Kibana is a data visualization dashboard for Elasticsearch. It provides visualization capabilities on top of the content indexed on an Elasticsearch cluster. Users can create bar, line and scatter plots, or pie charts and maps on top of large volumes of data.