Install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04

In this tutorial, you will learn how to install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04 systems. Elastic Stack 8 is the major release that came with a lot of improvements/enhancements over the previous ELK 7.x.

Install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04

To install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04, proceed as follows;

Install Elastic 8 Repositories on Ubuntu

Download and install the Elastic repository public signing key;

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch \
| sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/elasticsearch-keyring.gpg

Install Elastic APT repository;

echo "deb https://artifacts.elastic.co/packages/8/apt stable main" | \
sudo tee /etc/apt/sources.list.d/elastic-8.list

Update the package cache;

sudo apt update

Install Elasticsearch 8 on Ubuntu

You can install Elasticsearch 8 on Ubuntu by running the command below;

sudo apt install elasticsearch -y

As you might already know, Elastic Stack 8 comes with security enabled by default. This is evident by the Elasticsearch installation output. See example output below;

Setting up elasticsearch (8.5.2) ...
--------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : DkHFDl=j=x2wbmhunCCT

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with 
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with 
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with 
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

-------------------------------------------------------------------------------------------------
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service

As you can see from the above;

  • Authentication and authorization are enabled (password for elastic superadmin user is generated and printed to the output).
  • TLS for the transport and HTTP layers is enabled and configured.

Configure Elasticsearch 8

By default, Elasticsearch configures itself as single-node cluster. So if you are running ELK stack on a single node, nothing much is needed for configurations.

In this tutorial, we are using ELK stack single node. We wont there delve into the cluster setup in this guide.

This is how the default Elasticsearch configuration looks like (with comment lines removed);

cat /etc/elasticsearch/elasticsearch.yml
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes: ["jellyfish"]
http.host: 0.0.0.0

For a single-node cluster, the default configuration is enough to get you started.

However, adjust JVM heap size depending on your RAM size. Elasticsearch by default automatically sets the JVM heap size based on a node’s roles and total memory, which is recommend for most production environments.

In my demo server, i have 3GB of RAM and this is the current default JVM settings;

curl -k -XGET "https://localhost:9200/_cat/nodes?h=heap*&v" -u elastic --cacert /etc/elasticsearch/certs/http_ca.crt

Sample settings;

heap.current heap.percent heap.max
     505.2mb           33    1.4gb

so it is set to almost half my RAM. And so I updated JVM settings as follows;

echo -e "-Xms512M\n-Xmx512M" > /etc/elasticsearch/jvm.options.d/jvm.options

Also, configure Elasticsearch to listen on all interfaces. It is usually bound to loopback address by default;

vim /etc/elasticsearch/elasticsearch.yml

The setting to bind it to all interfaces is. Otherwise specify specific IP instead of 0.0.0.0.

network.host: 0.0.0.0

That is just it. Save and exit the config file.

Start and enable Elasticsearch to run on boot;

sudo systemctl daemon-reload
sudo systemctl enable --now elasticsearch.service

Confirm if all good;

curl -k -XGET https://localhost:9200 -u elastic --cacert /etc/elasticsearch/certs/http_ca.crt

Note the credentials used here are for the elastic user created during Elasticsearch install and is available on the installation output;

Output;

Enter host password for user 'elastic':
{
  "name" : "jellyfish",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "fvQnnRpLR32MxPFqk5racg",
  "version" : {
    "number" : "8.5.2",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "a846182fa16b4ebfcc89aa3c11a11fd5adf3de04",
    "build_date" : "2022-11-17T18:56:17.538630285Z",
    "build_snapshot" : false,
    "lucene_version" : "9.4.1",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

Install Kibana 8 on Ubuntu

Next, install Kibana;

sudo apt install kibana

Sample install output;

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  kibana
0 upgraded, 1 newly installed, 0 to remove and 150 not upgraded.
Need to get 226 MB of archives.
After this operation, 589 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/8/apt stable/main amd64 kibana amd64 8.5.2 [226 MB]
Fetched 226 MB in 1min 37s (2,343 kB/s)                                                                                                                                    
Selecting previously unselected package kibana.
(Reading database ... 74449 files and directories currently installed.)
Preparing to unpack .../kibana_8.5.2_amd64.deb ...
Unpacking kibana (8.5.2) ...
Setting up kibana (8.5.2) ...
Creating kibana group... OK
Creating kibana user... OK
Created Kibana keystore in /etc/kibana/kibana.keystore

You can check related logs as follows;

journalctl -u elasticsearch -f

Configure Kibana

The default settings with a single-node cluster is enough to get you up and running.

To allow external access to Kibana, you need to configure it to bind to non-loopback address by changing the value of the server.host.

vim /etc/kibana/kibana.yml
server.host: "192.168.56.124"

Save and exit the file.

Start Kibana and enable it run on system boot;

sudo systemctl enable --now kibana

Generate Elasticsearch Kibana Enrollment token for secured Kibana connection;

/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana

Sample output;

eyJ2ZXIiOiI4LjUuMiIsImFkciI6WyIxMC4wLjIuMTU6OTIwMCJdLCJmZ3IiOiIyOTU5MzE2YTFmNjIyODkyZWI0ZDVlNjhkNjFmZGY5ZGY0N2JkNGExY2JjNjIxNTczMjhmNzkwZGM5ZGRlNzA4Iiwia2V5Ijoiak9qdXhJUUJqU2ZRSXBja3FGQm46VzgyR2MxZ2pSOVdFdmlVR3U3TDJIQSJ9

Access Kibana Web Interface

Kibana listens on port 5601 by default;

ss -altnp | grep :5601
LISTEN 0      511        192.168.56.124:5601      0.0.0.0:*    users:(("node",pid=4108,fd=18))

To access and activate Kibana, there is a URL that generated and written to Kibana logs when Kibana is started;

journalctl -u kibana -f
Nov 29 20:03:00 jellyfish systemd[1]: Started Kibana.
Nov 29 20:03:02 jellyfish kibana[4108]: [2022-11-29T20:03:02.478+00:00][INFO ][node] Kibana process configured with roles: [background_tasks, ui]
Nov 29 20:03:12 jellyfish kibana[4108]: [2022-11-29T20:03:12.112+00:00][INFO ][plugins-service] Plugin "cloudExperiments" is disabled.
Nov 29 20:03:12 jellyfish kibana[4108]: [2022-11-29T20:03:12.119+00:00][INFO ][plugins-service] Plugin "profiling" is disabled.
Nov 29 20:03:12 jellyfish kibana[4108]: [2022-11-29T20:03:12.187+00:00][INFO ][http.server.Preboot] http server running at http://192.168.56.124:5601
Nov 29 20:03:12 jellyfish kibana[4108]: [2022-11-29T20:03:12.250+00:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
Nov 29 20:03:12 jellyfish kibana[4108]: [2022-11-29T20:03:12.252+00:00][INFO ][preboot] "interactiveSetup" plugin is holding setup: Validating Elasticsearch connection con>
Nov 29 20:03:12 jellyfish kibana[4108]: [2022-11-29T20:03:12.282+00:00][INFO ][root] Holding setup until preboot stage is completed.
Nov 29 20:03:12 jellyfish kibana[4108]: i Kibana has not been configured.
Nov 29 20:03:12 jellyfish kibana[4108]: Go to http://192.168.56.124:5601/?code=283920 to get started.

Pay attention to line; Go to http://192.168.56.124:5601/?code=283920

When prompted for verification later on, it will be 283920.

You need to access the URL from the browser.

Hence, open the port on Firewall;

ufw allow 5601/tcp

Or

iptables -A INPUT -p tcp --dport 5601 -j ACCEPT

Be sure to save the rules.

Then access Kibana address given;

Install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04

Paste the enrollment token generated above and click Configure Elastic.

Next, copy the configuration verification code from address, http://192.168.56.124:5601/?code=283920

Or you can simply run the command below to get the code;

/usr/share/kibana/bin/kibana-verification-code
Your verification code is:  283 920

Enter the code and verify;

Install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04

Click Verify.

If it fails with, Couldn’t configure Elastic, you can manually configure it by clicking Configure manually.

To proceed with manual configuration, first generate the kibana_system user password. This is the user that Kibana uses to connect and communicate with Elasticsearch.

/usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana_system

Sample output;

This tool will reset the password of the [kibana_system] user to an autogenerated value.
The password will be printed in the console.
Please confirm that you would like to continue [y/N]y


Password for the [kibana_system] user successfully reset.
New value: +v*Kg1CYFH+dxDvtV+0a

Next, click Configure manually and enter Elasticsearch address;

Install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04

Check address to confirm connectivity.

Enter Kibana system user password and confirm that you recognize the certificate.

Install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04

Again Configure Elastic and re-enter the verification code. You might want to retrieve it again using the command, /usr/share/kibana/bin/kibana-verification-code.

If all is good, Kibana-Elasticsearch connection is established and you are taken to login page.

Login as elastic superadmin user.

Install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04

And there you go.

Install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04

And there you go.

Logstash component is optional and you can only use it if you require custom processing/parsing if your logs.

That closes our guide on how to install ELK Stack 8 on Ubuntu 22.04/Ubuntu 20.04.

Proceed to install beats to collect logs and push them into ELK stack;

Install Filebeat 8 on Rocky Linux

Other Tutorials

Monitor HAProxy Logs with ELK Stack

Install ELK Stack 8 on Rocky Linux

Install Wazuh Server with ELK Stack on Debian 11

Founder of itnixpro.com|Linux Engineer|Author at Itnixpro.com

Leave a Comment