Adding TLS to your Elastic Web Interface

Once in a while I reset my Elasticsearch config to stay up to date on new developments and see how I can improve my setup.  I thought to add a secure certificate to the front end of my elastic instance.  There’s alot of documents out there but not something that brings it all together.  This is not a config to install x-pack secure authentication and have access to certificate checking.  This blog is strictly installing a secure certificate.

The steps we’ll be following:

• Setup a domain name and associate an A record
• Use LetsEncrypt to generate a new certificate
• Install the certificate in Nginx and Kibana

I performed these steps on a Ubuntu 20.04 server configuration running vanilla Elasticsearch and Kibana 7.15.1.  More information on this installation can be found here.

Setup a Domain (or Subdomain) for a Fully Qualified Domain Name:

I own a domain called secognition.com, I registered it through https://www.gandi.net.  They offer many services relatively inexpensively.  In this case, I wanted to add a subdomain to secognition.com and point the name to my new Elasticsearch server.  I went into my domain configuration gandi.net and added delphi as an A record.  This creates the subdomain and I pointed it to the internal IP of my new server which was 10.255.100.100.  You will want to stay logged into your domain configuration as one more step will be required in the certificate part of the procedure.

Small warning: Adding internal routable IPs to the internet DNS table is not recommended.  Since I don’t have an internal DNS to point to, adding this won’t break anything.  If ever you were to present your Elasticsearch server for use on the internet for folks to browse, you would need to change this internal IP to your new internet IP and modify your internal DNS to point to the internal IP.

Ok so we’ve added the DNS record, saved it and can ping the fully qualified domain name.  Next, setup LetsEncrypt.

LetEncrypt TLS Certificates Installation

The next step involves installing some additional applications on your elasticsearch server.

The command above will install the certbot application to interact with LetsEncrypt.  LetsEncrypt is a service from the Electronic Frontier Foundation (EFF) where they offer free encryption certificates that last for a period of 3 months.  These certificates will need to be renewed, but the certbot can help out by automating this functionality.

Once certbot is installed, we can proceed to generate the certificates and download them to our new server.

certbot --manual certonly -d delphi.yoursever.com -m <your e-mail address> --preferred-challenge dns --agree-tos

The command above tells LetsEncrypt the following:

• We want a manual creation of the certificate. (–manual)
• We only want the certificate and it’s corresponding keys. (certonly)
• -d will be the name of your new domain or subdomain
• -m Is for the e-mail address LetsEncrypt can contact you at when a renewal is coming up and for other EFF related activities.
• –preferred-challenge This is to advise LetsEncrypt that instead of validating your identity through a local webserver on the internet, you will enter a TXT entry on your DNS record to validate that you did request this certificate.
• –agree-tos is to agree to the Terms of Service.

Once you enter the command you’ll be presented with a code to enter into your DNS records.

Something similar to:

_acme-challenge.yourserver.com  with some code after it.

Log back into your domain DNS management screen and create a TXT type record.

You will need to enter the _acme_challenge part as the hostname to your TXT.  No need to add the .yourdomain.com part, this will be done automatically.

You will then enter the random code LetsEncrypt provided into the value part.  Save the entry and press enter to continue the process, the certbot will attempt to validate the new TXT record.   This may take a few minutes. be patient.

Once the validation of the certbot is complete, the certificates will be saved in the /etc/letsencrypt directory.

Install the certificate in Nginx and Kibana

Almost completed.  Now we need to configure the certificates in Nginx and Kibana.  Thankfully, certbot can do this for us for Nginx.

Replace delphi.yourserver.com with your own fully qualified domain name.

You should then see the following choices.  Since we already have the certificates downloaded, choose 1 to not renew and replace the existing certs.

certbot will search for the nginx configuration file for your domain.  The server_name entry should be the same as what you put into the certbot command.  If found, certbot will ask if you want to  modify the file to add the locations of the TLS certificates and give you a choice to redirect unencrypted traffic (port 80) to port 443 https. I chose 2.

Your nginx domain config file should resemble something like this now.

server {

server_name delphi.myserver.com;

location / {
proxy_pass https://localhost:5601;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}

listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/delphi.myserver.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/delphi.myserver.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot


}server {
if ($host = delphi.myserver.com) {
return 301 https://$host$request_uri;
} # managed by Certbot


listen 80;

server_name delphi.myserver.com;
return 404; # managed by Certbot


}

Make sure to modify the domain names (delphi.myserver.com) to your own domain names and restart the nginx process.

dave@delphi:~$ sudo service nginx restart
dave@delphi:~$

For Kibana, we’ll need to copy over the certificates to the /etc/kibana directory.  I created a SSL directory and copied the letsencrypt certificates into the directory then changed the permissions and ownership.  You will have to perform this step everytime the certificates are renewed.

mkdir /etc/kibana/ssl
cp -pr /etc/letsencrypt/live/delphi.myserver.com /etc/kibana/ssl
chmod 750 /etc/kibana/ssl/delphi.myserver.com
chmod 640 /etc/kibana/ssl/delphi.myserver.com/*
chown -R root.kibana /etc/kibana/ssl/delphi.myserver.com/

In Kibana edit your /etc/kibana/kibana.yml config as follows, uncomment the server. lines and add in the directories to your certificates:

server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/ssl/delphi.myserver.com/fullchain1.pem
server.ssl.key: /etc/kibana/ssl/delphi.myserver/privkey1.pem

Restart your kibana and elasticsearch processes to apply the new configuration

dave@delphi:~$ sudo service elasticsearch restart
[sudo] password for dave:
dave@delphi:~$ sudo service kibana restart
dave@delphi:~$

Check the status of your processes

Then attempt to browse using HTTPS to your Elasticsearch server.

I hope this was informative and saved you some time – I spent approximately 2 hours figuring out which entries to add and which to remove to make this work correctly.  If all goes well this should not take more than 30 minutes.

Thanks for reading.  Stay safe and let me know how things went in the comments if there are any steps I may have missed or ways to gain efficiencies in the process.