Part One: How to Monitor Nginx using Elastic Stack on a CentOS 7 VPS or Dedicated Server

Installing and Configuring Elastic Stack Components

The Elastic Stack is a suite of innovative, open-source tools used to analyze different sets of data. The Stack includes tools such as Metricbeat and Filebeat which are useful in collecting web server and system logs. The tools then send the logs to Elasticsearch where they are analyzed, searched, and visualized using a browser-based application called Kibana.

On the other hand, Nginx is a robust web server which can serve as a load balancer for (HTTPS, UDP, and TCP), HTTP cache, mail proxy server (POP3, IMAP, and SMTP), or reverse proxy. It's powerful, open-source software designed to deliver optimal stability and performance.

This tutorial is compiled to help you learn how to install the different components of the Elastic Stack.

Ready to go? Let's roll!

Pre-installation instructions

  • All commands that demand eminent prerogatives are prefixed with sudo.
  • Familiarize yourself with setting up your Linode’s timezone and hostname
  • Before you embark on the installations, you must configure your web server stack with Nginx on a CentOS server.
  • Create a secure, standard user account, remove unnecessary network services, and harden your SSH access.

Once you meet all the above conditions, you can begin the installation. Run the command below to update your system:

$ sudo yum update

Install OpenJDK 8

Next, install OpenJDK 8 application on CentOS 7, since Elasticsearch requires the latest version of Java.Run the command below to institute the headless package of OpenJDK:

$ sudo yum install -y java-1.8.0-openjdk-headless

Check if the version you have installed is the latest version (at least Java 1.8.0). Use the command:

$ java -version

If the installed version is similar to this:

openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

Then, your Java is prime for utilization by Elasticsearch. You can now proceed with the next step.

Install the Elastic Yum Repository

The Elastic Yum Repository is a comprehensive suite that contains all the packages we need for this tutorial.

Use the command below to import the Elastic signing key:

$ sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Next, develop a Yum repository to utilize the Elastic yum repository:

elastic.repo

[elasticsearch-6.x]
name=Elastic repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Run the command below to update the Yum cache and facilitate the availability of all new packages:

$ sudo yum update

Install the stack components

We intend to use various components of the Elastic Stack for machine metrics and log analysis. Therefore, it's important you install the following parts:

Elasticsearch

Elasticsearch stores the metrics and logs received from each beat. For this reason, we’ll install and configure it first to provide the datastore for Kibana and Beats. Run the command below to install the Elasticsearch package:

$ sudo yum install -y elasticsearch

Next, set the Java Virtual Machine heap size to half of the available server memory. Access the /etc/elasticsearch/jvm.options file and modify the Xmx and Xms values. Do not change the other values:

/etc/elasticsearch/jvm.options

-Xms512m
-Xmx512m

At this point, we’ll install two very important plugins that enable Filebeat to process and parse certain docs accurately. The first plugin is the ingest-user-agent, which makes it easy for Elasticsearch to flawlessly parse user-agent strings. Run the command below:

$ sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-user-agent

The second plugin is the Geoip processor.

Note: If the elasticsearch-plugin request for permissions to edit the /etc/elasticsearch path, confirm, then run the command below:

$ sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-geoip

Next, start and enable Elasticsearch service:

$ sudo systemctl enable elasticsearch
$ sudo systemctl start elasticsearch

The service may take some minutes to start, give it time. Then check the available Elasticsearch API:

$ curl localhost:9200

To check if the service started successfully, run the following command to view the latest logs:

$ systemctl status elasticsearch

The Elasticsearch API will deliver a JSON response, such as the one below:

{
  "name" : "Q1R2Oz7",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "amcxppmvTkmuApEdTz673A",
  "version" : {
    "number" : "6.0.0",
    "build_hash" : "8f0685b",
    "build_date" : "2017-11-10T18:41:22.859Z",
    "build_snapshot" : false,
    "lucene_version" : "7.0.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Once the Elasticsearch application is successfully installed, you can now install the other components.

Filebeat

Run the command below to install Filebeat:

$ sudo yum install filebeat

Metricbeat

Use this command to install this package:

$ sudo yum install metricbeat

Kibana

Next, install Kibana packae using:

sudo yum install kibana

Configure the Stack

By now you have installed all the necessary component, but they are not properly configured. Next we configure these components

ElasticSearch

For the sake of this tutorial, we need to change the default Elasticsearch settings since we’ve used a single server in the setup.

First, create a makeshift JSON file featuring an index template which directs Elasticsearch to change the number of shards to 1 and not the default number (5). Then the number of replicas should be set to 0 for all the matching index name:

template.json

"template": "*",
"settings":
"index":
"number_of_shards": 1,
"number_of_replicas": 0

Next, use curl to generate and index templates featuring the settings that will be used on a the indices created:

$ curl -H'Content-Type: application/json' -XPUT http://localhost:9200/_template/defaults -d @template.json

Elastcisearch will give the output below:

{"acknowledged":true}

Kibana

To start and enable Kibana service, use the command:

$ sudo systemctl enable kibana
$ sudo systemctl start kibana

We’ll open the web application via the SSH tunnel. Run the following command in a separate terminal window to access Kibana via your local browser:

$ ssh -L 5601:localhost:5601 username@<Linode public IP> -N

The username can be replaced with your ideal Linux username, whereas the <Linode public IP> can be replaced with the public IP address of the Linode.

Filebeat

Here, we utilize the NGINX module so as to handle the most import configuration and give the relevant instructions required by the Stack to process the logs.

Create a /etc/filebeat/filebeat.yml file using your text editor and add the following details:

/etc/filebeat/filebeat.yml

filebeat.config.modules:
    path: ${path.config}/modules.d/*.yml

setup.kibana:
    host: "localhost:5601"

output.elasticsearch:
    hosts: ["localhost:9200"]

setup.dashboards.enabled: true

Remove .disabled from the file to enable your Filebeat modules, then enable NGINX module using:

$ sudo mv /etc/filebeat/modules.d/nginx.yml.disabled /etc/filebeat/modules.d/nginx.yml

Run the command below to enable and start Filebeat:

$ sudo systemctl enable filebeat
$ sudo systemctl start filebeat

To configure this service, first create  /etc/metricbeat/metricbeat.yml and incorporate the content below:

/etc/metricbeat/metricbeat.yml

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml

setup.kibana:
  host: "localhost:5601"

output.elasticsearch:
  hosts: ["localhost:9200"]

setup.dashboards.enabled: true

Next, rename the Nginx, Kibana, and Elasticsearch module configuration files:

$ sudo mv /etc/metricbeat/modules.d/elasticsearch.yml.disabled /etc/metricbeat/modules.d/elasticsearch.yml
$ sudo mv /etc/metricbeat/modules.d/kibana.yml.disabled /etc/metricbeat/modules.d/kibana.yml
$ sudo mv /etc/metricbeat/modules.d/nginx.yml.disabled /etc/metricbeat/modules.d/nginx.yml

Then start and enable your Metricbeat service.

That's it! You have successfully installed all the components you required to effortlessly monitor Nginx server. Part Two: How to Monitor Nginx using Elastic Stack on CentOS 7 will help you, use these parts to easily monitor your web server.

Special Note: some leading web hosts provide free 1-click installations of other tools for monitoring server performance. At Hostadvice, you can find extensive information the hosting plans and what they include, as well as user and expert reviews on the best web hosting services.

Check out the top 3 Dedicated server hosting services:

Was this article helpful?