Installing ELK stack

Followed https://techviewleo.com/install-elastic-stack-elk-on-debian/ ports:

elasticsearch: 9200
logstash: 5044
kibana: 5601
filebeat

change elasticsearch JVM heap size to use only 1G in /etc/elasticsearch/jvm.options -Xms1g -Xmx1g

Test ElsticSearch

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic http://localhost:9200

Project: analysing server activity

Use a VPN connection.
Beats(FileBeat, HeartBeat) can output data to Elasticsearch directly or via Logstash

FileBeat: apache & ssh log files

FileBeat: Beats are lightweight agents that are installed on client hosts to collect different types of data for forwarding into the stack.
Once data is pushed to Elasticsearch, it can be analyzes, visualized, and monitored in Kibana

Sources:

Apache logs: /var/log/apache2/access.log, /var/log/apache2/error.log
ssh logs: /var/log/auth.log

Setup: On remote (IP x.x.x.2)

  
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["X.X.X.Y:9200"]  # ElasticSearch host IP 

  # Protocol - either `http` (default) or `https`.
  protocol: "http"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "XXX"

test elasticsearch access, from remote: curl http://X.X.X.Y:9200

HeartBeat

Heartbeat is a lightweight shipper that monitors server uptime. It periodically checks the status of different services and tell us whether they're available. Similar to the ping command

I am running it from the ELK stack machine to check the status of my servers with ICMP monitors

test configuration: heartbeat -e

ELK stack machine

On ELK stack machine: (IP x.x.x.1)

have elasticsearch and kibana running
test elasticsearch: curl http://localhost:9200