Installing ELK stack
Followed https://techviewleo.com/install-elastic-stack-elk-on-debian/ ports:
- elasticsearch: 9200
- logstash: 5044
- kibana: 5601
- filebeat
change elasticsearch JVM heap size to use only 1G in /etc/elasticsearch/jvm.options
-Xms1g
-Xmx1g
Test ElsticSearch
curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic http://localhost:9200
Project: analysing server activity
- Use a VPN connection.
- Beats(FileBeat, HeartBeat) can output data to Elasticsearch directly or via Logstash
FileBeat: apache & ssh log files
- FileBeat: Beats are lightweight agents that are installed on client hosts to collect different types of data for forwarding into the stack.
- Once data is pushed to Elasticsearch, it can be analyzes, visualized, and monitored in Kibana
Sources:
- Apache logs: /var/log/apache2/access.log, /var/log/apache2/error.log
- ssh logs: /var/log/auth.log
Setup: On remote (IP x.x.x.2)
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["X.X.X.Y:9200"] # ElasticSearch host IP
# Protocol - either `http` (default) or `https`.
protocol: "http"
# Authentication credentials - either API key or username/password.
#api_key: "id:api_key"
username: "elastic"
password: "XXX"
- test elasticsearch access, from remote:
curl http://X.X.X.Y:9200
HeartBeat
Heartbeat is a lightweight shipper that monitors server uptime. It periodically checks the status of different services and tell us whether they're available. Similar to the ping command
I am running it from the ELK stack machine to check the status of my servers with ICMP monitors
ELK stack machine
On ELK stack machine: (IP x.x.x.1)
- have elasticsearch and kibana running
- test elasticsearch: curl http://localhost:9200