ELK: Difference between revisions
From artserver wiki
Line 21: | Line 21: | ||
==FileBeat: apache & ssh log files== | ==[https://www.elastic.co/beats/filebeat FileBeat]: apache & ssh log files== | ||
* FileBeat: Beats are lightweight agents that are installed on client hosts to collect different types of data for forwarding into the stack. | * [https://www.elastic.co/beats/filebeat FileBeat]: Beats are lightweight agents that are installed on client hosts to collect different types of data for forwarding into the stack. | ||
* Once data is pushed to Elasticsearch, it can be analyzes, visualized, and monitored in Kibana | * Once data is pushed to Elasticsearch, it can be analyzes, visualized, and monitored in Kibana | ||
Line 50: | Line 50: | ||
''' | ''' | ||
==HeartBeat== | ==[https://www.elastic.co/beats/heartbeat HeartBeat]== | ||
Heartbeat is a lightweight shipper that monitors server uptime. It podically checks the status of different services and tell us whether they're available. Similar to the ping command | [https://www.elastic.co/beats/heartbeat Heartbeat] is a lightweight shipper that monitors server uptime. It podically checks the status of different services and tell us whether they're available. Similar to the ping command | ||
=ELK stack machine== | =ELK stack machine== |
Revision as of 12:14, 27 September 2022
Installing ELK stack
Followed https://techviewleo.com/install-elastic-stack-elk-on-debian/ ports:
- elasticsearch: 9200
- logstash: 5044
- kibana: 5601
- filebeat
change elasticsearch JVM heap size to use only 1G in /etc/elasticsearch/jvm.options
-Xms1g
-Xmx1g
Test ElsticSearch
curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic http://localhost:9200
Project: analysing server activity
- Use a VPN connection.
- Beats(FileBeat, HeartBeat) can output data to Elasticsearch directly or via Logstash
FileBeat: apache & ssh log files
- FileBeat: Beats are lightweight agents that are installed on client hosts to collect different types of data for forwarding into the stack.
- Once data is pushed to Elasticsearch, it can be analyzes, visualized, and monitored in Kibana
Sources:
- Apache logs: /var/log/apache2/access.log, /var/log/apache2/error.log
- ssh logs: /var/log/auth.log
Setup: On remote (IP x.x.x.2)
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["X.X.X.Y:9200"] # ElasticSearch host IP
# Protocol - either `http` (default) or `https`.
protocol: "http"
# Authentication credentials - either API key or username/password.
#api_key: "id:api_key"
username: "elastic"
password: "XXX"
- test elasticsearch access, from remote:
curl http://X.X.X.Y:9200
HeartBeat
Heartbeat is a lightweight shipper that monitors server uptime. It podically checks the status of different services and tell us whether they're available. Similar to the ping command
ELK stack machine=
On ELK stack machine: (IP x.x.x.1)
- have elasticsearch and kibana running
- test elasticsearch: curl http://localhost:9200