random

- collection of un-sorted bollocks
git clone git://git.acid.vegas/random.git
elkstack.md (3954B)
      1 # ELK Stack
      2 > Elasticsearch, Logstash, & Kibana
      3 
      4 This is just a little write-up on my research in deploying the ELK stack.
      5 
      6 ## Prerequisites
      7 ```shell
      8 sudo apt-get install -y gpg apt-transport-https`
      9 wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
     10 echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
     11 sudo apt-get update && sudo apt-get install elasticsearch kibana logstash
     12 sudo certbot certonly --standalone --preferred-challenges http -d elastic.domain.org
     13 ```
     14 
     15 ## Setup Elasticsearch
     16 * Copy your certificates to `/etc/elasticsearch/certs`:
     17 ```shell
     18 mkdir -p /etc/elasticsearch/certs/
     19 sudo cp /etc/letsencrypt/live/elastic.domain.org/fullchain.pem /etc/elasticsearch/certs/fullchain.pem
     20 sudo cp /etc/letsencrypt/live/elastic.domain.org/privkey.pem   /etc/elasticsearch/certs/privkey.pem
     21 sudo chmod -R 777 /etc/elasticsearch/certs/
     22 ```
     23 
     24 * Edit your `/etc/elasticsearch/elasticsearch.yml` and change the follow options:
     25 ```yaml
     26 cluster.name: BeeHive
     27 node.name: gibson
     28 network.host: 0.0.0.0    
     29 bootstrap.memory_lock: true
     30 xpack.security.audit.enabled: true
     31 xpack.security.http.ssl:
     32   enabled: true
     33   key: /etc/elasticsearch/ssl/privkey.pem
     34   certificate: /etc/elasticsearch/ssl/fullchain.pem
     35 ```
     36 
     37 * System changes:
     38 ```shell
     39 sudo su  
     40 	ulimit -n 65535
     41 	ulimit -u 4096
     42 
     43 echo "elasticsearch  -  nofile  65535" > /etc/security/limits.conf
     44 mkdir -p /etc/systemd/system/elasticsearch.service.d/
     45 echo "[Service]\nLimitMEMLOCK=infinity" > /etc/systemd/system/elasticsearch.service.d/override.conf
     46 sudo swapoff -a
     47 sudo sysctl -w vm.swappiness=1         # Add these
     48 sudo sysctl -w vm.max_map_count=262144 # to /etc/systctl.conf
     49 sudo sysctl -w net.ipv4.tcp_retries2=5 # 
     50 ```
     51 
     52 * Set the password for Kibana:
     53 `./usr/share/elasticsearch/bin/elasticsearch-reset-password -u kibana_system`
     54 `./usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token --scope kibana # Save this for when we access Kibana the first time`
     55 `./usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node # enrollment token for a new node`
     56 
     57 ## Setup Kibana
     58 * Copy your certificates to `/etc/kibana/certs`:
     59 ```shell
     60 mkdir -p /etc/kibana/certs/
     61 sudo cp /etc/letsencrypt/live/elastic.domain.org/fullchain.pem /etc/kibana/certs/fullchain.pem
     62 sudo cp /etc/letsencrypt/live/elastic.domain.org/privkey.pem   /etc/kibana/certs/privkey.pem
     63 ```
     64 
     65 * Edit your `/etc/kibana/kibana.yml` and change the follow options:
     66 ```yaml
     67 server.host: "0.0.0.0"
     68 server.publicBaseUrl: "https://elastic.domain.org"
     69 server.ssl.enabled: true 
     70 server.ssl.certificate: /etc/kibana/certs/fullchain.pem
     71 server.ssl.key: /etc/kibana/certs/privkey.pem
     72 elasticsearch.hosts: ["https://elastic.domain.org:9200"]
     73 elasticsearch.username: "kibana_system"
     74 elasticsearch.password: "changeme" # Use the password from the reset command we did earlier
     75 ```
     76 
     77 ## Setup Logstash
     78 * Copy your certificates to `/etc/logstash/certs`:
     79 ```shell
     80 mkdir -p /etc/logstash/certs/
     81 sudo cp /etc/letsencrypt/live/elastic.domain.org/fullchain.pem /etc/logstash/certs/cacert.pem
     82 ```
     83 
     84 * Edit your `/etc/logstash/logstash.yml` and change the follow options:
     85 ```yaml
     86 input {
     87   beats {
     88     port => 5044
     89   }
     90 }
     91 output {
     92   elasticsearch {
     93     hosts => ["https://elastic.domain.org:9200"]
     94     index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
     95     user => "elastic"
     96     password => "changeme"
     97     cacert => "/etc/logstash/cacert.pem"
     98   }
     99 }
    100 ```
    101 
    102 * `logstash-plugin install logstash-input-irc`
    103 
    104 ## Start the ELK stack:
    105 ```shell
    106 sudo systemctl daemon-reload
    107 sudo systemctl enable elasticsearch.service && sudo systemctl start elasticsearch.service
    108 sudo systemctl enable kibana.service        && sudo systemctl start kibana.service
    109 sudo systemctl enable logstash.service      && sudo systemctl start logstash.service
    110 ```