{{tag>Brouillon}}

= Notes Elasticsearch Logstash Kibana

Voir :
* https://www.scaleway.com/en/docs/setup-elastic-stack-on-scaleway/
* https://linuxfr.org/news/amazon-opensearch-fruit-d-une-rivalite-avec-elastic
* https://www.youtube.com/watch?v=IJ1IkALLChI
* https://krakensystems.co/blog/2018/logstash-nginx-logs-part-1

Voir aussi :
* Loki (remplace Elasticsearch Logstash)
** Voir [[https://linuxfr.org/news/loki-centralisation-de-logs-a-la-sauce-prometheus|Loki, centralisation de logs à la sauce Prometheus]]
* https://vector.dev
* Metricbeat
* [[https://gist.github.com/g3rhard/b755db2aae0ecf5ee40c3ebf50ab520f|Promtail & Loki]]

**OpenSearch** remplace ElasticSearch

Vérif syntax Grok : https://grokdebug.herokuapp.com/





== Notes perso

A lire
* https://www.bmc.com/blogs/elasticsearch-filebeat-nginx/
* https://pawelurbanek.com/elk-nginx-logs-setup

API
* https://discuss.elastic.co/t/importing-dashboard-via-curl-fails-with-500-error/230421/6
* https://discuss.elastic.co/t/kibana-dashboard-import-api-not-working-from-api-export/208961/6

RSYSLOG
* https://medium.com/bolt-labs/using-json-for-nginx-log-format-793743064fc4
* https://www.elastic.co/fr/blog/how-to-centralize-logs-with-rsyslog-logstash-and-elasticsearch-on-ubuntu-14-04
* https://devconnected.com/monitoring-linux-logs-with-kibana-and-rsyslog/

Fluentd
* https://medium.com/@behroozam/how-to-parse-nginx-access-log-with-fluentd-and-send-it-to-elasticsearch-f66cf95bef43

NGINX JSON
* https://gist.github.com/NiceGuyIT/58dd4d553fe3017cbfc3f98c2fbdbc93
* https://community.centminmod.com/threads/how-to-configure-nginx-for-json-based-access-logging.19641/
* https://programmersought.com/article/68596876612/

DOCKER
* https://blog.atolcd.com/la-stack-elastic-donnees-metiers/
* https://maddevs.io/blog/log-collecting-with-elk-and-rsyslog/



== Elasticsearch

=== Config

''/etc/elasticsearch/jvm.options.d/mem.options''
<code ->
-Xms512m
-Xmx512m
</code>

Sécurité :
Voir : https://www.elastic.co/guide/en/elasticsearch/reference/7.12/security-minimal-setup.html

''/etc/elasticsearch/elasticsearch.yml''
<code yaml>
xpack.security.enabled: true
</code>

Attention, cette commande ne peut s’exécuter qu'une seule fois !
<code bash>
/usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto
</code>


== Logstash

Voir aussi : 
* Filebeat
* Fluentd


=== Config

Conf Java Mem

''/etc/logstash/jvm.options''
<code ->
## JVM configuration

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

#-Xms1g
#-Xmx1g
-Xms512m
-Xmx512m
</code>


Exemple Nginx

Voir : https://www.elastic.co/guide/en/logstash/7.9/logstash-config-for-filebeat-modules.html#parsing-nginx


Note : préférer Filebeat

''/etc/logstash/conf.d/nginx-exemple.conf''
<code ruby>
input {
 file {
   path => ["/var/log/nginx/access.log", "/var/log/nginx/error.log"]
   type => "nginx"
 }
}
filter {
  if [fileset][module] == "nginx" {
    if [fileset][name] == "access" {
      grok {
        match => { "message" => ["%{IPORHOST:[nginx][access][remote_ip]} - %{DATA:[nginx][access][user_name]} \[%{HTTPDATE:[nginx][access][time]}\] \"%{WORD:[nginx][access][method]} %{DATA:[nginx][access][url]} HTTP/%{NUMBER:[nginx][access][http_version]}\" %{NUMBER:[nginx][access][response_code]} %{NUMBER:[nginx][access][body_sent][bytes]} \"%{DATA:[nginx][access][referrer]}\" \"%{DATA:[nginx][access][agent]}\""] }
        remove_field => "message"
      }
      mutate {
        add_field => { "read_timestamp" => "%{@timestamp}" }
      }
      date {
        match => [ "[nginx][access][time]", "dd/MMM/YYYY:H:m:s Z" ]
        remove_field => "[nginx][access][time]"
      }
      useragent {
        source => "[nginx][access][agent]"
        target => "[nginx][access][user_agent]"
        remove_field => "[nginx][access][agent]"
      }
      geoip {
        source => "[nginx][access][remote_ip]"
        #target => "[nginx][access][geoip]"
      }
    }
    else if [fileset][name] == "error" {
      grok {
        match => { "message" => ["%{DATA:[nginx][error][time]} \[%{DATA:[nginx][error][level]}\] %{NUMBER:[nginx][error][pid]}#%{NUMBER:[nginx][error][tid]}: (\*%{NUMBER:[nginx][error][connection_id]} )?%{GREEDYDATA:[nginx][error][message]}"] }
        remove_field => "message"
      }
      mutate {
        rename => { "@timestamp" => "read_timestamp" }
      }
      date {
        match => [ "[nginx][error][time]", "YYYY/MM/dd H:m:s" ]
        remove_field => "[nginx][error][time]"
      }
    }
  }
}
output {
  elasticsearch {
    hosts => localhost
    
    #user => elastic
    #password => PassWord
    
    #manage_template => false
    #index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    index => "logstash-plop-%{+YYYY.MM.dd}"
  }
#  stdout { codec => rubydebug }
}
</code>


=== Debug

<code bash>
su - logstash -s /bin/bash

# Validation de la conf / vérif de la syntax
/usr/share/logstash/bin/logstash --config.test_and_exit --path.settings /etc/logstash -f /etc/logstash/conf.d/plop.conf -f

# Debug
/usr/share/logstash/bin/logstash --debug --path.settings /etc/logstash -f /etc/logstash/conf.d/plop.conf -f
</code>


=== Autres

Notes en vrac
<code ->
file {
	path => "/var/log/apache2/apache.log"
	start_position => "beginning"
	type => "apache"
}

elasticksearch

-p 9200-e discovery.type=single-node
</code>


== Kibana

=== Nginx reverse proxy

''/etc/nginx/sites-available/kibana.acme.fr''
<code nginx>
server {
        server_name kibana.acme.fr;

        root /var/www/html;

        location / {
                proxy_pass http://127.0.0.1:5601;
                include /etc/nginx/proxy_params;
                client_max_body_size 10M;

                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "upgrade";
                proxy_cache_bypass $http_upgrade;
        }

    access_log /var/log/nginx/kibana.acme.fr.log;
    error_log /var/log/nginx/kibana.acme.fr.err;

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/kibana.acme.fr/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/kibana.acme.fr/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    if ($host = kibana.acme.fr) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


        listen 80;
        listen [::]:80;

        server_name kibana.acme.fr;
    return 404; # managed by Certbot
}
</code>


=== Sécurité

''/etc/kibana/kibana.yml''
<code yaml>
elasticsearch.username: "elastic"
</code>

<code bash>
/usr/share/kibana/bin/kibana-keystore create
/usr/share/kibana/bin/kibana-keystore add elasticsearch.password
</code>


== Filebeat

Voir aussi fluentd

Dans certain cas, remplace Logstash

''filebeat.yml''
<code yaml>
output.elasticsearch:
  hosts: ["http://localhost:9200"]
  username: "elastic" 
  password: "P@ssw0rd" 

setup.kibana:
  host: "http://localhost:5601"
</code>

<code bash>
filebeat modules enable system nginx
filebeat setup
filebeat -e
</code>

Il suffit de chercher des Dashboard commençant par "[Filebeat System]" et [Filebeat Nginx]" pour avoir déjà une conf prête à l'emploi


=== Brouillons

<code bash>
filebeat setup -e \
  -E output.logstash.enabled=false \
  -E output.elasticsearch.hosts=['localhost:9200'] \
  -E output.elasticsearch.username=filebeat_internal \
  -E output.elasticsearch.password=YOUR_PASSWORD \
  -E setup.kibana.host=localhost:5601


filebeat setup -e \
        -E 'setup.template.overwrite=true' \
        -E 'setup.kibana.host="localhost:5601"' \
        -E 'output.logstash.enabled=false' \
        -E 'output.elasticsearch.hosts=["localhost:9200"]'
        
filebeat keystore create
#filebeat keystore add ES_PWD
filebeat keystore add elastic
filebeat keystore list
</code>


FIXME