Nginx Access Logs to Grafana
Config NGINX access logs
Add the following code under the HTTP {} section as follows:
vi /etc/nginx/nginx.conf
log_format grafana_logs
'{"@timestamp":"$time_iso8601",'
'"host":"$hostname",'
'"server_ip":"$server_addr",'
'"client_ip":"$remote_addr",'
'"xff":"$http_x_forwarded_for",'
'"domain":"$host",'
'"url":"$uri",'
'"referer":"$http_referer",'
'"args":"$args",'
'"upstreamtime":"$upstream_response_time",'
'"responsetime":"$request_time",'
'"request_method":"$request_method",'
'"status":"$status",'
'"size":"$body_bytes_sent",'
'"request_body":"$request_body",'
'"request_length":"$request_length",'
'"protocol":"$server_protocol",'
'"upstreamhost":"$upstream_addr",'
'"file_dir":"$request_filename",'
'"http_user_agent":"$http_user_agent"'
'}';
vi /etc/nginx/conf.d/web.conf
access_log /var/log/nginx/filebeat.vietcapitalbank.com.vn.access.log grafana_logs;
nginx -t
nginx -s reload
-----------------------
1.Redis
yum install redis
systemctl enable redis
systemctl start redis
openssl rand -base64 36
gQr8qHroGX7gqj6Gf+AsFSaHhbjN2AJCKak3k34HX4l0/kR5
vi /etc/redis/redis.conf
...
bind 0.0.0.0
port 6379
daemonize yes
supervised systemd
requirepass gQr8qHroGX7gqj6Gf+AsFSaHhbjN2AJCKak3k34HX4l0/kR5
...
2.Filebeat
Cài đặt Filebeat trên máy chủ Nginx
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.8.23-x86_64.rpm
systemctl enable filebeat
systemctl start filebeat
vi /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /var/log/nginx/filebeat.vietcapitalbank.com.vn.access.log
json.keys_under_root: true
json.overwrite_keys: true
json.add_error_key: true
filebeat.config.modules:
path: ${path.config}/modules.d/\*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 3
output.redis:
hosts: ["127.0.0.1:6379"]
password: "gQr8qHroGX7gqj6Gf+AsFSaHhbjN2AJCKak3k34HX4l0/kR5"
key: "grafana_logs"
db: 0
timeout: 5
systemctl start filebeat
3.Logstash
yum install logstash
vi /etc/logstash/conf.d/web.conf
input {
redis {
data_type =>"list"
key =>"grafana_logs"
host =>"127.0.0.1"
port => 6379
password => "gQr8qHroGX7gqj6Gf+AsFSaHhbjN2AJCKak3k34HX4l0/kR5"
db => 0
}
}
filter {
geoip {
target => "geoip"
source => "client_ip"
database => "/usr/share/elasticsearch/modules/ingest-geoip/GeoLite2-City.mmdb"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
remove_field => ["[geoip][latitude]", "[geoip][longitude]", "[geoip][country_code]", "[geoip][country_code2]", "[geoip][country_code3]", "[geoip][timezone]", "[geoip][continent_code]", "[geoip][region_code]"]
}
mutate {
convert => [ "size", "integer" ]
convert => [ "status", "integer" ]
convert => [ "responsetime", "float" ]
convert => [ "upstreamtime", "float" ]
convert => [ "[geoip][coordinates]", "float" ]
remove_field => [ "ecs","agent","host","cloud","@version","input","logs_type" ]
}
useragent {
source => "http_user_agent"
target => "ua"
remove_field => [ "[ua][minor]","[ua][major]","[ua][build]","[ua][patch]","[ua][os_minor]","[ua][os_major]" ]
}
}
output {
elasticsearch {
hosts => ["127.0.0.1"]
#user => "myusername"
#password => "mypassword"
index => "logstash-nginx-sysadmins-%{+YYYY.MM.dd}"
}
}
Download GeoLite2-City.mmdb from https://github.com/P3TERX/GeoLite.mmdb
systemctl restart logstash
4.Elasticsearch
yum install elasticsearch
systemctl enable elasticsearch
systemctl start elasticsearch
5.Grafana
yum install grafana
systemctl restart grafana-server
Access http://IP:3000/?orgId=1 to config datasource elasticsearch
Import dashboard from https://grafana.com/grafana/dashboards/16496
