跳到主要內容

FluentD 實作 Error Log

FluentD 實作 Error Log

本篇將介紹使用 DotNet 專案 log4net 套件,紀錄的 log
針對 Error Level 的訊息透過FluentD 提取出來
在紀錄中 增加 trace ID
設入 MongoDB , 及加入 Slack 通知

延伸閱讀

FluentD 參數說明
FluentD 實作 Nginx Access Log
FluentD 實作 Nginx Access Log 補充


log4net


<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <log4net>
    <appender name="All" type="log4net.Appender.RollingFileAppender">
      <file value="/var/log/my.log" />
      <appendToFile value="true" />
      <rollingStyle value="Size" />
      <datePattern value="yyyy-MM-dd" />
      <maximumFileSize value="5MB" />
      <maxSizeRollBackups value="10" />
      <staticLogFileName value="true" />
      <PreserveLogFileNameExtension value="true" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="[%date] [%thread] %level %logger - %message%newline" />
      </layout>
    </appender>
    <root>
      <appender-ref ref="All" />
    </root>
  </log4net>
</configuration>

Dockerfile


FROM fluent/fluentd:v1.8.1-1.0

# Use root account to use apk
USER root

# below RUN includes plugin as examples elasticsearch is not required
# you may customize including plugins as you wish
RUN apk add --no-cache --update --virtual .build-deps \
        sudo build-base ruby-dev \
     && sudo gem install fluent-plugin-mongo \
     && sudo gem install fluent-plugin-add-uuid \
     && sudo gem install fluent-plugin-slack \
     && sudo gem sources --clear-all \
     && apk del .build-deps \
     && rm -rf /home/fluent/.gem/ruby/2.5.0/cache/*.gem

VOLUME ["/fluentd/etc","/fluentd/log","/var/log"]

fluent.conf


<source>
    @type tail
    path /var/log/my.log
    pos_file /var/log/td-agent/my.log4net.log.pos
    tag log4net.web
    multiline_flush_interval 10s
    #format /^\[(?<logdt>[^\]]*)\] \[(?<thread>[^ ]*)\] (?<level>[^ ]*) (?<logger>[^ ]*) - (?<message>[^*]*)?$/
    <parse>
      @type multiline
      format_firstline  /\[\d{4}-\d{1,2}-\d{1,2}/
      format1 /^\[(?<logdt>[^\]]*)\] \[(?<thread>[^ ]*)\] (?<level>[^ ]*) (?<logger>[^ ]*) - (?<message>.*)/
    </parse>
  </source>
  <filter log4net.web>
    @type grep
    <regexp>
      key level
      pattern /ERROR/
    </regexp>
  </filter>
  <filter log4net.web>
    @type adduuid
    key _uuid
  </filter>
  <match log4net.web>
    @type copy
    <store>
      @type mongo
      略...
    </store>
    <store>
      @type slack
      token xoxb-${sleck bot token}
      username fluentd-dev
      webhook_url https://hooks.slack.com/services/${hook path}
      title %s
      title_keys tag
      message %s %s
      message_keys logdt,_uuid
    </store>
  </match>
</worker>

留言

這個網誌中的熱門文章

FluentD 存取 File Log

其他文章參考 FluentD 存取 Nginx Access Log (1/2) FluentD 存取 Nginx Access Log (2/2) 以上是 Nginx + FluentD + (ES|Mongo) Demo 針對access.log 做解析 現在以Log4X 產生的日誌檔作為範例說明 會遇到的問題有 日誌內容會有多行的情況 # multiline 希望每條日誌內容加入 UUID 以便追蹤 # https://github.com/chaeyk/fluent-plugin-add-uuid 使用 Slack 作為通知的通道 # https://github.com/sowawa/fluent-plugin-slack Log4X Layout Format Example <?xml version="1.0" encoding="utf-8" ?> <configuration> <log4net> <appender name="All" type="log4net.Appender.RollingFileAppender"> <file value="/var/log/web.log" /> <appendToFile value="true" /> <rollingStyle value="Size" /> <datePattern value="yyyy-MM-dd" /> <maximumFileSize value="5MB" /> <maxSizeRollBackups value="10" /> <staticLogFileName value="true" /> <PreserveLogFileNameExtension value=...

申請免費 SSL,自動續訂

參考 acme.sh 搭配 GoDaddy 自動續期 Let's Encrypt 免費萬用憑證 使用 acme.sh + Cloudflare 申請免費 Wildcard SSL (Let’s Encrypt) 節略如下 安裝 acme.sh # 安裝 acme.sh ,安裝後重新登入 curl https://get.acme.sh | sh # 自動更新 acme.sh --upgrade --auto-upgrade acme.sh 設定存取 Goddy vi ~/.acme.sh/account.conf # Goddy API GD_Key="" GD_Secret="" acme.sh 設定存取 Cloudflare # Cloudflare API Keys # Global API Key [View] export CF_Key="" export CF_Email="" 申請網域(Domain)的萬用憑證,成功後會顯示憑證存放的路徑 $> acme.sh --issue --dns dns_gd -d ${domain} -d *.${domain} 安裝憑證 # 建立 /etc/nginx/ssl/${domain} 路徑 $> acme.sh --install-cert -d ${domain} --key-file /etc/nginx/ssl/${domain}/key.pem --fullchain-file /etc/nginx/ssl/${domain}/cert.pem --reloadcmd "sudo nginx -s reload"

DotNet Core 專案部署腳本

DotNet core SDK 首先在 Server 上準備編譯環境 Dockerfile #2.2 3.0 3.1 FROM mcr.microsoft.com/dotnet/core/sdk:3.1 RUN mkdir /web WORKDIR /web build docker image shell script docker build -t dotnetcoresdk:3.1 . start docker container shell script docker run -it -d \ --name dotnet-core-sdk-3.1 \ -v /opt/web:/web \ dotnetcoresdk:3.1 Jenkins Execute shell script on remote hosting using ssh #切換至專案目錄 cd /opt/web/project/path #取得最新版本 git pull #切換至專案目錄 && 刷新 Dotnet Library docker exec -i dotnet-core-sdk-3.1 bash -c "cd project/path && dotnet restore" #切換至專案目錄 && 刪除上一次編譯的檔案 && 編譯 docker exec -i dotnet-core-sdk-3.1 bash -c "cd project/path && rm -rf bin/Release && dotnet publish -c Release" #docker-compose.yml 參 DotNet core Runtime Section #!--rmi all 將原本執行的容器關閉並移除Image docker-compose down --rmi all #將新版程式包入 Image 並開始容器 docker-compose up -d DotNet core Runtime 專案中包含 Dockerfile & docker-compose.yml d...