filebeat.yml(中文配置詳解)

來源:本站原創 elk 超過871 views圍觀 0條評論
################### Filebeat Configuration Example #########################

############################# Filebeat ######################################
filebeat:
  # List of prospectors to fetch data.
  prospectors:
    # Each - is a prospector. Below are the prospector specific configurations
    -
      # Paths that should be crawled and fetched. Glob based paths.
      # To fetch all ".log" files from a specific level of subdirectories
      # /var/log/*/*.log can be used.
      # For each file found under this path, a harvester is started.
      # Make sure not file is defined twice as this can lead to unexpected behaviour.
      # 指定要監控的日志,可以指定具體得文件或者目錄
      paths:
        - /var/log/*.log  (這是默認的)(自行可以修改)(比如我放在/home/hadoop/app.log里)
        #- c:\programdata\elasticsearch\logs\*

      # Configure the file encoding for reading files with international characters
      # following the W3C recommendation for HTML5 (http://www.w3.org/TR/encoding).
      # Some sample encodings:
      #   plain, utf-8, utf-16be-bom, utf-16be, utf-16le, big5, gb18030, gbk,
      #    hz-gb-2312, euc-kr, euc-jp, iso-2022-jp, shift-jis, ...
      # 指定被監控的文件的編碼類型,使用plain和utf-8都是可以處理中文日志的
      #encoding: plain

      # Type of the files. Based on this the way the file is read is decided.
      # The different types cannot be mixed in one prospector
      #
      # Possible options are:
      # * log: Reads every line of the log file (default)
      # * stdin: Reads the standard in
      # 指定文件的輸入類型log(默認)或者stdin
      input_type: log

      # Exclude lines. A list of regular expressions to match. It drops the lines that are
      # matching any regular expression from the list. The include_lines is called before
      # 在輸入中排除符合正則表達式列表的那些行。
      # exclude_lines. By default, no lines are dropped.
      # exclude_lines: ["^DBG"]

      # Include lines. A list of regular expressions to match. It exports the lines that are
      # matching any regular expression from the list. The include_lines is called before
      # exclude_lines. By default, all the lines are exported.
      # 包含輸入中符合正則表達式列表的那些行(默認包含所有行),include_lines執行完畢之后會執行exclude_lines
      # include_lines: ["^ERR", "^WARN"]

      # Exclude files. A list of regular expressions to match. Filebeat drops the files that
      # are matching any regular expression from the list. By default, no files are dropped.
      # 忽略掉符合正則表達式列表的文件
      # exclude_files: [".gz$"]

      # Optional additional fields. These field can be freely picked
      # to add additional information to the crawled log files for filtering
      # 向輸出的每一條日志添加額外的信息,比如“level:debug”,方便后續對日志進行分組統計。
      # 默認情況下,會在輸出信息的fields子目錄下以指定的新增fields建立子目錄,例如fields.level
      # 這個得意思就是會在es中多添加一個字段,格式為 "filelds":{"level":"debug"}
      #fields:
      #  level: debug
      #  review: 1

      # Set to true to store the additional fields as top level fields instead
      # of under the "fields" sub-dictionary. In case of name conflicts with the
      # fields added by Filebeat itself, the custom fields overwrite the default
      # fields.
      # 如果該選項設置為true,則新增fields成為頂級目錄,而不是將其放在fields目錄下。
      # 自定義的field會覆蓋filebeat默認的field
      # 如果設置為true,則在es中新增的字段格式為:"level":"debug"
      #fields_under_root: false

      # Ignore files which were modified more then the defined timespan in the past.
      # In case all files on your system must be read you can set this value very large.
      # Time strings like 2h (2 hours), 5m (5 minutes) can be used.
      # 可以指定Filebeat忽略指定時間段以外修改的日志內容,比如2h(兩個小時)或者5m(5分鐘)。
      #ignore_older: 0

      # Close older closes the file handler for which were not modified
      # for longer then close_older
      # Time strings like 2h (2 hours), 5m (5 minutes) can be used.
      # 如果一個文件在某個時間段內沒有發生過更新,則關閉監控的文件handle。默認1h
      #close_older: 1h

      # Type to be published in the 'type' field. For Elasticsearch output,
      # the type defines the document type these entries should be stored
      # in. Default: log
      # 設定Elasticsearch輸出時的document的type字段 可以用來給日志進行分類。Default: log
      #document_type: log

      # Scan frequency in seconds.
      # How often these files should be checked for changes. In case it is set
      # to 0s, it is done as often as possible. Default: 10s
      # Filebeat以多快的頻率去prospector指定的目錄下面檢測文件更新(比如是否有新增文件)
      # 如果設置為0s,則Filebeat會盡可能快地感知更新(占用的CPU會變高)。默認是10s
      #scan_frequency: 10s

      # Defines the buffer size every harvester uses when fetching the file
      # 每個harvester監控文件時,使用的buffer的大小
      #harvester_buffer_size: 16384

      # Maximum number of bytes a single log event can have
      # All bytes after max_bytes are discarded and not sent. The default is 10MB.
      # This is especially useful for multiline log messages which can get large.
      # 日志文件中增加一行算一個日志事件,max_bytes限制在一次日志事件中最多上傳的字節數,多出的字節會被丟棄
      #max_bytes: 10485760

      # Mutiline can be used for log messages spanning multiple lines. This is common
      # for Java Stack Traces or C-Line Continuation
      # 適用于日志中每一條日志占據多行的情況,比如各種語言的報錯信息調用棧
      #multiline:

        # The regexp Pattern that has to be matched. The example pattern matches all lines starting with [
        # 多行日志開始的那一行匹配的pattern
        #pattern: ^\[

        # Defines if the pattern set under pattern should be negated or not. Default is false.
        # 是否需要對pattern條件轉置使用,不翻轉設為true,反轉設置為false。  【建議設置為true】
        #negate: false

        # Match can be set to "after" or "before". It is used to define if lines should be append to a pattern
        # that was (not) matched before or after or as long as a pattern is not matched based on negate.
        # Note: After is the equivalent to previous and before is the equivalent to to next in Logstash
        # 匹配pattern后,與前面(before)還是后面(after)的內容合并為一條日志
        #match: after

        # The maximum number of lines that are combined to one event.
        # In case there are more the max_lines the additional lines are discarded.
        # Default is 500
        # 合并的最多行數(包含匹配pattern的那一行)
        #max_lines: 500

        # After the defined timeout, an multiline event is sent even if no new pattern was found to start a new event
        # Default is 5s.
        # 到了timeout之后,即使沒有匹配一個新的pattern(發生一個新的事件),也把已經匹配的日志事件發送出去
        #timeout: 5s

      # Setting tail_files to true means filebeat starts readding new files at the end
      # instead of the beginning. If this is used in combination with log rotation
      # this can mean that the first entries of a new file are skipped.
      # 如果設置為true,Filebeat從文件尾開始監控文件新增內容,把新增的每一行文件作為一個事件依次發送,
      # 而不是從文件開始處重新發送所有內容
      #tail_files: false

      # Backoff values define how agressively filebeat crawls new files for updates
      # The default values can be used in most cases. Backoff defines how long it is waited
      # to check a file again after EOF is reached. Default is 1s which means the file
      # is checked every second if new lines were added. This leads to a near real time crawling.
      # Every time a new line appears, backoff is reset to the initial value.
      # Filebeat檢測到某個文件到了EOF(文件結尾)之后,每次等待多久再去檢測文件是否有更新,默認為1s
      #backoff: 1s

      # Max backoff defines what the maximum backoff time is. After having backed off multiple times
      # from checking the files, the waiting time will never exceed max_backoff idenependent of the
      # backoff factor. Having it set to 10s means in the worst case a new line can be added to a log
      # file after having backed off multiple times, it takes a maximum of 10s to read the new line
      # Filebeat檢測到某個文件到了EOF之后,等待檢測文件更新的最大時間,默認是10秒
      #max_backoff: 10s

      # The backoff factor defines how fast the algorithm backs off. The bigger the backoff factor,
      # the faster the max_backoff value is reached. If this value is set to 1, no backoff will happen.
      # The backoff value will be multiplied each time with the backoff_factor until max_backoff is reached
      # 定義到達max_backoff的速度,默認因子是2,到達max_backoff后,變成每次等待max_backoff那么長的時間才backoff一次,
      # 直到文件有更新才會重置為backoff
      # 根據現在的默認配置是這樣的,每隔1s檢測一下文件變化,如果連續檢測兩次之后文件還沒有變化,下一次檢測間隔時間變為10s
      #backoff_factor: 2

      # This option closes a file, as soon as the file name changes.
      # This config option is recommended on windows only. Filebeat keeps the files it's reading open. This can cause
      # issues when the file is removed, as the file will not be fully removed until also Filebeat closes
      # the reading. Filebeat closes the file handler after ignore_older. During this time no new file with the
      # same name can be created. Turning this feature on the other hand can lead to loss of data
      # on rotate files. It can happen that after file rotation the beginning of the new
      # file is skipped, as the reading starts at the end. We recommend to leave this option on false
      # but lower the ignore_older value to release files faster.
      # 這個選項關閉一個文件,當文件名稱的變化。#該配置選項建議只在windows
      #force_close_files: false

    # Additional prospector
    #-
      # Configuration to use stdin input
      #input_type: stdin

  # General filebeat configuration options
  #
  # Event count spool threshold - forces network flush if exceeded
  # spooler的大小,spooler中的事件數量超過這個閾值的時候會清空發送出去(不論是否到達超時時間)
  #spool_size: 2048

  # Enable async publisher pipeline in filebeat (Experimental!)
  # 是否采用異步發送模式(實驗功能)
  #publish_async: false

  # Defines how often the spooler is flushed. After idle_timeout the spooler is
  # Flush even though spool_size is not reached.
  # spooler的超時時間,如果到了超時時間,spooler也會清空發送出去(不論是否到達容量的閾值)
  #idle_timeout: 5s

  # Name of the registry file. Per default it is put in the current working
  # directory. In case the working directory is changed after when running
  # filebeat again, indexing starts from the beginning again.
  # 記錄filebeat處理日志文件的位置的文件,默認是在啟動的根目錄下
  #registry_file: .filebeat

  # Full Path to directory with additional prospector configuration files. Each file must end with .yml
  # These config files must have the full filebeat config part inside, but only
  # the prospector part is processed. All global options like spool_size are ignored.
  # The config_dir MUST point to a different directory then where the main filebeat config file is in.
  # 如果要在本配置文件中引入其他位置的配置文件,可以寫在這里(需要寫完整路徑),但是只處理prospector的部分
  #config_dir:

###############################################################################
############################# Libbeat Config ##################################
# Base config file used by all other beats for using libbeat features

############################# Output ##########################################

# Configure what outputs to use when sending the data collected by the beat.
# Multiple outputs may be used.
output:

  ### Elasticsearch as output
  elasticsearch:            (這是默認的,filebeat收集后放到es里)(自行可以修改,比如我有時候想filebeat收集后,然后到redis,再到es,就可以注銷這行)
    # Array of hosts to connect to.
    # Scheme and port can be left out and will be set to the default (http and 9200)
    # In case you specify and additional path, the scheme is required: http://localhost:9200/path
    # IPv6 addresses should always be defined as: https://[2001:db8::1]:9200
    hosts: ["localhost:9200"]        (這是默認的,filebeat收集后放到es里)(自行可以修改,比如我有時候想filebeat收集后,然后到redis,再到es,就可以注銷這行)
文章出自:CCIE那點事 http://www.qdxgqk.live/ 版權所有。本站文章除注明出處外,皆為作者原創文章,可自由引用,但請注明來源。 禁止全文轉載。
本文鏈接:http://www.qdxgqk.live/?p=3649轉載請注明轉自CCIE那點事
如果喜歡:點此訂閱本站
  • 相關文章
  • 為您推薦
  • 各種觀點
?
暫時還木有人評論,坐等沙發!
發表評論

您必須 [ 登錄 ] 才能發表留言!

?
?
萌宠夺宝游戏