環境
vagrant@ubuntu-bionic:~/fluentd$ uname -a Linux ubuntu-bionic 4.15.0-88-generic #88-Ubuntu SMP Tue Feb 11 20:11:34 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
手順
環境準備
vagrant@ubuntu-bionic:~/fluentd$ sudo apt-get update # git install vagrant@ubuntu-bionic:~/fluentd$ sudo apt-get install git Reading package lists... Done Building dependency tree Reading state information... Done git is already the newest version (1:2.17.1-1ubuntu0.5). git set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. # ruby install vagrant@ubuntu-bionic:~/fluentd$ sudo apt-get install ruby 〜 # check vagrant@ubuntu-bionic:~/fluentd$ ruby -v ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu] # git clone vagrant@ubuntu-bionic:~/fluentd$ git clone https://github.com/fluent/fluentd.git vagrant@ubuntu-bionic:~/fluentd$ cd fluentd
cloneに成功したらbundlerをインストールします。
vagrant@ubuntu-bionic:~/fluentd$ sudo gem install bundler Fetching: bundler-2.2.14.gem (100%) Successfully installed bundler-2.2.14 Parsing documentation for bundler-2.2.14 Installing ri documentation for bundler-2.2.14 Done installing documentation for bundler after 6 seconds 1 gem installed vagrant@ubuntu-bionic:~/fluentd$ sudo bundle install --path vendor/bundle 〜〜
いくつかエラーが発生
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/nio4r-2.5.7/ext/nio4r
/usr/bin/ruby2.5 -r ./siteconf20210317-5059-1lne13a.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h
extconf failed, exit code 1
Gem files will remain installed in /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/nio4r-2.5.7 for inspection.
Results logged to /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nio4r-2.5.7/gem_make.out
An error occurred while installing nio4r (2.5.7), and Bundler cannot continue.
Make sure that `gem install nio4r -v '2.5.7' --source 'https://rubygems.org/'` succeeds before bundling.
In Gemfile:
async-http was resolved to 0.54.1, which depends on
async-io was resolved to 1.30.2, which depends on
async was resolved to 1.28.9, which depends on
nio4r
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/cool.io-1.7.1/ext/cool.io
/usr/bin/ruby2.5 -r ./siteconf20210317-5059-1ic7b86.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h
extconf failed, exit code 1
Gem files will remain installed in /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/cool.io-1.7.1 for inspection.
Results logged to /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0/cool.io-1.7.1/gem_make.out
An error occurred while installing cool.io (1.7.1), and Bundler cannot continue.
Make sure that `gem install cool.io -v '1.7.1' --source 'https://rubygems.org/'` succeeds before bundling.
In Gemfile:
fluentd was resolved to 1.12.1, which depends on
cool.io
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/http_parser.rb-0.6.0/ext/ruby_http_parser
/usr/bin/ruby2.5 -r ./siteconf20210317-5059-1s7iw08.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h
extconf failed, exit code 1
Gem files will remain installed in /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/http_parser.rb-0.6.0 for inspection.
Results logged to /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0/http_parser.rb-0.6.0/gem_make.out
An error occurred while installing http_parser.rb (0.6.0), and Bundler cannot continue.
Make sure that `gem install http_parser.rb -v '0.6.0' --source 'https://rubygems.org/'` succeeds before bundling.
In Gemfile:
fluentd was resolved to 1.12.1, which depends on
http_parser.rb
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/msgpack-1.4.2/ext/msgpack
/usr/bin/ruby2.5 -r ./siteconf20210317-5059-tcrl33.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h
extconf failed, exit code 1
Gem files will remain installed in /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/msgpack-1.4.2 for inspection.
Results logged to /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0/msgpack-1.4.2/gem_make.out
An error occurred while installing msgpack (1.4.2), and Bundler cannot continue.
Make sure that `gem install msgpack -v '1.4.2' --source 'https://rubygems.org/'` succeeds before bundling.
In Gemfile:
fluentd was resolved to 1.12.1, which depends on
msgpack
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/strptime-0.2.5/ext/strptime
/usr/bin/ruby2.5 -r ./siteconf20210317-5059-10t806e.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h
extconf failed, exit code 1
Gem files will remain installed in /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/strptime-0.2.5 for inspection.
Results logged to /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0/strptime-0.2.5/gem_make.out
An error occurred while installing strptime (0.2.5), and Bundler cannot continue.
Make sure that `gem install strptime -v '0.2.5' --source 'https://rubygems.org/'` succeeds before bundling.
In Gemfile:
fluentd was resolved to 1.12.1, which depends on
strptime
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/oj-3.11.3/ext/oj
/usr/bin/ruby2.5 -r ./siteconf20210317-5059-htog6q.rb extconf.rb
mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h
extconf failed, exit code 1
Gem files will remain installed in /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/oj-3.11.3 for inspection.
Results logged to /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0/oj-3.11.3/gem_make.out
An error occurred while installing oj (3.11.3), and Bundler cannot continue.
Make sure that `gem install oj -v '3.11.3' --source 'https://rubygems.org/'` succeeds before bundling.
In Gemfile:
oj
native extensions:とはCやC++などで書かれるライブラリを必要とするRubyのコードあたりを指します (http://woshidan.hatenadiary.jp/entry/2017/01/21/150948) これらのgemはCやC++のライブラリを必要とする為、事前にインストールしましょう。
ということでruby-devをインストールし必要なライブラリを用意します。
sudo apt-get install ruby-dev
すると少しエラーが変わりました。
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/nio4r-2.5.7/ext/nio4r
/usr/bin/ruby2.5 -r ./siteconf20210317-6623-18n6i2v.rb extconf.rb
checking for unistd.h... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/bin/$(RUBY_BASE_NAME)2.5
/usr/lib/ruby/2.5.0/mkmf.rb:456:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.
from /usr/lib/ruby/2.5.0/mkmf.rb:590:in `try_cpp'
from /usr/lib/ruby/2.5.0/mkmf.rb:1097:in `block in have_header'
from /usr/lib/ruby/2.5.0/mkmf.rb:947:in `block in checking_for'
from /usr/lib/ruby/2.5.0/mkmf.rb:350:in `block (2 levels) in postpone'
from /usr/lib/ruby/2.5.0/mkmf.rb:320:in `open'
from /usr/lib/ruby/2.5.0/mkmf.rb:350:in `block in postpone'
from /usr/lib/ruby/2.5.0/mkmf.rb:320:in `open'
from /usr/lib/ruby/2.5.0/mkmf.rb:346:in `postpone'
from /usr/lib/ruby/2.5.0/mkmf.rb:946:in `checking_for'
from /usr/lib/ruby/2.5.0/mkmf.rb:1096:in `have_header'
from extconf.rb:15:in `<main>'
To see why this extension failed to compile, please check the mkmf.log which can be found here:
/home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nio4r-2.5.7/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/nio4r-2.5.7 for inspection.
Results logged to /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0/nio4r-2.5.7/gem_make.out
ログを見れみるとgccを実行しようとしているが入っていないので入れる。(gccはコンパイラ)
どうやらgemでインストールする一部のライブラリはCで書かれておりコンパイルの必要があるみたいです。
vagrant@ubuntu-bionic:~/fluentd$ sudo apt-get install gcc
今度はこのエラー
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/nio4r-2.5.7/ext/nio4r
/usr/bin/ruby2.5 -r ./siteconf20210317-11655-1gfdz4j.rb extconf.rb
checking for unistd.h... yes
checking for linux/aio_abi.h... yes
checking for linux/io_uring.h... no
checking for sys/select.h... yes
checking for port_event_t in poll.h... no
checking for sys/epoll.h... yes
checking for sys/event.h... no
checking for port_event_t in port.h... no
checking for sys/resource.h... yes
creating Makefile
current directory: /home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/gems/nio4r-2.5.7/ext/nio4r
make "DESTDIR=" clean
sh: 1: make: not found
makeをいれよう…（requirementsのところに入れてもらえませんかね…）
vagrant@ubuntu-bionic:~/fluentd$ sudo apt-get install make
よし正常に終了した。
vagrant@ubuntu-bionic:~/fluentd$ sudo bundle install --path vendor/bundle [DEPRECATED] The `--path` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set --local path 'vendor/bundle'`, and stop using this flag Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine. Fetching gem metadata from https://rubygems.org/....... Fetching gem metadata from https://rubygems.org/. Fetching gem metadata from https://rubygems.org/. Resolving dependencies... Using rake 13.0.3 Using fiber-local 1.0.0 Fetching nio4r 2.5.7 Using timers 4.3.3 Using protocol-http 0.21.0 Using protocol-hpack 1.4.2 Using bundler 2.2.14 Using concurrent-ruby 1.1.8 Fetching cool.io 1.7.1 Installing nio4r 2.5.7 with native extensions Installing cool.io 1.7.1 with native extensions Using docile 1.3.5 Using flexmock 2.3.6 Fetching http_parser.rb 0.6.0 Installing http_parser.rb 0.6.0 with native extensions Fetching msgpack 1.4.2 Using sigdump 0.2.4 Fetching strptime 0.2.5 Installing strptime 0.2.5 with native extensions Installing msgpack 1.4.2 with native extensions Using webrick 1.7.0 Fetching yajl-ruby 1.4.1 Fetching oj 3.11.3 Installing oj 3.11.3 with native extensions Installing yajl-ruby 1.4.1 with native extensions Using parallel 1.20.1 Using power_assert 2.0.0 Using rr 1.2.1 Using simplecov-html 0.12.3 Using simplecov_json_formatter 0.1.2 Using timecop 0.9.4 Using console 1.10.1 Using protocol-http1 0.13.2 Using protocol-http2 0.14.2 Using tzinfo 2.0.4 Using serverengine 2.2.3 Using parallel_tests 0.15.4 Using test-unit 3.4.0 Using simplecov 0.21.2 Fetching async 1.28.9 Using tzinfo-data 1.2021.1 Fetching test-unit-rr 1.0.5 Installing async 1.28.9 Installing test-unit-rr 1.0.5 Using fluentd 1.12.1 from source at `.` Fetching async-io 1.30.2 Fetching async-pool 0.3.5 Installing async-io 1.30.2 Installing async-pool 0.3.5 Fetching async-http 0.54.1 Installing async-http 0.54.1 Bundle complete! 11 Gemfile dependencies, 39 gems now installed. Bundled gems are installed into `./vendor/bundle`
テストを実行してみる
終わるまでに数十分かかるので実行には注意！
vagrant@ubuntu-bionic:~/fluentd$ bundle exec rake test
起動してみる
vagrant@ubuntu-bionic:~/fluentd$ bundle exec fluentd -c fluent.conf
2021-03-17 14:40:13 +0000 [info]: parsing config file is succeeded path="fluent.conf"
2021-03-17 14:40:13 +0000 [info]: gem 'fluentd' version '1.12.1'
2021-03-17 14:40:13 +0000 [info]: adding forwarding server '192.168.0.12:24224' host="192.168.0.12" port=24224 weight=60 plugin_id="object:2affdfb11868"
2021-03-17 14:40:13 +0000 [info]: [forward_output] adding forwarding server '192.168.0.11:24224' host="192.168.0.11" port=24224 weight=60 plugin_id="forward_output"
2021-03-17 14:40:13 +0000 [info]: using configuration file: <ROOT>
<source>
@type forward
@id forward_input
</source>
<source>
@type http
@id http_input
port 8888
</source>
<source>
@type monitor_agent
@id monitor_agent_input
port 24220
</source>
<source>
@type debug_agent
@id debug_agent_input
bind "127.0.0.1"
port 24230
</source>
<match debug.**>
@type stdout
@id stdout_output
</match>
<match system.**>
@type forward
@id forward_output
<service>
host "192.168.0.11"
</service>
<secondary>
<service>
host "192.168.0.12"
</service>
</secondary>
</match>
</ROOT>
2021-03-17 14:40:13 +0000 [info]: starting fluentd-1.12.1 pid=16470 ruby="2.5.1"
2021-03-17 14:40:13 +0000 [info]: spawn command to main: cmdline=["/usr/bin/ruby2.5", "-r/var/lib/gems/2.5.0/gems/bundler-2.2.14/lib/bundler/setup", "-Eascii-8bit:ascii-8bit", "/home/vagrant/fluentd/vendor/bundle/ruby/2.5.0/bin/fluentd", "-c", "fluent.conf", "--under-supervisor"]
2021-03-17 14:40:14 +0000 [info]: adding match pattern="debug.**" type="stdout"
2021-03-17 14:40:14 +0000 [info]: adding match pattern="system.**" type="forward"
2021-03-17 14:40:14 +0000 [info]: #0 adding forwarding server '192.168.0.12:24224' host="192.168.0.12" port=24224 weight=60 plugin_id="object:2ab569a6086c"
2021-03-17 14:40:14 +0000 [info]: #0 [forward_output] adding forwarding server '192.168.0.11:24224' host="192.168.0.11" port=24224 weight=60 plugin_id="forward_output"
2021-03-17 14:40:14 +0000 [info]: adding source type="forward"
2021-03-17 14:40:14 +0000 [info]: adding source type="http"
2021-03-17 14:40:14 +0000 [info]: adding source type="monitor_agent"
2021-03-17 14:40:14 +0000 [info]: adding source type="debug_agent"
2021-03-17 14:40:14 +0000 [info]: #0 starting fluentd worker pid=16484 ppid=16470 worker=0
2021-03-17 14:40:14 +0000 [info]: #0 [debug_agent_input] listening dRuby uri="druby://127.0.0.1:24230" object="Fluent::Engine" worker=0
2021-03-17 14:40:14 +0000 [info]: #0 [forward_input] listening port port=24224 bind="0.0.0.0"
2021-03-17 14:40:14 +0000 [info]: #0 fluentd worker is now running worker=0
別のターミナルを開いてメッセージを送信
vagrant@ubuntu-bionic:~/fluentd$ curl -X POST -d 'json={"action":"login","user":2}' http://localhost:8888/debug
するとメッセージが表示されましたね。
2021-03-17 14:42:45.452403771 +0000 debug: {"action":"login","user":2}
お疲れ様でした！
