refact: make custom config also defined within env blocks.

[Colstuwjx]

Hi team,

This PR is an alternative for #244 , we have been tried to use the #244 PR proposed configuration way, however, it didn't work well in my cluster. We don't want to maintain another configmap to map to the custom_configs env, and it also hard to set such multiple-line content with a single env kv pair.

As a result, We try to propose another solution for solving such customization for filebeat ( even the other log-collector ), just take the same way that Tags do and it seems also match our log-pilot design.

Feel free to merge this PR or close it. Thanks.

[codecov-io]

Codecov Report

Merging #264 into master will increase coverage by 0.65%. The diff coverage is 56.25%.

@@            Coverage Diff             @@
##           master     #264      +/-   ##
==========================================
+ Coverage   21.46%   22.11%   +0.65%     
==========================================
  Files           7        7              
  Lines         778      755      -23     
==========================================
  Hits          167      167              
+ Misses        581      559      -22     
+ Partials       30       29       -1

Impacted Files	Coverage Δ
pilot/piloter.go	50% <ø> (+30%)	:arrow_up:
pilot/filebeat_piloter.go	6.57% <ø> (ø)	:arrow_up:
pilot/pilot.go	26.87% <56.25%> (+0.69%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update af6da63...1e1938d. Read the comment docs.

[Colstuwjx]

@chenquanzhao , I'm sorry, I have tried the #244 way, and this PR is my answer to customize the log-collect config for each container.

Just as the PR description said, feel free to merge or close this PR. Thanks.

[ymping]

@jzwlqx Hope this PR cat be accept. I tested it on my cluster, it's working fine. Use example :

environment:
   aliyun_logs_access: /usr/local/tomcat/logs/localhost_access_log.*.txt
   aliyun_logs_access_configs: multiline.pattern='^[0-9]{3}.*',multiline.negate=true,multiline.match=after

And I also used #244 way to config my yaml file, seems not working.

[Colstuwjx]

@ymping , I have been created two commits in my forked log-pilot, included this PR and #275 , you can use this forked version until this PR is accepted by upstream, although it needs build image by manually, the forked version see this repo.