Fluentd - Server name not found in configuration files - TinyIds

[harsh288]

I'm using fluentd and the plugin that I'm using is fluent-plugin-sql & activerecord-sqlserver-adapter when I have the host as "HOSTNAME/SQLSERVERINSTANCE" it gives an error "TinyTDS: Server name not found in configuration files", however when I just keep the HOSTNAME without "/" it works, this actually an issue in ruby on rails, there is the workaround I have seen like to use dataserver instead of the host but it is not working in fluentd using the plugin fluent-plugin-sql

<source>
  @type sql
  host HOSTNAME/SQLSERVERINSTANCE
  database db_name
  adapter sqlserver
  username user
  password pwd
  <table>
    table tbl_name
    update_column insert_timestamp
  </table>
</source>

<match **>
  @type stdout
</match>

@repeatedly @kenhys @frsyuki @cosmo0920 @ashie @ganmacs

[kenhys]

Hmm, if dataserver HOSTNAME/SQLSERVERINSTANCE is supported as configuration parameter, does it solve this situation?

[harsh288]

@kenhys It solves in activerecord-sqlserver-adapter however not in fluent-plugin-sql as it does not understand the keyword dataserver, which may be because that is not the accepted parameter.

[kenhys]

Hmm, what "not in fluent-plugin-sql" mean?

Even though dataserver parameter is supported, will it not resolved at all?

diff --git a/lib/fluent/plugin/in_sql.rb b/lib/fluent/plugin/in_sql.rb
index babec57..36ac9b9 100644
--- a/lib/fluent/plugin/in_sql.rb
+++ b/lib/fluent/plugin/in_sql.rb
@@ -41,6 +41,8 @@ module Fluent::Plugin
     config_param :socket, :string, default: nil
     desc 'PostgreSQL schema search path'
     config_param :schema_search_path, :string, default: nil
+    desc 'SQLServer dataserver (HOST\INSTANCENAME)'
+    config_param :dataserver, :string, default: nil
 
     desc 'path to a file to store last rows'
     config_param :state_file, :string, default: nil
@@ -192,6 +194,7 @@ module Fluent::Plugin
         socket: @socket,
         schema_search_path: @schema_search_path,
       }
+      config[:dataserver] = @dataserver if @dataserver
 
       # creates subclass of ActiveRecord::Base so that it can have different
       # database configuration from ActiveRecord::Base.

[harsh288]

@kenhys

I don't see the same code in location https://github.com/fluent/fluent-plugin-sql/blob/master/lib/fluent/plugin/in_sql.rb

[kenhys]

Yes, as the above code is PoC code. If it works, it may have a value to make it as a PR.

[harsh288]

@kenhys

Could you please help me with how to test that ?? I will test and let you know, so then we can make a PR. Also, the hostname is mandatory so if dataserver is passed as a parameter then hostname should not be mandatory.

[kenhys]

Apply patch and modify your configuration.

Something like this:

<source>
  @type sql
  host HOSTNAME
  database db_name
  adapter sqlserver
  dataserver HOSTNAME/SQLSERVERINSTANCE
  ...
</source>

[harsh288]

@kenhys

You mentioned patch, so I believe this is the location "C:\opt\td-agent\lib\ruby\gems\2.7.0\gems\fluent-plugin-sql-2.2.0\lib\fluent\plugin\in_sql.rb", I have just edited and made changes as per your suggestion, that is the only thing I need to do, no compilation or creating package ?? I tried with the below configuration where the "host" is mandatory and if I remove the "host" it gives the below error, and if I keep host then it takes host value and it tries to connect over there ignoring dataserver

<source>
  @type sql
  host HOSTNAME
  database db_name
  adapter sqlserver
  dataserver HOSTNAME/SQLSERVERINSTANCE
  ...
</source>

If I exclude "host"

In case if I keep blank "host" then

[kenhys]

Thanks for testing!

I have just edited and made changes as per your suggestion, that is the only thing I need to do, no compilation or creating package ??

Yes. It is what I mean. modifying it is enough.

if I keep host then it takes host value and it tries to connect over there ignoring dataserver

Hmm, I guess specifying both of host and dataserver parameters are harmless. but it might not be true for this case.

[harsh288]

@kenhys

So any idea what should I do in this case to make it work ??

[harsh288]

@ashie @kenhys Do you have a solution for this issue, it is very common to have multiple instances and the plugin should support it, if you have some temporary fix let me know.

[agup006]

Hey @harsh288, would this be a feature you would be able to contribute?

[harsh288]

Hi @agup006 I wish to, however, I don't have any experience in ruby so I'm requesting moderators to help over here.

[harsh288]

Hello @ashie @kenhys

We would be using the latest version of fluentd, so, are we going to support this feature, if not, could you suggest some workaround?