fluent-plugin-pghstore icon indicating copy to clipboard operation
fluent-plugin-pghstore copied to clipboard

Fluentd output plugin which can write to PostgreSQL hstore extension.

= fluent-plugin-pghstore

== Component

=== PgHStoreOutput

Output to PostgreSQL hstore database.

Output table should have tag, time and record column.:

CREATE TABLE #{tablename} ( tag TEXT[], time TIMESTAMP WITH TIME ZONE, record HSTORE );

A tag is splited by "." and stored in the TEXT[]. (Yes, postgres can handle an array!)

== Requirement

  • PostgreSQL 9.0 or higher
    • hstore changed at PostgreSQL 9.0.
  • postgres-contrib
  • ruby-pg

=== How to Install hstore

hstore is in the contrib.

9.1 or higher:

psql -c "CREATE EXTENSION hstore;"

9.0:

psql -f ${PGHOME}/share/contrib/hstore.sql

== Configuration

Example:

<match apache.*> type pghstore database test table test table_option CREATE INDEX time_index ON testb (time);

=== Options

  • Required
    • database
      • database name
  • Optional
    • table
      • tablename.
      • If not set, use +fluentd_store+.
      • If not exists, creates automatically.
    • host
    • port
    • user
    • password
    • table_option
      • Add some SQL. This SQL is called only once when table is created from this plugin.

== Limitation

  • Nested output is not allowd.
  • Since using only one connection, the performance may become bad. When you meet this, use connection pooling and write patch!

== Thanks

This source code is mainly borrowed from

{fluent-plugin-datacounter}{https://rubygems.org/gems/fluent-plugin-datacounter}. Thank you for tagomoris.

== Copyright

Copyright:: Copyright (c) 2012- Shirou WAKAYAMA License:: Apache License

== ChangeLog

  • 0.2.2 / 2013-04-20

0.2.1 change caused error in 9.2. 0.2.1 change was not necessary. Thank you kaz29!

  • 0.2.1 / 2012-09-11

PostgreSQL 9.2 {removes hstore's => operator}{http://wiki.postgresql.org/wiki/What's_new_in_PostgreSQL_9.2#Remove_hstore.27s_.3D.3E_operator}. This plugin checks the server's version and change the operator.

But not fully tested yet.