logstash-input-mongodb icon indicating copy to clipboard operation
logstash-input-mongodb copied to clipboard

Incorrectly parses ISODate

Open daniel-frak opened this issue 5 years ago • 0 comments

Given an example value in MongoDB:

ISODate("2020-01-22T09:45:04.613Z")

The retrieved field value is:

2020-01-22 09:48.04 UTC

I'm not sure if something is wrong with my config but I would expect an output like this:

2020-01-22T09:45:04.613Z

Due to the weird date format retrieved the ElasticSearch output fails as it cannot parse a date like that.

Logstash.conf:

input {
    mongodb {
        codec => "json"
        uri => 'mongodb://user:pass@mongodb:27017/mycollection?authSource=admin'
        placeholder_db_dir => '/usr/share/logstash/logstash-mongodb/'
        placeholder_db_name => 'myobject_logstash_sqlite.db'
        collection => 'myobject'
        batch_size => 5000
        generateId => 'true'
        parse_method => "simple"
    }
}
output {
    elasticsearch {
        hosts => ["elasticsearch:9200"]
        index => "myobject"
    }
    stdout {
        codec => rubydebug
    }
}

Additional info

I'm using Spring Boot 2.2.2 with Spring Data MongoDB to persist MyObject instances with an Instant field:

@Document(indexName = "article")
public class ArticleReadModel {

    @Id
    @Field(type = FieldType.Text)
    private String id;

    @Field(type = FieldType.Date)
    private Instant createdAt;

    // ...
}

daniel-frak avatar Jan 22 '20 10:01 daniel-frak