lnav
lnav copied to clipboard
tstack
No timestamp column for a custom format
Given a custom format (attached), I'm not seeing columns such as timestamp appear when performing an SQL query - in fact I'm not even convinced that I've got the custom format right. I'm raising the issue here though because the custom format looks right after reading the doc quite a few times.
Here's my command:
echo "2016-09-17T19:55:07Z ip-10-0-2-202.us-west-2.compute.internal DEBUG AgentInfo akka.tcp://[email protected]:2552/user/reaper/agent-info - akka.tcp://[email protected]:9004/user/reaper/agent-monitor/[email protected]:2552 requested agent info data."|lnav
When I go to query mode I see:
...so why is there no timestamp column available?
Here's the custom json:
{
"conductr" : {
"title" : "ConductR's log file format",
"description" : "The log file format for ConductR.",
"url" : "http://conductr.typesafe.com",
"regex" : {
"std" : {
"pattern" : "^(?<timestamp>\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}Z)\\s(?<hostname>\\S+)\\s(?<level>\\S+)\\s+(?<logger>.+)(?<mdc>\\[.*\\])?\\s-\\s(?<body>.*)"
}
},
"level-field" : "level",
"level" : {
"error" : "ERROR",
"warning" : "WARNING",
"info" : "INFO",
"debug" : "DEBUG"
},
"value" : {
"hostname" : {
"kind" : "string",
"collate" : "ipaddress",
"identifier" : true
},
"logger" : {
"kind" : "string",
"identifier" : true
},
"mdc" : {
"kind" : "string",
"identifier" : false
}
},
"sample" : [
{
"line" : "2015-08-24T08:08:02Z Christophers-MacBook-Pro.local INFO Slf4jLogger - Slf4jLogger started"
},
{
"line" : "2015-08-24T08:08:03Z Christophers-MacBook-Pro.local INFO ActorSystemImpl [ActorSystem=conductr] - ConductR is starting up..."
},
{
"line" : "2015-08-24T08:08:05Z Christophers-MacBook-Pro.local INFO ActorSystemImpl [ActorSystem=conductr] - Preparing to shutdown"
},
{
"line" : "2015-08-24T08:08:05Z Christophers-MacBook-Pro.local INFO ActorSystemImpl [ActorSystem=conductr] - Actor system is terminated [false]"
},
{
"line" : "2015-08-24T08:08:05Z Christophers-MacBook-Pro.local INFO ActorSystemImpl [ActorSystem=conductr] - Shutting down actor system"
},
{
"line" : "2015-08-24T08:08:05Z Christophers-MacBook-Pro.local INFO ActorSystemImpl [ActorSystem=conductr] - Awaiting termination..."
},
{
"line" : "2015-08-24T08:08:05Z Christophers-MacBook-Pro.local INFO RemoteActorRefProvider$RemotingTerminator akka.tcp://[email protected]:9004/system/remoting-terminator - Shutting down remote daemon."
},
{
"line" : "2015-08-24T08:08:05Z Christophers-MacBook-Pro.local INFO RemoteActorRefProvider$RemotingTerminator akka.tcp://[email protected]:9004/system/remoting-terminator - Remote daemon shut down; proceeding with flushing remote transports."
},
{
"line" : "2016-08-05T00:59:45Z 7b1767ba733f INFO ScaleScheduler [sourceThread=conductr-akka.actor.default-dispatcher-19, akkaSource=akka://conductr/user/reaper/scale-scheduler, sourceActorSystem=conductr, requestId=2c9494b4-bb55-4f42-8ca7-89757cf2bed9, bundleId=79d74c307f29e08640ce705f39670848, scale=1, tag=conductr.scaleScheduler.scaleBundleRequested, akkaTimestamp=00:59:45.062UTC] - Scale bundle requested: scale=1"
}
]
}
}
Sorry, the timestamp field gets mapped to the log_time column in the sqlite mapping. It's a bit non-obvious. You can take a look at all of the table definitions by executing the following in lnav:
;.schema
That should dump the full schema.
Ah nice. Would it be useful to show all of those columns available via .schema into the "known message fields" associated with the screen shot above?