`internal_logs` not following expected behavior when log_schema.timestamp_key is set

[awangc]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

When setting log_schema.timestamp_key = "@timestamp" globally, the expectation is that the source will use @timestamp field instead of vector's default of timestamp, but both fields are present for internal_logs source:

Configuration

data_dir = "./data_dir"
log_schema.host_key = "host.hostname"
log_schema.timestamp_key = "@timestamp"

  [sources.vector_logs]
    type = "internal_logs"

  [sinks.out]
    type = "console"
    inputs = [ "vector_logs" ]
    target = "stdout"
    encoding.codec = "json"

Version

vector 0.23.3 (x86_64-apple-darwin af8c9e1 2022-08-10)

Debug Output

❯ vector -c test.toml
2022-08-29T14:36:42.794755Z  INFO vector::app: Log level is enabled. level="vector=info,codec=info,vrl=info,file_source=info,tower_limit=trace,rdkafka=info,buffers=info,kube=info"
2022-08-29T14:36:42.794904Z  INFO vector::app: Loading configs. paths=["test.toml"]
2022-08-29T14:36:42.797169Z  INFO vector::topology::running: Running healthchecks.
2022-08-29T14:36:42.797239Z  INFO vector::topology::builder: Healthcheck: Passed.
2022-08-29T14:36:42.797308Z  INFO vector: Vector has started. debug="false" version="0.23.3" arch="x86_64" build_id="af8c9e1 2022-08-10"
2022-08-29T14:36:42.797368Z  INFO vector::app: API is disabled, enable by setting `api.enabled` to `true` and use commands like `vector top`.
{"@timestamp":"2022-08-29T14:36:42.797407Z","host":{"hostname":"HQSML-1719970"},"level":"\"vector=info,codec=info,vrl=info,file_source=info,tower_limit=trace,rdkafka=info,buffers=info,kube=info\"","message":"Log level is enabled.","metadata":{"kind":"event","level":"INFO","module_path":"vector::app","target":"vector::app"},"pid":40929,"source_type":"internal_logs","timestamp":"2022-08-29T14:36:42.794659Z"}
{"@timestamp":"2022-08-29T14:36:42.797432Z","host":{"hostname":"HQSML-1719970"},"message":"Loading configs.","metadata":{"kind":"event","level":"INFO","module_path":"vector::app","target":"vector::app"},"paths":"[\"test.toml\"]","pid":40929,"source_type":"internal_logs","timestamp":"2022-08-29T14:36:42.794888Z"}
{"@timestamp":"2022-08-29T14:36:42.797438Z","host":{"hostname":"HQSML-1719970"},"message":"Running healthchecks.","metadata":{"kind":"event","level":"INFO","module_path":"vector::topology::running","target":"vector::topology::running"},"pid":40929,"source_type":"internal_logs","timestamp":"2022-08-29T14:36:42.797163Z"}
{"@timestamp":"2022-08-29T14:36:42.797443Z","host":{"hostname":"HQSML-1719970"},"message":"Healthcheck: Passed.","metadata":{"kind":"event","level":"INFO","module_path":"vector::topology::builder","target":"vector::topology::builder"},"pid":40929,"source_type":"internal_logs","timestamp":"2022-08-29T14:36:42.797213Z"}
{"@timestamp":"2022-08-29T14:36:42.797454Z","arch":"x86_64","build_id":"af8c9e1 2022-08-10","debug":"false","host":{"hostname":"HQSML-1719970"},"message":"Vector has started.","metadata":{"kind":"event","level":"INFO","module_path":"vector::internal_events::process","target":"vector"},"pid":40929,"source_type":"internal_logs","timestamp":"2022-08-29T14:36:42.797301Z","version":"0.23.3"}
{"@timestamp":"2022-08-29T14:36:42.797461Z","host":{"hostname":"HQSML-1719970"},"message":"API is disabled, enable by setting `api.enabled` to `true` and use commands like `vector top`.","metadata":{"kind":"event","level":"INFO","module_path":"vector::app","target":"vector::app"},"pid":40929,"source_type":"internal_logs","timestamp":"2022-08-29T14:36:42.797363Z"}

Example Data

No response

Additional Context

No response

References

No response