vector icon indicating copy to clipboard operation
vector copied to clipboard

Azure DCR-based custom logs -> Logs ingestion API

Open Idriel opened this issue 1 year ago • 1 comments

A note for the community

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

Trying to send Custom Logs into Azure Log Analytics Workspace over new DCR-based rules where we have complete control over Tables.

Also to note that the Azure Monitor HTTP Data Collector API has been deprecated and will no longer be functional as of 9/14/2026. It's been replaced by the Logs ingestion API.

Attempted Solutions

Current azure_monitor_logs is working but in v1, not in new v2 of Tables and logs ingestion API.

Proposal

Please implement new Azure standard for Logs Ingestion

References

No response

Version

vector 0.39.0 (x86_64-unknown-linux-gnu 73da9bb 2024-06-17 16:00:23.791735272)

Idriel avatar Jul 31 '24 18:07 Idriel

I am also facing the same problem. data is not inserting into custom tables. I am using user assigned identity for authentication.

Assign the role of "Monitoring Metrics Publisher" as per the documentation to user MI. Using python sdk and create custom table with only 2 columns and trying to insert data but python app did not throwing any exception or error.

And data is also not inserting into custom table.

ramsharan-choudhary-bzy avatar Aug 05 '24 08:08 ramsharan-choudhary-bzy

I've started taking a look at this (it's my first time using Rust for something non-trivial, so any feedback on what I've done wrong is welcome): https://github.com/vectordotdev/vector/compare/master...jlaundry:vector:feature-azure_logs_ingestion

Still a couple things I need to do before creating a PR--docs, unit tests, integration tests, find a way to not have to wait 5 mins during each cargo build--but the basic functionality seems to work.

jlaundry avatar Nov 02 '24 23:11 jlaundry

wow, nice job @jlaundry. Another benefit is the usage of azure-identity and no more workspace secret, would be great to move to managed identities. Looking forward to seeing your PR.

wiperpaul avatar Jan 29 '25 13:01 wiperpaul