firecrawl
firecrawl copied to clipboard
[Feat]: Add OpenLIT Integration for Monitoring
Problem Description Currently there is no way to monitor our web crawlers built with FireCrawl. Developers have to build their own observability pipeline which is not easy for everyone.
Proposed Feature Add Documentation for OpenLIT / Monitoring
Alternatives Considered NA
Implementation Suggestions I can contribute a Doc under the integrations section in FireCrawl Docs
Use Case This would enable developers to get complete insights into applications operations and performance in Production.
Additional Context I am one of the maintainers of OpenLIT and we support Observability for web crawler agents built with FireCrawl, It is all OpenTelemetry-native so the traces and metrics can be sent to any platform like Grafana or any OSS OTel tools
I think the biggest benefit for developers is when they are using LLMs along with a web crawler as then OpenLIT helps to give the complete picture of what actions were taken.
Figured the mention in the docs as an integration or a seperate doc for monitoring might be useful for developers using FireCrawl hence adding the reference in documentation page.