feat: add Olostep integration

[aadithyanr]

trafficstars

User description

Add Olostep Integration

This PR adds Olostep as a new integration provider to Julep, offering superior web scraping capabilities.

Features

• Faster response times (1-6 seconds vs. traditional crawlers)
• Built-in bot detection avoidance (no proxy needed)
• Parallel processing up to 100K requests
• Better content extraction with multiple formats

Implementation

• Added TypeSpec definition for Olostep
• Created integration implementation with full async support
• Added documentation for usage
• Updated provider registry

Files Added

• typespec/tools/olostep.tsp - TypeSpec definition
• integrations-service/integrations/utils/integrations/olostep.py - Integration logic
• integrations-service/tests/mocks/olostep.py - Test mocks
• documentation/docs/integrations/olostep.mdx - Documentation

Files Modified

• typespec/tools/models.tsp - Added Olostep to provider list
• integrations-service/integrations/providers.py - Registered provider
• integrations-service/integrations/env.py - Added API key variable

PS: Let me know if you'd like any changes or have questions about the implementation.

---

PR Type

Enhancement, Documentation, Tests

---

Description

• Added Olostep integration for advanced web scraping.

  • Implemented scrape method with async support.
  • Defined TypeSpec models for setup and arguments.

• Updated provider registry and environment configuration.

  • Registered Olostep as a provider in providers.py.
  • Added OLOSTEP_API_KEY to environment variables.

• Provided comprehensive documentation for Olostep usage.

  • Detailed configuration, methods, and examples in olostep.mdx.

• Added test mocks for Olostep integration.

---

Changes walkthrough 📝

	Relevant files
Configuration changes	env.py Add Olostep API key to environment variables                          integrations-service/integrations/env.py Added olostep_api_key environment variable. +1/-0
Enhancement	olostep.py Add Pydantic models for Olostep responses                                integrations-service/integrations/models/olostep.py Defined Pydantic models for Olostep responses. Added OlostepResponse and OlostepOutput classes. +12/-0    providers.py Register Olostep provider and methods                                        integrations-service/integrations/providers.py Registered Olostep as a provider. Added scrape method and provider info. +22/-0    olostep.py Implement Olostep scrape method with retry logic                  integrations-service/integrations/utils/integrations/olostep.py Implemented scrape method for Olostep API. Added retry logic and payload handling. Created helper functions for API key and response formatting. +76/-0    models.tsp Update TypeSpec models with Olostep provider                          typespec/tools/models.tsp Added olostep to integration provider list. +1/-0      olostep.tsp Add TypeSpec definitions for Olostep integration                  typespec/tools/olostep.tsp Defined TypeSpec models for Olostep setup and arguments. Added models for scraping parameters and responses. +83/-0
Tests	olostep.py Add test mocks for Olostep integration                                      integrations-service/tests/mocks/olostep.py Added mock response for Olostep API. +13/-0
Documentation	olostep.mdx Add documentation for Olostep integration                                documentation/docs/integrations/web_browser_automation/olostep.mdx Documented Olostep integration setup and methods. Included examples and best practices. Detailed error handling and features. +184/-0

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

---

EntelligenceAI PR Summary

Purpose: Enhance web scraping with Olostep for faster, bot-resistant, parallel processing.

Changes:

• New Feature: Integrated Olostep provider; added API key environment variable and response models.
• Documentation: Comprehensive Olostep setup and usage guides introduced.
• Test: Mock responses for Olostep API added for improved testing.
• Enhancement: Updated models.tsp and created olostep.tsp for setup and argument models.

Impact: Improved data extraction speed and reliability, enhanced user understanding, and more robust testing.