Matt Joyce

ok, here is a draft, or example of how the script would be used to - completely theoretical. ``` name: CI/CD on: push: branches: - main jobs: build-and-publish: runs-on: ubuntu-latest...

The purpose of the parameter is to set the period between checks, so...(check, poll, status) x ( period, interval, frequency) check-period check-interval check-frequency poll-period **poll-interval** poll-frequency status-period status-interval status-frequency

Can confirm, this happens to me too, with python-sdk. However, only happens when I provide Scrape options. this works : `scraped_data = app.scrape_url(url)` this fails : ``` params = {...

Ok, so this does not work. ``` payload = { "url": "https://www.humblebundle.com/", "pageOptions": { "includeHtml": True, "waitFor": 123, "onlyMainContent": True, } } ``` but this does. ``` payload = {...

@rafaelsideguide , I see a lot of these, every scrape actually. > api-1 | Error fetching w/ playwright server -> URL: https://www.humblebundle.com/ with status: 404 and 'Page Cannot Be Found'...

@rafaelsideguide yes, pretty sure that is fixed by that patch. ``` app = FirecrawlApp(api_url="http://localhost:3002") url = "https://www.humblebundle.com" params={"pageOptions": { "includeHtml": False, "waitFor": 123, "onlyMainContent": True,}} scraped_data = app.scrape_url(url,params=params) ``` Works...

@ChrisMeye , the idea has merit, but the problem is there are a lot of different chunking strategies, and the right one is going to be dependent on the specific...

Interesting reading [5_Levels_Of_Text_Splitting](https://github.com/FullStackRetrieval-com/RetrievalTutorials/blob/main/tutorials/LevelsOfTextSplitting/5_Levels_Of_Text_Splitting.ipynb) [Embedding short and long content](https://www.pinecone.io/learn/chunking-strategies/)

Potentially cleave this part of to a micro service and provide a way to have alternate endpoint.

Interesting, relevant and very active project : https://github.com/jxnl/instructor