Fabric icon indicating copy to clipboard operation
Fabric copied to clipboard

[Bug]: Readme documents the ability to scrape a web page that seems absent from the actual binary

Open DarwinJS opened this issue 1 year ago • 4 comments

What happened?

v1.4.100

The readme here: https://github.com/danielmiessler/fabric?tab=readme-ov-file#usage

Says " -U, --updatepatterns Update patterns" " -u, --scrape_url= Scrape website URL to markdown using Jina AI"

Fabric -h says: " --update, -u Update patterns"

My actual experience of -u on v1.4.100 does a pattern update.

Trying to use --scrape_url gives: fabric: error: unrecognized arguments: --scrape_url=<the url used>

So I can't find out if and how fabric can read a webpage as input?

Version check

  • [X] Yes I was.

Relevant log output

No response

Relevant screenshots (optional)

No response

DarwinJS avatar Nov 15 '24 11:11 DarwinJS

Is curling it and piping an effective method or does the raw html get in the way?

DarwinJS avatar Nov 15 '24 11:11 DarwinJS

@DarwinJS you can use pandoc to turn html to plaintext, so curl some web page and pipe to pandoc and to fabric

ksylvan avatar Mar 17 '25 21:03 ksylvan

fabric-ai -u <url> works for me as expected. Version: v1.4.185 installed via homebrew.

johnsaigle avatar May 06 '25 18:05 johnsaigle

Yeah, you are correct. fabric -U updates patterns for me and fabric -u does the web scraping.

I learned something new about fabric!

$ fabric -u 'https://github.com/danielmiessler/fabric/issues/1138' -p summarize_micro 
### ONE SENTENCE SUMMARY:
Fabric's README mentions web scraping feature absent in binary v1.4.100.

### MAIN POINTS:
- README claims Fabric can scrape URLs using Jina AI.
- Binary v1.4.100 lacks --scrape_url functionality, shows error.
- Later version v1.4.185 reportedly supports URL scraping.

### TAKEAWAYS:
- Check Fabric version for URL scraping compatibility.
- Use pandoc with curl as workaround for scraping.
- Update to latest version for potential fixes.

ksylvan avatar May 06 '25 21:05 ksylvan