go_spider
go_spider copied to clipboard
Metadata on Page
It'd be great if there was a way to store some metadata on Page via AddUrl, this would make the page processor easier to implement as opposed to doing some switching on urls or other http parameters. I think just a map of string to string would be fine.
Thank you for the feedback. That is very useful. I am going to add these functions in the next version.