rapid icon indicating copy to clipboard operation
rapid copied to clipboard

Add URL and DomainName generators

Open wfscheper opened this issue 3 years ago • 5 comments

Generating proper URLs is tricky, and seems like something a library like rapid should provide. The implementation in hypothesis seems like a good starting point, though the URL generator is limited to http(s) schemes.

wfscheper avatar Oct 02 '20 03:10 wfscheper

Can you please tell what is your use case, and how configurable the generation should be, in your opinion? I don't have much experience using the hypothesis domain/URL generators.

flyingmutant avatar Oct 02 '20 09:10 flyingmutant

I want to test some code that parses a string as a URL in order to separate the scheme/host/port from the path, dropping any queries or fragments. I used the hypothesis URL strategy in another project, and thought it would make a reasonable addition to rapid.

As for an API, the domain strategy takes arguments for maximum length (total characters) and maximum elements (total domain segments), which seems reasonable. If a user needed to enforce a minimum for either property, writing a Filter wouldn't be hard.

For the URL generator, I was thinking it should return a *url.URL. That gives users a nice interface for Maps and Filters to work with. Conceivably, someone might want a generator for invalid URLs, but I don't think url.URLs do any validation beyond url.Parse so I think that use-case can be satisfied with a Map.

So in very rough pseudocode:

func Domain(maxLength, maxElements int) *Generator // cast generated values to string

func URL() *Generator // cast generated values to *url.URL

I can't think of any customizations for URL generation or alternative generators that wouldn't be easily done via a Map or a Filter. Maybe a URL generator that took a slice of schemes to sample from for applications that only deal with http URLs?

wfscheper avatar Oct 02 '20 12:10 wfscheper

Does this seem reasonable? I can start a pull request so we can look at some actual code.

wfscheper avatar Oct 15 '20 01:10 wfscheper

Looks reasonable, PR would be great!

I'd avoid any parameters in Domain for now: RFC 1035 specifies same limits as hypothesis defaults, and I can't come up with a common scenario when someone would want to change them.

For URL, I'd stick with HTTP/HTTPS schemes, just like Hypothesis does.

flyingmutant avatar Oct 15 '20 08:10 flyingmutant

Finally got around to trying to implement this. Not completely sure I'm using the best implementation here, as I'm not completely sure how to use some of the internal bits of rapid. Any suggestions for improvement are welcome.

I opted for Domain and DomainOf, since it wasn't that much harder to implement those two parameters.

The option to generate URLs with schemes other than http/https is available in the private implementation, but I didn't bother to expose it through a public API.

If you prefer the Domain generator follow that pattern, I can easily remove the DomainOf public function.

wfscheper avatar Nov 01 '20 00:11 wfscheper