funda-scraper
funda-scraper copied to clipboard
Change type bool args to store_true action
Hi @whchien, thanks for building this scraper! I was using this recently from the CLI and I noticed that some args are not correctly parsed. For example, running the command from the README:
❯ python funda_scraper/scrape.py --area amsterdam --want_to rent --find_past False --page_start 1 --n_pages 3 --raw_data True
When printing args.find_past
, this evaluates to True meaning that it unintentionally uses past data even though I expected recent data.
This is because some args are parsed as type=bool
, which unfortunately often leads to a different behavior than expected. "False" is interpreted as string and that evaluates True in Python. On the other hand, inputs like '' and 0 will evaluate as False. The same problem occurs with the --raw_data
and --save
arguments.
One way to avoid this problem is to use action=store_true
instead. I have made these changes along with a few small quality-of-life improvements.