[Feature request] Simple way to permanently store and use Blacklist of domains

[yufio]

There's usually a list of domains that you always want to exclude like googletagmanager.com which for me would contain at least 10 or so domains, but that could of course grow to contain domains from ad blocking lists.

Maybe a command line option that allows to specify a simple text file with new line separated blacklisted domains would work for this case quite well. Instead of specifying all of them in one command.

WDYT?

[snshn]

Hello Yuri,

that sounds like a good idea! I believe it can be achieved by using cat and some other methods, described here: https://www.baeldung.com/linux/file-line-command-argument

Please let me know if it works.

[htcrefactor]

I came here to see if this feature was suggested by anyone. Great idea!

[yufio]

@snshn yeah, that most likely would work.

What I'd love to have is not to have to do it at all. I'd rather just run monolith https://lyrics.github.io/db/P/Portishead/Dummy/Roads/ -o portishead-roads-lyrics.html and automatically apply the blacklist from the file (that for example is in the same dir as monolith with default name blacklist.txt). I now reread the initial comment and I see I didn't mention that.

I am not an avid shell user so I spent around 15 min to try to come up with an actual example how cat would work and I couldn't 🙈. I guess it would save some time for people like me to be able to just run monolith -B blacklist_permissive.txt https://lyrics.github.io/db/P/Portishead/Dummy/Roads/ -o portishead-roads-lyrics.html.

[htcrefactor]

Maybe the contents of blacklist_permissive.txt can be fed to the -B options?

I can't guarantee, but for example, ls -a -l and ~ls $(options.txt)~ ls $(cat options.txt) look identical, assuming options.txt is as below.

options.txt

-a
-l

If you have a list of domains to block, can you try and tell me if this works? Your command should look something like ~monolith -I -B -d $(blacklists.txt)~monolith -I -B -d $(cat blacklists.txt).

(I updated my comment to fix a mistake)

[yufio]

@htcrefactor which terminal emulator do you use? I tried echo $(blacklist.txt) in zsh and bash on MacOS but I get zsh/bash: command not found: blacklist.txt

[htcrefactor]

I've tested it on the default terminal emulator for Ubuntu 22.04 (Jammy), but I need to figure out what it is. I'll make sure to get back to you with more information later.

[yufio]

@htcrefactor not urgent or important. Please take your time and thanks for looking into it.

[htcrefactor]

Maybe monolith -I -B -d $(cat blacklists.txt) may work for you? This also looks more correct and compliant.

I found a relevant concept called command substitution in GNU's bash manual. According to the manual, ls -a -l, ls $(cat options.txt), and ls $(< options.txt) are all equal commands (when the previous assumption on options.txt are held valid).

I've tested it on Git Bash via Microsoft Terminal on Windows 11. Have a look at the below screenshot.

[htcrefactor]

I fixed my previous comment(quoted below) because it was incorrect.

Maybe the contents of blacklist_permissive.txt can be fed to the -B options?

I can't guarantee, but for example, ls -a -l and ~ls $(options.txt)~ ls $(cat options.txt) look identical, assuming options.txt is as below.

If you have a list of domains to block, can you try and tell me if this works? Your command should look something like ~monolith -I -B -d $(blacklists.txt)~monolith -I -B -d $(cat blacklists.txt).

[yufio]

@htcrefactor this is what I tried but it doesn't seem to work monolith -I -B -d $(cat blacklist.txt) -o humans-are-not-automatically-strategic.html https://www.lesswrong.com/posts/PBRWb2Em5SNeWYwwB/humans-are-not-automatically-strategic

error: Found argument 'https://www.lesswrong.com/posts/PBRWb2Em5SNeWYwwB/humans-are-not-automatically-strategic' which wasn't expected, or isn't valid in this context

USAGE:
    monolith [OPTIONS] <target>

For more information try --help

Same for: monolith -I -B -d $(cat blacklist.txt) https://www.lesswrong.com/posts/PBRWb2Em5SNeWYwwB/humans-are-not-automatically-strategic -o humans-are-not-automatically-strategic.html

error: Found argument 'https://www.lesswrong.com/posts/PBRWb2Em5SNeWYwwB/humans-are-not-automatically-strategic' which wasn't expected, or isn't valid in this context

USAGE:
    monolith [OPTIONS] <target>

For more information try --help

[snshn]

It looks like you might need to place -d prior to every line extracted from blacklist.txt... Possibly xargs could be used as described here https://unix.stackexchange.com/questions/393351/pass-contents-of-file-as-argument-to-bash-script but I think it may be way simpler than that.

Something like this: echo $(awk 'NF {print "-d " $0}' blacklist.txt)

This seems to be working: monolith -I -B $(awk 'NF {print "-d " $0}' blacklist.txt) https://www.lesswrong.com/posts/PBRWb2Em5SNeWYwwB/humans-are-not-automatically-strategic -o humans-are-not-automatically-strategic.html

[yufio]

@snshn Yep, that worked. Thanks for looking into it. I guess if we leave this for posterity the feature request can be closed 🤷‍♂️

[snshn]

Glad to hear. I'd like to keep it open until there's a support for an input file and/or config in Monolith, if you don't mind.