katana consuming all CPU?

[marcelo321]

katana version:

[INF] Current version: v0.0.1

Steps To Reproduce:

So katana is consuming absolutely all CPU and like "killing" all other processes because it leaves no cpu for other processes. Is this normal or there is a way to control this? I am not using any special flags or headless, so shouldn't be this crazy..?

If there is a way to have this under control please let me know. the most similar tool to this would be hakrawler, which doesn't consume much cpu/ram at all (afaik).

[ehsandeep]

@marcelo321 how big is the URL list? Also what are the CPU specification of your machine?

[marcelo321]

Hi @ehsandeep,

So this is happening constantly when doing something like echo host.com | waybackurls | katana optionshere

I am using a pretty big VPS (8GB RAM and 4CPU), but I assume most people rely on more cheaper VPS options.

edit: removed few things

Check with the following example:

echo zonduu.me | waybackurls | katana

the site zonduu.me is my site that is hosted in github with little to no directories/files, so should be a light crawl.. but it will consume all CPU of your vps/machine.

Below would be the htop output of the command shared before:

Notice how the CPU is filled just from katana program alone.