BUG: Dead-hosts run continuously without completing

[bigdargon]

Description

After a few days of running Dead-hosts, it seems to me that with a large number of domains (more than 17,000 domains) there will be some problems:

1. hostsVN-dead-hosts repeats from the beginning of list but does not continue to other domains when running a new Action. The counter is more than 150,000 domains link, while the list is only about 17,000 domains link
2. hostsVN-gambling-dead-hosts still runs the Action, doesn't repeat but doesn't seem to continue working. The list has more than 21,500 domains link, but currently stops at 18,000 domains link, the domain will continue when the upstream updates more new domains (ex: https://github.com/bigdargon/hostsVN-gambling-dead-hosts/commit/179a448901561b2a651033ab0a5a7651b5d18256)

• Updated: There are over 75,000 domains. Dead-hosts Action runs without end

3. hostsVN-adult-dead-hosts list has more than 12,000 domains link, running Actions without repeating errors like the 2 repos above.

• Updated: There are over 19,000 domains. Dead-hosts Action runs without end

It seems that the number of test domains will decrease over time when running Action. The longer the Action runs, the fewer domains are checked. Repo hostsVN-gambling-dead-hosts: https://github.com/bigdargon/hostsVN-gambling-dead-hosts/commits/master/output/origin.list/counter.json Repo hostsVN-adult-dead-hosts: https://github.com/bigdargon/hostsVN-adult-dead-hosts/commits/master/output/origin.list/counter.json

Configuration

cli_testing:
    autocontinue: true
    chancy_tester: true
    ci:
        active: true
        branch: master
        command: hash dead_hosts_launcher && dead_hosts_launcher --save
        commit_message: '[Autosave][Dead-Hosts::hostsVN-dead-hosts]'
        distribution_branch: master
        end_command: hash dead_hosts_launcher && dead_hosts_launcher --end
        end_commit_message: '[Final/Result][Dead-Hosts::hostsVN-dead-hosts]'
        max_exec_minutes: 20
    cooldown_time: 0.2
    display_mode:
        all: true
        dots: true
        execution_time: true
        less: false
    max_workers: 5
dns:
    server:
    - 45.90.28.0
    - 45.90.30.0
lookup:
    collection: false

[bigdargon]

Now, the Action only runs the trigger schedule and does not continue to run the next Action https://github.com/bigdargon/hostsVN-dead-hosts/commit/9bf717ffb0f2ab2b74e957038a4868d7069238f2 https://github.com/bigdargon/hostsVN-dead-hosts/commit/90070d9857e487eee10f27565a0a6efd4151ffbc

I will try a manual reset to run the Action again.

[bigdargon]

Repo same issue https://github.com/dead-hosts/blacklist_git_anudeepND The counter.json file has not been updated for 2 years https://github.com/dead-hosts/blacklist_git_anudeepND/blob/master/output/origin.list/counter.json

[bigdargon]

The action that runs continuously until the file size limit is reached is stopped https://github.com/bigdargon/hostsVN-dead-hosts/actions/runs/4350925983/jobs/7602096247

Current branch master is up to date.
remote: error: Trace: 6a0e18bb01ded603680028163ae88e5d633a08172600cc79b354de966f0a9640        
remote: error: See http://git.io/iEPt8g for more information.        
remote: error: File output/origin.list/splitted/ACTIVE is 100.06 MB; this exceeds GitHub's file size limit of 100.00 MB        
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com./        
To https://github.com/***/hostsVN-dead-hosts
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://github.com/***/hostsVN-dead-hosts'

[funilrys]

@bigdargon yeah GitHub changed their limits. That makes GitHub an unreliable source for our niche. That's why I'm working on a "lightweight" alternative that doesn't require git at all for testing.

Regarding the dead-hosts. Yes, I also saw that some repos are not updating. That's the main reason behind my idea of creating something new and more efficient, with a new WebApp.

[funilrys]

@bigdargon Can you please reach me through discord, matrix, telegram, or email? I'd like to chat about this issue.