hub .index.json not part of the rpm/deb pacakge for crowdsec

[L1ghtn1ng]

What happened?

When installing crowdsec it tries to reach out to pull down the hub index.json file but it assumes it has outbound internet access to do that. It should never assume that it has inital outbound access which is a bit silly of a security product to think that.

What did you expect to happen?

Crowdsec.service to be able to start on install

How can we reproduce it (as minimally and precisely as possible)?

Restrict a VM's outbound access so it cannot initially talk to crowdsec infra api urls and install the crowdsec rpm/deb. Watch it fail to start the service as it cannot pull down the .index.json file from the hub

Anything else we need to know?

No response

Crowdsec version

$ cscli version
1.5.4

OS version

# On Linux:
$ cat /etc/os-release
CentOS 7.9 and Ubuntu 22.04

</details>

[github-actions[bot]]

@L1ghtn1ng: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[LaurenceJJones]

I understand that if we bundled it, it would increase the install size by 100mb, and it may become out of date when being bundled, hence why we download it on install.

Even if we bundled it, the next issue is that it doesn't come with any parsers / scenarios. How would you install these without downloading manually (with a system that does have Internet access) and moving them across.

Edit: we do want to handle these installs better. I personally can't see an ideal solution unless the user zips an install and transfer it across.

[L1ghtn1ng]

All you need is to have the index.json file bundled that has the minimal requirements for it to work out of the box on Linux same for for the parsers, then once it’s up and running you can then install the proxy configuration to allow crowdsec to talk to its infrastructure and then update/install parsers etc and then it can keep everything up to date then. It just does not make sense to assume upon installing the crowdsec package and associated firewall bouncer that it will have have internet access straight away. As this may very well put the nail in the coffin for crowdsec role out at my day job if these issues cannot be addressed and there are plenty of people that have setups like this that you are losing money on due to this type of issue.

On 2 Oct 2023, at 17:46, Laurence Jones @.***> wrote:



I understand that if we bundled it, it would increase the install size by 100mb, and it may become out of date when being bundled, hence why we download it on install.

Even if we bundled it, the next issue is that it doesn't come with any parsers / scenarios. How would you install these without downloading manually (with a system that does have Internet access) and moving them across.

— Reply to this email directly, view it on GitHubhttps://github.com/crowdsecurity/crowdsec/issues/2507#issuecomment-1743369471, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AA3V2QSXNLRIIQKRSC3TRZDX5LVWPAVCNFSM6AAAAAA5PWBS4KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBTGM3DSNBXGE. You are receiving this because you were mentioned.Message ID: @.***>