Improvement/Ispconfig vhosts apache logs

[lukyrys]

Is possible add detecting ispconfig apache logs? in default ispconfig does not merge all vhosts to default access.log or other_vhosts_access.log under /var/log/apache2 Per webiste apache logs is placed under /var/log/ispconfig/httpd/< website vhost >/access.log ( /var/log/ispconfig/httpd/*/access.log ) access.log is symlink to actual date log in format 20210115-access.log

+ i noticed false detect ispconfig auth log placed at /var/log/ispconfig/auth.log for sshd auth.log

[buixor]

Are you refering to this : https://www.ispconfig.org/ ?

[lukyrys]

@buixor yes

[buixor]

Hello @lukyrys !

@sbs2001 opened an MR regarding this, but needs some testing on your side, can you help please ?

Thanks !

[LaurenceJJones]

Hey wanted to get involved in another good first issue!

Looking into the MR that was opened it would of worked but it seems that find with the -name parameter does not accept wildcard in the path variable.

So the way the wizard is setup even this should work since find is recursive by default

/var/log/ispconfig/httpd/access.log

[LaurenceJJones]

So coming back after failing the first time.

The issue is, the latest access.log for lspconfig is a symlink to the latest log file, so wizard.sh will always miss it. However @mmetc has been working on an improvement setup tool that can detect running services. So we can detect apache2 running in lspconfig the user would just need to edit the path.

[mmetc]

I have not followed until now, but I am not sure service detection has anything to do with this - we already know that the service is there. The issue is locating the log files.

There are several sub-issues, correct me if I'm wrong:

• latest.log is a symlink, when it rotates crowdsec is confused. Can we confirm this?
• we can use globbing to read the yyyymmdd-access.log files, but afaik it only supports one level (i.e. no /var/log/ispconfig/httpd/*/*.log ). Can we confirm this?
• if we catch all *.log files, we will have both latest.log and the file that latest.log links to -- hence we process stuff twice and the scenarios are triggered more easily. We can easily avoid that with an exclude regexp (from v 1.4.2) or by using *-access.log

[ThomasCr]

Any Update or Workaround?

[mmetc]

Hi @ThomasCr , by workaround you mean it doesn't work even with a manual setup?

Can you try this: in /etc/crowdsec/acquis.d/ispconfig.yaml

filenames:
  - /var/log/ispconfig/httpd/*/access.log
labels:
  type: apache2

then add this to /etc/crowdsec/config.yaml

crowdsec_service:
    acquisition_dir: /etc/crowdsec/acquis.d

Symlinks and double globbing are supported, so even /var/log/ispconfig/httpd//-access.log should work.

If your issue is automating the configuration, the current wizard.sh script is basically frozen. We are testing a new functionality in 1.5.0 but it's not ready yet. Adding the above file is a noop if the software is not there, so it could be enough for you.

[LaurenceJJones]

latest.log is a symlink, when it rotates crowdsec is confused. Can we confirm this?

Yes crowdsec doesn't know symlink has been updated so doesn't follow any updates