LibClamAV Error: cli_realloc(): Attempt to allocate 190844928 bytes.

[benvakauta]

Describe the bug

Replace this text with a clear and concise description of the bug or feature request.

How to reproduce the problem

Replace this text with specific steps needed to reproduce the issue.

Replace this text with the output from the ClamAV command: clamconf -n

Attachments

If applicable, add screenshots to help explain your problem.

If the issue is reproducible only when scanning a specific file, attach it to the ticket.

[val-ms]

Hi @benvakauta your bug report doesn't have any details about how to reproduce the error in the issue title. Can you please share the file that causes this error, and include the output from running clamconf -n on your system?

[LouisRoss]

I saw this issue as well. Here's what I know

---

I don't think the platform information from clamconf -n below shows it, but my system has 16Gb of RAM, and excluding clamscan, htop shows 3.11G used. So, there should be lots available for a realloc(). My root / filesystem shows 286 GB free of 436 GB, so there is plenty of room for temporary files.

---

When executing this: sudo clamscan --max-filesize=3999M --max-scansize=3999M --exclude-dir=/sys/* -i -r /

There were hundreds of this line: LibClamAV Error: cli_realloc(): Attempt to allocate 190844928 bytes. Please report to https://github.com/Cisco-Talos/clamav/issues

---

louis@Research4:~$ clamconf -n Checking configuration files in /etc/clamav

Config file: clamd.conf

PreludeAnalyzerName = "ClamAV" LogFile = "/var/log/clamav/clamav.log" LogFileMaxSize = "4294967295" LogTime = "yes" LogRotate = "yes" ExtendedDetectionInfo = "yes" LocalSocket = "/var/run/clamav/clamd.ctl" LocalSocketGroup = "clamav" LocalSocketMode = "666" MaxConnectionQueueLength = "15" MaxThreads = "12" ReadTimeout = "180" SendBufTimeout = "200" SelfCheck = "3600" User = "clamav" BytecodeTimeout = "60000" MaxScanTime = "120000" MaxRecursion = "16" PCREMatchLimit = "10000" PCRERecMatchLimit = "5000"

Config file: freshclam.conf

LogFileMaxSize = "4294967295" LogTime = "yes" LogRotate = "yes" UpdateLogFile = "/var/log/clamav/freshclam.log" Checks = "24" DatabaseMirror = "db.local.clamav.net", "database.clamav.net" MaxAttempts = "5"

clamav-milter.conf not found

Software settings

Version: 0.103.8 Optional features supported: MEMPOOL IPv6 FRESHCLAM_DNS_FIX AUTOIT_EA06 BZIP2 LIBXML2 PCRE2 ICONV JSON

Database information

Database directory: /var/lib/clamav daily.cvd: version 26887, sigs: 2031667, built on Tue Apr 25 00:23:39 2023 bytecode.cvd: version 334, sigs: 91, built on Wed Feb 22 13:33:21 2023 main.cvd: version 62, sigs: 6647427, built on Thu Sep 16 05:32:42 2021 Total number of signatures: 8679185

Platform information

uname: Linux 5.19.0-40-generic #41~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Mar 31 16:00:14 UTC 2 x86_64 OS: linux-gnu, ARCH: x86_64, CPU: x86_64 Full OS version: Ubuntu 22.04.2 LTS zlib version: 1.2.11 (1.2.11), compile flags: a9 platform id: 0x0a21818108000000000b0300

Build information

GNU C: 11.3.0 (11.3.0) CPPFLAGS: -Wdate-time -D_FORTIFY_SOURCE=2

[aerobiotic]

lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.3 LTS Release: 22.04 Codename: jammy

I am seeing the same errors. When I checked the system in the morning after kicking off a scan in the morning the previous day. This software is very slow. My system was very slow. Ran the system monitor. 100% memory used. 100% swap used. Closed the browser that freed up some swap memory. Stopped clamscan system memory at 20% and swap at 51%. Looks like a memory leak.

The software made a 1.1G error output file.

805 Sep 16 10:36 scan-2023-09-16-10-36-51.log 1.1G Sep 17 07:00 scan-err-2023-09-16-10-36-51.log

lots of these errors `LibClamAV Error: cli_malloc(): Attempt to allocate 968113945 bytes. Please report to https://github.com/Cisco-Talos/clamav/issues LibClamAV Warning: Couldn't grow the blob: we may be low on memory LibClamAV Error: cli_realloc(): Attempt to allocate 190844928 bytes. Please report to https://github.com/Cisco-Talos/clamav/issues

much later

LibClamAV Warning: cli_scanxz: decompress file size exceeds limits - only scanning 2801795072 bytes LibClamAV Warning: PNG: Unexpected early end-of-file. LibClamAV Warning: PNG: Unexpected early end-of-file. LibClamAV Warning: PNG: Unexpected early end-of-file.

a bit near the end of the errors

LibClamAV Warning: Bytecode run timed out in interpreter after 5000 opcodes LibClamAV Warning: Bytecode 76 failed to run: Exceeded time limit LibClamAV Warning: cli_scanxz: decompress file size exceeds limits - only scanning 3337617408 bytes LibClamAV Warning: PNG: Unexpected early end-of-file. LibClamAV Warning: cli_scangpt: detected a non-protective MBR LibClamAV Warning: Bytecode run timed out in interpreter after 5000 opcodes LibClamAV Warning: Bytecode 77 failed to run: Exceeded time limit LibClamAV Warning: Bytecode run timed out in interpreter after 25000 opcodes LibClamAV Warning: Bytecode 76 failed to run: Exceeded time limit LibClamAV Warning: PNG: Unexpected early end-of-file. LibClamAV Warning: PNG: Unexpected early end-of-file.

`

`#!/bin/bash

validate() { startingFolder=$(pwd) script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd)" . $script_dir/styleGuideHelpers.sh

buildVars 1 }

doScanning() { sudo systemctl stop clamav-freshclam.service sleep 5

sudo freshclam

echo "Deep scan of / root. If you stop this script you must run this: " echo " sudo systemctl start clamav-freshclam.service"

sudo clamscan --bell --max-filesize=3999M --max-scansize=3999M --exclude-dir=/sys/* -i -r /

sudo systemctl start clamav-freshclam.service

echo "Completed" }

main() { validate "$@"

doScanning > >(tee -a scan-$zts.log) 2> >(tee -a scan-err-$zts.log >&2) } `

[bilkitty]

Managed to complete a scan with --stdout option as a workaround: sudo clamscan --stdout --max-filesize=1999M --max-scansize=1999M --exclude-dir=/sys/* -i -r /

Make sure to kill your browser, if Chrome.

[msugakov]

Hi,

I think I have a reproduction case.

First, system details

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.3 LTS
Release:	22.04
Codename:	jammy

$ clamscan --version
ClamAV 0.103.11/27171/Wed Jan 31 10:46:17 2024

$ free -h
               total        used        free      shared  buff/cache   available
Mem:            62Gi        22Gi       4,6Gi       1,6Gi        35Gi        38Gi
Swap:          2,0Gi       1,0Mi       2,0Gi

Here's how to reproduce:

$ curl --fail -O https://storage.googleapis.com/definitions.stackrox.io/scanner-data/2.32.2/pg-definitions.sql.gz
$ clamscan --infected --scan-archive=yes --alert-exceeds-max=yes --max-filesize=4095M --max-scansize=4095M --max-htmlnormalize=4095M --recursive pg-definitions.sql.gz

Eventually, it starts showing lots of these lines

LibClamAV Error: cli_realloc(): Attempt to allocate 190844928 bytes. Please report to https://github.com/Cisco-Talos/clamav/issues

Removing --max-htmlnormalize=4095M option makes the errors go away. It's unclear to me what does this option do and what value can be set safely. Here's what man says:

       --max-htmlnormalize=#n
              Maximum  size  of  HTML  file  to normalize. You may pass the value in kilobytes in format xK or xk, or
              megabytes in format xM or xm, where x is a number (default: 10 MB, max: <4 GB).

A particular value for this file that goes without errors is --max-htmlnormalize=1898M. One megabyte more, --max-htmlnormalize=1899M, and the errors appear again.

Also, I observed a significant time difference.

• --max-htmlnormalize=1898M (which causes no errors) runs for 1:08,08 total time (1m8s).
• --max-htmlnormalize=1899M --max-scantime=0 (which causes errors) executes for 7:47,49 total (7m47s).

Finally, if you don't have that version of ubuntu, this is reproducible in docker. For example,

$ docker run --rm -it --entrypoint=/bin/bash ubuntu:22.04
$ apt update
$ apt install clamav curl
$ curl --fail -O https://storage.googleapis.com/definitions.stackrox.io/scanner-data/2.32.2/pg-definitions.sql.gz
$ freshclam
$ clamscan --infected --scan-archive=yes --alert-exceeds-max=yes --max-filesize=4095M --max-scansize=4095M --max-htmlnormalize=1899M --recursive pg-definitions.sql.gz
# lots of
LibClamAV Error: cli_realloc(): Attempt to allocate 190844928 bytes. Please report to https://github.com/Cisco-Talos/clamav/issues
# eventually followed by
/pg-definitions.sql.gz: Heuristics.Limits.Exceeded.MaxScanTime FOUND

----------- SCAN SUMMARY -----------
Known viruses: 8683703
Engine version: 0.103.11
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 2021.86 MB
Data read: 199.12 MB (ratio 10.15:1)
Time: 320.502 sec (5 m 20 s)
Start Date: 2024:01:31 16:15:34
End Date:   2024:01:31 16:20:54

If I change ubuntu:22.04 to debian:bookworm (docker run --rm -it --entrypoint=/bin/bash debian:bookworm, remaining commands are the same) which has newer version of ClamAV, I see different errors:

# lots of
LibClamAV Warning: cli_realloc(): File or section is too large to scan (1073745920 bytes).                      For your safety, ClamAV limits how much memory an operation can allocate to 1073741824 bytes
# eventually followed by
/pg-definitions.sql.gz: Heuristics.Limits.Exceeded.MaxScanTime FOUND

----------- SCAN SUMMARY -----------
Known viruses: 8683852
Engine version: 1.0.3
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 2021.86 MB
Data read: 199.12 MB (ratio 10.15:1)
Time: 320.739 sec (5 m 20 s)
Start Date: 2024:01:31 16:10:09
End Date:   2024:01:31 16:15:30

Is it a bug in --max-htmlnormalize implementation or could you please recommend a safe combination of command line arguments?