Memory usage keeps increasing on md5sum compute

[shashi-banger]

Mountpoint for Amazon S3 version

mount-s3 1.0.1-unofficial+7643a22

AWS Region

us-east-1

Describe the running environment

Running on a local PC docker container. ALso experienced OOMKilled when running as a pod on AWS EKS

What happened?

• Docker started in privileged mode
• mount of s3 bucket was successful
• When trying to execute md5sum on a 10GB file and monitoring docker stats memory usage keeps increasing steadily to 2GB and above
• Even tried to chunk by chunk processing using the following python code snippet

def generate_file_md5(filepath):
    bufsize = 2**20 * 6
    buf = bytearray(bufsize)
    bufview = memoryview(buf)
    md5_hash = hashlib.md5()
    with open(filepath, 'rb', buffering=0) as f:
        while True:
            nread = f.readinto(bufview)
            if not nread:
                break
            md5_hash.update(bufview[:nread])
    return md5_hash.hexdigest()

same behaviour with above python code execution also

Relevant log output

CONTAINER ID   NAME        CPU %     MEM USAGE / LIMIT     MEM %     NET I/O          BLOCK I/O    PIDS
5363c739f1c3   keen_pike   20.20%    1.071GiB / 7.772GiB   13.78%    13.8GB / 108MB   0B / 270kB   16