unblob: initial integration

[qkaiser]

Description

unblob is an accurate, fast, and easy-to-use extraction suite. It parses unknown binary blobs for more than 30 different archive, compression, and file-system formats, extracts their content recursively, and carves out unknown chunks that have not been accounted for.

unblob is mostly used by reverse engineers and pentesters alike, we don't have exact numbers on active users but we see ~1700 monthly downloads from PyPi, which is not the only distribution channel since unblob is also available on Kali Linux packages repository and through docker registries.

Corpus

unblob comes with 367 integration files covering all formats currently supported. These files are zipped up during the fuzzer build stage to provide a dedicated corpus to the fuzzer.

Fuzzing Harness

A single fuzzing harness is available right now, focusing on search_chunks in order to test the different unblob handlers. Unblob extractors are not covered, but if it's ever the case, we patched unblob in our fuzzing harness not to launch extractor subprocesses.

Since unblob consumes files, we have to create one from the raw bytes provided by Atheris. This is done through unblob's File.from_bytes API, which creates an mmap'ed file in shared memory. The performance impact is therefore kept to a minimum.

We've been running the fuzzer locally for some time, but would love to see it integrated into OSS-Fuzz !

[google-cla[bot]]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[github-actions[bot]]

qkaiser is integrating a new project:
- Main repo: https://github.com/onekey-sec/unblob
- Criticality score: 0.48640

[qkaiser]

My bad ! I set the primary_contact as a list rather than a string in the project.yaml file which broke the email check of the presubmit check. It's fixed now :)

[DonggeLiu]

Thanks, @qkaiser! May I ask who are the high-profile users of this project?

[qkaiser]

Thanks, @qkaiser! May I ask who are the high-profile users of this project?

We know it's embedded in commercial products, in open source projects such as EMBA and research projects like FirmwareDroid. Outside of commercial products, most users are security researchers / hackers looking into embedded firmwares.

I hope it answers your question.