Awesome Kaitai

A curated list of Kaitai Struct tools and resources

• Main Website (Umbrella Repository, Github Pages, Issues)

Compiler

• Kaitai Struct: compiler to translate .ksy => .cpp / .cs / .dot / .java / .js / .php / .pm / .py / .rb (Repository, Releases)
• kaitaigo: alternative compiler to translate .ksy => .go, written in Golang
• nimitai: The compiler implemented as macro in Nimlang (.ksy => Nim AST)

Visualizers, IDEs, hex editors

Arbitrary file formats

These tools allow to develop your own Kaitai Struct specifications in iterative fashion by visualizing data locations in hex dump as one develops .ksy.

• Kaitai Web IDE (Repository, Github Page, Wiki Documentation)
  • Web IDE Docs - Images for the documentation
  • webide-usercontent.kaitai.io - Used in sandboxing for the WebIDE
• Development version of the IDE (Github Page)
• Kaitai Struct: visualizer and hex viewer tool (Repository)
• Kaitai Struct: visualizer and hex viewer tool GUI in Java (Repository, Issues)
• Kaitai Struct extension for VSCode (Marketplace, Overview video, Repository)
• Hobbits (Repository)

Fixed set of file formats

These visualizers / hex editors allow only fixed set of precompiled file format specifications to be used.

• Veles (Homepage, Repository)
• Kaitai Struct plugin for Binary Ninja (Repository)
• pytai (Repository)

Tools

• kaitaiStructCompile.py - Automate ksy compilation into python files (Repository)
• ksylint - A linter for .ksy files
• ksy-dl - Downloads .ksy files and their dependencies straight from the official kaitai-struct format gallery.

CI

Docker images

• registry.gitlab.com/kaitaistructcompile.py/kaitai_struct_python_docker:latest - an image with CPython, GraalVM, GraalPython, KSC, python runtime, kaitaiStructCompile.py, and its CLI backend. Unstable versions of the software are used. Currently Debian-based, but sometimes this will be migrated to Alpine.
• blacktop/kaitai
• librespace/kaitai — docker image of the kaitaistruct-compiler used in the Libre Space Foundation Database
• davefr/kaitai-ksc — Kaitai Struct compiler in a container (Repository)

Converter

• Convert C to ksy file (Snippet)
• Convert Synalyze It! Grammars to ksy files (Repository)
• Converting Kaitai structs to wireshark LUA plugins (Repository) - deprecated

Formats

• Kaitai Struct library of binary file formats (Repository, Github Pages)
• APFS (apple file system) format (Repository, Issues)
• DICOM (Digital Imaging and Communications in Medicine) file format spec for Kaitai Struct (Repository, Issues)
• EDID (VESA Enhanced Extended Display Identification Data) structure for Kaitai Struct (Repository)
• Java bytecode spec for Kaitai Struct (Repository, Issues)
• Windows resource file spec for Kaitai Struct (Repository, Issues)

Help, Documentation & Community

• Kaitai Struct Documentation (Repository, Github Pages)
• Stackoverflow
• Gitter channel
• Official Twitter account

Runtimes

• C++ using STL (Repository, Issues)
• C#/.NET (Repository, Issues)
• Go (Repository, Issues)
• JavaScript (Repository, Issues)
  • Examples (Repository)
  • Webpack loader for kaitai-struct .ksy definitions (Repository, Issues)
• Java (Repository, Issues)
• Lua (Repository, Issues)
• Nim (Repository, Issues)
• Perl (Repository, Issues)
• PHP (Repository, Issues)
• Python (Repository, Issues)
  • PIP packet
• Ruby (Repository, Issues)
• Rust (Repository, Issues)
• Swift (Repository, Issues)

Testing

• Tests for all languages (Repository)
• Compiled test files (Repository)
• Test results (Test Artifacts Repository)
• Benchmarking suite (Repository)

Misc

• KaitaiFS: mount any filesystem specified with a .ksy as a real file system (Repository, Issues)
• Compression processing libraries (Repository)

Other Resources

• https://kaitai.io/workshop/
• https://avatao.com/blog-kaitai/
• https://archive.fosdem.org/2017/schedule/event/om_kaitai/ - Presentation on Kaitai from Mikhail Yakshin (GreyCat)
• https://vaughanhilts.me/blog/2016/11/16/reverse-engineering-trails-in-the-sky-ed-6-game-engine.html - Blog post on game reverse engineering
• https://pythonistac.wordpress.com/2017/03/09/python-network-packet-dissection-frameworks-shootout-scapy-vs-construct-vs-hachoir-vs-kaitai-struct/ - Blog post comparing different network packet dissection frameworks
• https://medium.com/@MorteNoir/database-reverse-engineering-part-2-main-approaches-ae9355b2d429 - A blog post about reverse-engineering unknown file formats with a proprietary car parts database as an example.

Similar projects / tools

• 3D Model Researcher - Studying binary files of 3D models
• BeeSchema - Binary Schema Library for C#
• bindata - Binary data parsing for Ruby
• construct - Python library to create declarative parsers
• dtfabric
• vstruct2 - Python structure definition and parsing library
• https://github.com/0xdabbad00/icebuddha
• https://github.com/fox-it/dissect.cstruct
• https://github.com/frodef/binary-types - Read and write binary records for Common Lisp
• https://github.com/j3pic/lisp-binary - A library to easily read and write complex binary formats (Common Lisp)
• https://github.com/padsproj/pads
• https://github.com/renyxa/re-lab/tree/master/oletoy

Hex Editors

• hecate - Terminal hex editor
• Hexinator - Windows Version of Synalyze It!
• HxD - Small, fast hex editor for Windows
• iBored - Cross-platform, sector based hex editor
• Synalyze It! - Hex editor with templates for binary analysis
• wxHex Editor - Cross-platform editor with file comparison
• hexalepis - Win/Unix gui+terminal, tweak engine, .ksy visualization
• Hex Editor Neo - fast binary file editor for Windows (supports Kaitai Struct in its Structure Viewer)

File Grammars

• 010 Editor Templates - Templates for the 010 Editor
• Construct formats - Parser for different file formats for the python construct package
• HFSPlus Grammars - HFS+ grammars for Synalysis
• iBored Templates - Templates are packed inside the .app
• Sleuth Kit file system grammars - Grammars for different file systems
• Synalyse It! Grammars - File type grammars for the Synalyze It! editor
• TestDisk grammars - Grammars used by TestDisk and PhotoRec
• WinHex Templates - Grammars for the WinHex editor and X-Ways
• Wireshark dissectors - Parsers for Wireshark