xls
xls copied to clipboard
modules/zstd: Rework ZSTD Decoder
This PR reworks the existing ZSTD Decoder to memory-based approach instead of relying on stream-based architecture.
The PR relies on peripherals (MemReader
and MemWriter
) for accessing external memory through AXI interface that are introduced in: https://github.com/google/xls/pull/1613. It is also based on https://github.com/google/xls/pull/1616.
The Decoder is able to decompress ZSTD frames that consist of RAW and RLE blocks. This is tested in a verilog simulation with a cocotb testbench.
The changes in this PR include:
- Rewriting
ZstdDecoder
proc to work with memory-based approach - Adding verilog-level tests of the
ZstdDecoder
- Introducing python library for generating ZSTD frames with
decodecorpus
- Adding python bindings for ZSTD reference library (zstandard) used for crosschecking decoding results in the simulation
- Adding cocotb testbench that generates multiple ZSTD frames with RAW and RLE blocks and decodes those with simulated ZstdDecoder
- Introducing python library for generating ZSTD frames with
- Reworking
FrameHeaderDecoder
,BlockHeaderDecoder
,Raw-
andRleBlockDecoders
to use memory-based approach instead of streaming input data from the input of the decoder - Adding
CsrConfig
andAxiCsrAccessor
procs that implement the internal registers of theZstdDecoder
and provide external (AXI) and internal (native) facing interfaces for accessing those - Simplifying:
-
RleBlockDecoder
proc -
BlockHeader
library
-
- Removing components not used in memory-based approach:
- magic number functions
-
BlockDecoder
proc -
DecDemux
proc
- Added
common_codegen_args
list to the BUILD file
This PR is currently a WIP
. It is opened to showcase the work done and to create a space for discussion on certain topics. The following list enumerates task that are currently being addressed and which are required for this PR to be completed before it is marked as ready for review
.
Next steps:
- [x] Update documentation - describe the new architecture of the decoder and how to interact with the IP from the software (CSRs)
- [x] Add DSLX tests for the ZstdDecoder proc - create facilities for preparing test data and write DSLX tests
- [x] Improve error handling - write error codes to the
Status
CSR - [ ] Improve verilog tests - reorganize testbench to handle decoding of multiple ZSTD frames in one test case
- [ ] Change the output format from stream-based to memory-based (write decoded frame to memory)
- [ ] Organize structs and new type aliases (cleanup)
- [x] Remove a workaround (https://github.com/google/xls/pull/1654/commits/85f1f80c7ddcf7016a282087bb342f16d313b1d6) for type check failure during IR generation for the
ZstdDecoder
proc (https://github.com/google/xls/issues/1655)
CC @proppy