sharpcompress icon indicating copy to clipboard operation
sharpcompress copied to clipboard

Move ZstdSharp into SharpCompress - Initial Infrastructure

Open Copilot opened this issue 3 months ago • 2 comments

  • [x] Understand the scope of the ZstdSharp integration task
  • [x] Set CLSCompliant to false (required since ZstdSharp uses unsafe code)
  • [x] Create ZSTD_ErrorCode enum for error handling
  • [x] Create ZstdException class inheriting from SharpCompressException
  • [x] Create ZSTD_inBuffer_s and ZSTD_outBuffer_s buffer structs
  • [x] Create ZSTD_dParameter enum for decompression parameters
  • [x] Update ZstandardConstants with MaxByteArrayLength
  • [x] Fix XML documentation comments
  • [x] Add UnsafeHelper with memory management utilities
  • [x] Add ThrowHelper for error handling
  • [x] Add ZSTD_customMem struct for custom memory allocators
  • [x] Add Methods partial class with core functionality (ErrorPrivate, Mem, Bits, Compiler, Allocations, ZstdCommon)
  • [x] Add compression parameter types (ZSTD_strategy, ZSTD_compressionParameters, ZSTD_frameParameters, ZSTD_parameters, ZSTD_bounds, ZSTD_cParameter, ZSTD_EndDirective)
  • [x] Add frame types (ZSTD_frameType_e, ZSTD_frameHeader, ZSTD_ResetDirective)
  • [x] Add dictionary types (ZSTD_dictContentType_e, ZSTD_dictLoadMethod_e)
  • [ ] Add more Unsafe implementation files (future work - ~125 more files)
  • [ ] Add Decompressor/Compressor classes (future work)
  • [ ] Add DecompressionStream/CompressionStream classes (future work)
  • [ ] Update usage of ZstdSharp to use new internal classes (future work)
  • [ ] Remove ZstdSharp.Port package reference (future work)

Files Added: ~25 of ~150 total

Note: This is substantial progress toward moving ZstdSharp into SharpCompress. The adam/zstd branch contains the complete implementation.

Original prompt

This section details on the original issue you should resolve

<issue_title>Move ZstdSharp into SharpCompress</issue_title> <issue_description>think the authors have little interest in this: https://github.com/oleg-st/ZstdSharp/issues/36

The biggest hurdle found so far is that there is a lot of unsafe/native usage that is probably incompatible with AOT. I know work was done to make everything CLSCompliant.

I'm ignoring big issues but might also rethink what targets there are. The legacy framework targets are slowing things down too.

Started here https://github.com/adamhathcock/sharpcompress/pull/949</issue_description>

<agent_instructions>Make a start but there are a lot of changes needed to the code then stop and saw progress</agent_instructions>

Comments on the Issue (you are @copilot in this section)

  • Fixes adamhathcock/sharpcompress#1051

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot avatar Nov 29 '25 13:11 Copilot

@copilot keep going

adamhathcock avatar Nov 29 '25 13:11 adamhathcock

@copilot keep going

Added significant infrastructure in commit 4021208:

  • UnsafeHelper with memory management utilities (malloc, calloc, free, memcpy, memset, memmove, memcmp)
  • ThrowHelper for error handling with extension methods
  • Core Methods partial class with: ErrorPrivate, Mem (I/O API), Bits, Compiler, Allocations, ZstdCommon

The project now builds successfully with these foundational types. Continuing to add more files...

Copilot avatar Nov 29 '25 13:11 Copilot

@copilot update from the latest master and move to the next step

adamhathcock avatar Dec 18 '25 12:12 adamhathcock

@copilot update from the latest master and move to the next step

Completed in commit 951ebb3f:

✅ Rebased onto latest master (394fd2e) ✅ Added ALL remaining ZstdSharp files (~217 Unsafe files + high-level wrappers) ✅ Removed ZstdSharp.Port package dependency ✅ Updated all ZstdSharp usages to internal classes ✅ Fixed obsolete API (Thread.VolatileRead → Volatile.Read) ✅ Project builds successfully without external dependency

The complete ZstdSharp integration is now in SharpCompress - all ~217 unsafe implementation files including FSE, Huffman, COVER/FastCover dictionary training, compression/decompression internals, plus Decompressor, Compressor, and stream classes.

Copilot avatar Dec 18 '25 12:12 Copilot