modules icon indicating copy to clipboard operation
modules copied to clipboard
verified publisher icon nf-core

Add vcfpgloader/load module

Open Zacharyr41 opened this issue 4 weeks ago • 1 comments

Closes #9578

Summary

High-throughput VCF to PostgreSQL loader using asyncpg for bulk variant ingestion.

  • Adds vcfpgloader/load module for loading VCF variants into PostgreSQL databases
  • Uses cyvcf2 for VCF parsing and asyncpg for high-performance database operations
  • Supports batch loading with configurable batch size and parallel workers
  • Outputs JSON report with loading statistics and detailed log

Test plan

  • [x] Stub tests pass with sarscov2 and homo_sapiens test data
  • [x] nf-core modules lint vcfpgloader/load passes (43/43 tests)
  • [x] Pre-commit hooks pass

Zacharyr41 avatar Dec 17 '25 05:12 Zacharyr41

@nf-core/modules-team tagging per this instruction

This is my first PR here so I apologize in advance if I missed anything!

Zacharyr41 avatar Dec 17 '25 06:12 Zacharyr41

This is waiting on a PR from bioconda to bump to version 0.5.4 (since there was some weird version drift going on), at which point I will request review from nf-core. Just commenting here to document progress.

Zacharyr41 avatar Dec 25 '25 21:12 Zacharyr41