Implement all uploads (and downloads) also in xlsx format in addition to xls and csv

[lukasmueller]

Expected Behavior

xls format is inadequate for most data, as it supports too few rows and columns

For Bugs:

Environment

Steps to Reproduce

[MFlores2021]

Accession upload Multi trial upload Observation upload

[MFlores2021]

Organize a hackathon in 2 weeks or so to change the formats

[dwaring87]

Here's a potential list of all of the Excel upload plugins:

./lib/CXGN/Location/ParseUpload/Plugin/LocationsExcelFormat.pm
./lib/CXGN/Pedigree/ParseUpload/Plugin/FamilyNameExcel.pm
./lib/CXGN/Pedigree/ParseUpload/Plugin/CrossesSimpleExcel.pm
./lib/CXGN/Pedigree/ParseUpload/Plugin/AdditionalInfoExcel.pm
./lib/CXGN/Pedigree/ParseUpload/Plugin/ProgeniesExcel.pm
./lib/CXGN/Pedigree/ParseUpload/Plugin/ValidateExistingProgeniesExcel.pm
./lib/CXGN/Pedigree/ParseUpload/Plugin/StoreExistingProgeniesExcel.pm
./lib/CXGN/Pedigree/ParseUpload/Plugin/CrossesExcelFormat.pm
./lib/CXGN/Pedigree/ParseUpload/Plugin/CrossInfoExcel.pm
./lib/CXGN/Pedigree/ParseUpload/Plugin/WishlistExcel.pm
./lib/CXGN/Stock/ParseUpload/Plugin/CatalogXLS.pm
./lib/CXGN/Stock/ParseUpload/Plugin/AccessionsXLS.pm
./lib/CXGN/Stock/Seedlot/ParseUpload/Plugin/SeedlotXLS.pm
./lib/CXGN/Stock/Seedlot/ParseUpload/Plugin/SeedlotHarvestedXLS.pm
./lib/CXGN/Stock/Seedlot/ParseUpload/Plugin/SeedlotMaintenanceEventXLS.pm
./lib/CXGN/Genotype/ParseUpload/Plugin/SSRProtocolExcel.pm
./lib/CXGN/Genotype/ParseUpload/Plugin/SSRExcel.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialPlantsSubplotWithNumberOfPlantsXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialPlantsSubplotWithPlantNumberXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialPlantsSubplotXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialSubplotsXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialPlantsXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialSubplotsWithSubplotNumberXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/SamplingTrialXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialPlantsWithNumberOfPlantsXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialPlotGPSCoordinatesXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/ProfileXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialUsedSeedlotsXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/GenotypeTrialXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/MultipleTrialDesignExcelFormat.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialSubplotsWithNumberOfSubplotsXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialPlantsWithPlantNumberXLS.pm
./lib/CXGN/Trial/ParseUpload/Plugin/TrialExcelFormat.pm
./lib/CXGN/Trial/ParseUpload/Plugin/SoilDataXLS.pm

[lukasmueller]

See #4132 for download issues using xls format. So downloads should also be adapted for xlsx (such as trial download, which easily exceeds xls format capabilities). Closing #4132 as a duplicate of this issue.

[lukasmueller]

Use Spreadsheet::ParseXLSX for parsing XLSX data?

[darekbienkowski]

At the moment branch with uploads from UI should work. It is tested on basic test packages with .xlsx file.

https://github.com/solgenomics/sgn/tree/topic/xlsx_files_extension

TODO:

• full test package - WIP
• uploads from Perl scripts, not UI WIP
• downloads - WIP

[lukasmueller]

implemented via #4296