sigmf-python icon indicating copy to clipboard operation
sigmf-python copied to clipboard

Multiple Recordings in an Archive

Open jhazentia opened this issue 2 years ago • 8 comments

This pull request is intended to add support for multiple recordings in archives. Fixes #11. Please let me know if you have any suggested improvements. Below is a summary of the changes:

  • Added README examples for loading a SigMF archive with multiple recordings, creating a SigMF archive, and creating SigMF archives with multiple recordings
  • Modified SigMFArchive to accept multiple SigMFFiles
  • Renamed SigMFArchive name to path
  • Modified SigMFArchive to append to existing tarfile when a fileobj is passed that references an existing, open tarfile. Note that, if desired, this can be changed back to always overwrite existing file.
  • Added pretty parameter to SigMFArchive to control pretty printing of SigMF metadata in archives
  • Added name parameter to SigMFFile. SigMFArchive will use the SigMFFile name parameter to create recording parent directories/file names in archive
  • Renamed SigMFArchiveReader name to path
  • Modified SigMFArchiveReader to read archives containing multiple recordings. The __len__(), __iter__(), and __getitem__() operate on the list of SigMFFiles instead of individual SigMFFile
  • Modified SigMFMetafile dump() and dumps() methods to append newline to the JSON to make this behavior consistent throughout the code.
  • Added __eq__() method to SigMFFile for testing
  • Misc other supporting changes and docstring updates.
  • Added supporting tests

Note that this PR does not include any changes to include a collection in archive, read collection from archive, or to update gui.py.

These changes are based on my colleagues' (Douglas Anderson @djanderson and Todd Schumann @ToddSchumann) changes: https://github.com/ntia/sigmf/tree/multi-recording-archive.

jhazentia avatar Jul 18 '23 20:07 jhazentia

Also note that this fixes a bug where SigMFFile offset_and_size was ignored by archive.py

jhazentia avatar Jul 18 '23 20:07 jhazentia

@gmabey what are your thoughts on this one?

777arc avatar Dec 12 '23 11:12 777arc

@gmabey what are your thoughts on this one?

Thanks for tagging me on this -- this PR had fallen off my radar, and I think I'm currently unsubcribed from changes to this project.

gmabey avatar Dec 14 '23 00:12 gmabey

If we want this feature then I'll probably need to dive in and resolve these merge conflicts. Lately been spending free coding time on AoC.

Teque5 avatar Dec 14 '23 16:12 Teque5

@gmabey Thanks for the feedback! I plan to address your feedback, but it will probably be after the new year. Is Python 3.6 support still needed? It looks like Python 3.6 and 3.7 are end of life now.

@Teque5 I can work on the merge conflicts when I address the other feedback.

jhazentia avatar Dec 22 '23 16:12 jhazentia

@jhazentia yeah so the reason why the SigMF project has persisted with python 3.6 support is (only because of) RedHat 8 ... that's just the reality of many users (including me), even though it's less than ideal.

gmabey avatar Dec 22 '23 16:12 gmabey

As I close all outstanding PRs, this one has been open for more than a year. As such I'm going to break it up into multiple smaller PRs and use @jhazentia's additions when possible starting with #98.

Since archive.py got mostly rewritten in #94 I'll wait to merge that before implementing the ability to archive one or more files, which was the original purpose of this PR.

Teque5 avatar Jan 10 '25 19:01 Teque5

Thanks @Teque5! I apologize I have not been able get back to this.

jhazentia avatar Jan 14 '25 22:01 jhazentia