sdk icon indicating copy to clipboard operation
sdk copied to clipboard

refactor: Added a generic `FileStream` (still in active development!)

Open edgarrmondragon opened this issue 1 year ago • 2 comments

References:

  • https://filesystem-spec.readthedocs.io/en/latest/api.html#base-classes

📚 Documentation preview 📚: https://meltano-sdk--2654.org.readthedocs.build/en/2654/

edgarrmondragon avatar Sep 06 '24 21:09 edgarrmondragon

CodSpeed Performance Report

Merging #2654 will not alter performance

Comparing 2648-feat-add-a-generic-filestream-interface (e527cee) with main (56b496f)

Summary

✅ 6 untouched benchmarks

codspeed-hq[bot] avatar Sep 06 '24 21:09 codspeed-hq[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.50%. Comparing base (5c68b09) to head (e527cee). Report is 99 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2654      +/-   ##
==========================================
+ Coverage   90.21%   90.50%   +0.29%     
==========================================
  Files          58       62       +4     
  Lines        4895     4994      +99     
  Branches      964      974      +10     
==========================================
+ Hits         4416     4520     +104     
+ Misses        331      328       -3     
+ Partials      148      146       -2     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Sep 06 '24 21:09 codecov[bot]

These are working now

FTP

{
    "filesystem": "ftp",
    "path": "fixtures/csv",
    "read_mode": "one_stream_per_file",
    "delimiter": "\t",
    "ftp": {
        "host": "127.0.0.1",
        "port": 21,
        "username": "my_ftp_user",
        "password": "my_ftp_password"
    }
}

SFTP

{
    "filesystem": "sftp",
    "path": "fixtures/csv",
    "read_mode": "one_stream_per_file",
    "delimiter": "\t",
    "sftp": {
        "host": "127.0.0.1",
        "port": 2022,
        "username": "my_ftp_user",
        "password": "my_ftp_password"
    }
}

edgarrmondragon avatar Sep 27 '24 23:09 edgarrmondragon