thread 'main' panicked at 'called `Option::unwrap()` on a `None` value'

[marigbede]

It seems that replibyte is having some issues with None values in the database. Is there a workaround for this?

thread 'main' panicked at 'called Option::unwrap()on aNone value', replibyte/src/source/postgres.rs:288:48 stack backtrace: 0: rust_begin_unwind at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:584:5 1: core::panicking::panic_fmt at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:143:14 2: core::panicking::panic at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:48:5 3: replibyte::source::postgres::transform_columns 4: replibyte::source::postgres::read_and_transform::{{closure}} 5: dump_parser::utils::list_sql_queries_from_dump_reader 6: <replibyte::source::postgres::Postgres as replibyte::source::Source>::read 7: <replibyte::tasks::full_dump::FullDumpTask<S> as replibyte::tasks::Task>::run 8: replibyte::commands::dump::run 9: replibyte::main

My yaml is below. Please note that REDACTED replaced actual values. source: connection_uri: REDACTED transformers: - database: public table: users columns: - name: full_name transformer_name: random datastore: gcp: bucket: REDACTED region: us-west1 access_key: REDACTED secret: REDACTED destination: connection_uri: REDACTED

The command is ./replibyte -c /local/replibyte_config.yaml dump create

[evoxmusic]

Hi @marigbede , thanks for the report. Can you reformat your yaml conf file?

[marigbede]

@evoxmusic Thanks for picking this up. Please see this attached file.

replibyte.txt

[evoxmusic]

Thx, here is the content

source:
  connection_uri: REDACTED
  transformers:
    - database: public
      table: users
      columns:
      - name: full_name
        transformer_name: random
datastore:
  gcp:
    bucket: REDACTED
    region: us-west1
    access_key: REDACTED
    secret: REDACTED
destination:
  connection_uri: REDACTED

[marigbede]

@evoxmusic its looking as if the code crashes when NONE is encountered in the database. Is this the same experience you have?

[marigbede]

So @evoxmusic I have bene able to trace this back here https://github.com/Qovery/Replibyte/blob/a8c56df6e1277e7eadf0806105de995c7fd4d809/replibyte/src/source/postgres.rs#L289.

From the tests that I performed showed that even without the transforms config, the issue still occurs.

[spatail]

I am running into the same error when attempting to dump data from Postgres to S3 with no transformers.

My config file:

source:
  connection_uri: postgres://[redacted]
datastore:
  aws:
    bucket: [redacted]
    region: [redacted]
    credentials:
      access_key_id: [redacted]
      secret_access_key: [redacted]

I am starting Replibyte using Docker: docker run -it --rm -v "$(pwd)/replibyte.yaml:/replibyte.yaml:ro" replibyte -n --config replibyte.yaml dump create

The stacktrace:

thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', replibyte/src/source/postgres.rs:289:48
0: rust_begin_unwind
             at ./rustc/9d1b2106e23b1abd32fce1f17267604a5102f57a/library/std/src/panicking.rs:498:5
   1: core::panicking::panic_fmt
             at ./rustc/9d1b2106e23b1abd32fce1f17267604a5102f57a/library/core/src/panicking.rs:116:14
   2: core::panicking::panic
             at ./rustc/9d1b2106e23b1abd32fce1f17267604a5102f57a/library/core/src/panicking.rs:48:5
   3: replibyte::source::postgres::transform_columns
   4: replibyte::source::postgres::read_and_transform::{{closure}}
   5: dump_parser::utils::list_sql_queries_from_dump_reader
   6: <replibyte::source::postgres::Postgres as replibyte::source::Source>::read
   7: <replibyte::tasks::full_dump::FullDumpTask<S> as replibyte::tasks::Task>::run
   8: replibyte::commands::dump::run
   9: replibyte::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[marigbede]

Hi @evoxmusic any updates on this?

[evoxmusic]

Unfortunately, I am super busy for this week and the week after (Qovery new release). I keep you posted, I promise I don't forget :)

[fabriceclementz]

Hi @evoxmusic and @marigbede I will try to take a look this week

[fabriceclementz]

Hi @marigbede and @spatail Can you share with us the part of the dump that fails? I'd like to reproduce this bug to confirm the fix provided in #167 is correct.

[evoxmusic]

Hi @marigbede @spatail - if you can share your dumps with @fabriceclementz it will help a lot 👍🏽

[spatail]

Hi @fabriceclementz @evoxmusic, thanks for taking a look at this.

Unfortunately I don't have any dumps to share as this happens before any dump data is written to S3. The only thing created in S3 before the error is a file called metadata.json with the following content:

{"v":"0.8.6","dumps":[]}

Let me know if I can provide anything else. I can try rebuilding the docker image from the pull-request with the fix and see if that gets me further.

[spatail]

FYI, I copied the fix into my local workspace, rebuilt the docker image, and was able to run it successfully.

However, now I am noticing that after it dumps one file (1.dump), the progress is stuck:

I have attempted this a few times now and it always gets stuck at the same spot in the progress bar, with just one dump file in S3. My other attempts have run for over 20 mins with no change to the progress bar.

Not sure if this helps but my database has 2 schemas, and I am speculating that it may have only dumped one but somehow got stuck on the other.

[spatail]

Upon inspecting the dump file, I see data from both schemas. However, it looks like the dump is just incomplete, i.e. there should be multiple dump files, but only one file is ever written to S3.

The database at the moment is ~ 1GB in size. Could the progress stoppage be related to data size?

[gkesarwani]

@spatail any luck with the dump creation being stuck, I am seeing same behavior after ~750MBs downloaded.

[marigbede]

Hi @spatail and @evoxmusic. Nice work on the updates. Now the error is getting a bit more defined. thread 'main' panicked at 'assertion failed: (left == right)left:1, right: 0: Column names do not match values: got 1 names and 0 values', replibyte/src/source/postgres.rs:364:5 stack backtrace: 0: rust_begin_unwind at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:584:5 1: core::panicking::panic_fmt at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:143:14 2: core::panicking::assert_failed_inner 3: core::panicking::assert_failed 4: replibyte::source::postgres::transform_columns 5: replibyte::source::postgres::read_and_transform::{{closure}} 6: dump_parser::utils::list_sql_queries_from_dump_reader 7: <replibyte::tasks::full_dump::FullDumpTask<S> as replibyte::tasks::Task>::run 8: replibyte::commands::dump::run 9: replibyte::main note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace. No dumps available.

What does this mean?