Restore dump are real data with transformers

[Nicolas-Her]

Hi,

i try to anonymize data on dump but when i restore it on my dev environnement i have real data

do i miss something ?

source:
  connection_uri: $DATABASE_URL
  transformers:
    - database:  ****
      table:  ****
      columns:
        - name: name
          transformer_name: first-name
        - name: email
          transformer_name: custom-wasm
          transformer_options:
            path: ****
        - name: photo_path
          transformer_name: custom-wasm
          transformer_options:
            path:  ****
        - name: remember_token
          transformer_name: random
        - name: guest_email
          transformer_name: email
    - database:  ****
      table:  ****
      columns:
        - name: floor_plan_path
          transformer_name: custom-wasm
          transformer_options:
            path:  ****
    - database:  ****
      table:  ****
      columns:
        - name: value
          transformer_name: custom-wasm
          transformer_options:
            path:  ****
    - database:  ****
      table:  ****
      columns:
        - name: value
          transformer_name: custom-wasm
          transformer_options:
            path:  ****

[TheKipmaster]

I have the same issue. Seemingly correctly configured .yml file outputs exactly the same data. No transformation takes place.

I am running the following commands, to no avail: cat test_dump.sql | replibyte -c conf.yml dump create -i -s postgresql replibyte -c conf.yml dump restore local -i postgres -v latest -o > test_transform.sql

Can anyone shed any light on this?

[sealed-rayboutotte]

Same issue here, I took a local dump, transformed it, then restored it back.Does transformation not work on existing dumps? Makes this useless as I need to use already existing dumps.

[sealed-rayboutotte]

Actually got this to work. It seems that I had to use the command below to get a dump. Its the dump format ... if it uses sql statements (i.e. insert) then it should transform the data. But if it uses COPY public.table (column_a, colum_b) FROM stdin; then data is NOT transformed.

pg_dump --column-inserts --no-owner -p 5434 -U postgres postgres

It would be nice if replibyte could support this more compressed dump format.