Migrate from SQLite to Postgres with Askar

[niall-shaw]

I thought this would be as simple as exporting the sqlite wallet, and importing into a postgres wallet. But AskarWallet.ts has the following error:

throw new WalletError('Import is only supported for SQLite backend')

So a couple of questions:

1. Why is importing not supported for Postgres, is it planned?
2. How would one migrate from SQLite to Postgres without exporting/importing?

[niall-shaw]

Related to https://github.com/hyperledger/aries-askar/issues/58

[jakubkoci]

I guess that it can be solved by exporting from SQLite and importing to Postgres, but I couldn't find any API for export/import.

[genaris]

At the time when we implemented initial Askar module, aries-askar did not include any feature to import/export stores natively, so our alternative was to just do the manual work of copying the entire database and re-keying the wallet.

However, now that https://github.com/hyperledger/aries-askar/pull/159 has been merged, I think it should be possible to copy an entire store (or an individual profile) to another one. It might even be possible to switch between backends, which would be certainly promising.

I guess as soon as https://github.com/hyperledger/aries-askar/pull/169 and https://github.com/hyperledger/aries-askar/pull/170 are merged, a new release of Askar JS wrappers can be done and we could incorporate its enhancements to AskarWallet.

[TimoGlastra]

Was already typing my reply, but you covered it pretty well @genaris :)

[niall-shaw]

Thanks again for the prompt replies @genaris @TimoGlastra !

[DJHunn39]

@genaris @TimoGlastra with both of those PRs merged, has a new release of the Askar JS wrapper been made/has that been incorporated into the AskarWallet module?

I'm just trying to update storage for an agent on @aries-framework 0.4.x to @credo-ts 0.5.x (an alpha build, for now), but I'm running seeing these logs when attempting to auto update an agent using postgres for storage:

INFO: Starting update process. Total of 1 update(s) will be applied to update the agent storage from version 0.4 to version 0.5
FATAL: Error updating storage with updateIdentifier 1707408853441 because the wallet backend does not support exporting.
         Make sure to do a manual backup of your wallet and disable 'backupBeforeStorageUpdate' before proceeding.
    updateIdentifier: "1707408853441"
    error: {}

This is similar to the error from OP, but it's because export is also not supported for non-SQLite back ends.

[genaris]

@genaris @TimoGlastra with both of those PRs merged, has a new release of the Askar JS wrapper been made/has that been incorporated into the AskarWallet module?

I'm just trying to update storage for an agent on @aries-framework 0.4.x to @credo-ts 0.5.x (an alpha build, for now), but I'm running seeing these logs when attempting to auto update an agent using postgres for storage:

INFO: Starting update process. Total of 1 update(s) will be applied to update the agent storage from version 0.4 to version 0.5
FATAL: Error updating storage with updateIdentifier 1707408853441 because the wallet backend does not support exporting.
         Make sure to do a manual backup of your wallet and disable 'backupBeforeStorageUpdate' before proceeding.
    updateIdentifier: "1707408853441"
    error: {}

This is similar to the error from OP, but it's because export is also not supported for non-SQLite back ends.

In case of PosgreSQL wallets, it is possible to update but you'll need to explicitly set backupBeforeStorageUpdate to false in your Agent config.

It is of course recommended that you make a backup of your DB before doing any update using this setting.

[DJHunn39]

@genaris @TimoGlastra with both of those PRs merged, has a new release of the Askar JS wrapper been made/has that been incorporated into the AskarWallet module? I'm just trying to update storage for an agent on @aries-framework 0.4.x to @credo-ts 0.5.x (an alpha build, for now), but I'm running seeing these logs when attempting to auto update an agent using postgres for storage:

INFO: Starting update process. Total of 1 update(s) will be applied to update the agent storage from version 0.4 to version 0.5
FATAL: Error updating storage with updateIdentifier 1707408853441 because the wallet backend does not support exporting.
         Make sure to do a manual backup of your wallet and disable 'backupBeforeStorageUpdate' before proceeding.
    updateIdentifier: "1707408853441"
    error: {}

This is similar to the error from OP, but it's because export is also not supported for non-SQLite back ends.

In case of PosgreSQL wallets, it is possible to update but you'll need to explicitly set backupBeforeStorageUpdate to false in your Agent config.

It is of course recommended that you make a backup of your DB before doing any update using this setting.

That's true, and I can do that for now, but is there a plan to add auto backup support for Postgres backends?