go-passbolt icon indicating copy to clipboard operation
go-passbolt copied to clipboard

Support Passbolt v5 API

Open speatzle opened this issue 11 months ago • 28 comments

Full Support for Resource Types, Encrypted Metadata and the v5 API.

This will Most likely mean Breaking the go-passbolt API by just exposing a Maps in the Resource Struct to Support Resource Types and Encrypted Metadata Properly.

https://www.passbolt.com/blog/the-road-to-passbolt-v5-encrypted-metadata-and-other-core-security-changes-2

https://docs.google.com/document/d/1yuucNxUJYcYFqsLUORTRqd4HZs1vROmtQQ6stGbdwpU/edit?pli=1&tab=t.0

speatzle avatar Jan 10 '25 12:01 speatzle

Hi @speatzle, I would love to work with you on this. When do we start?

Nelwhix avatar Jan 23 '25 19:01 Nelwhix

Hi, Sorry for the delay.

Thank you for your interest in helping out.

The big thing that v5 brings is encrypted metadata, but its probably also gonna expand on the Resource Types introduced in v3.

Back then i was already struggling to find time so instead of Properly implementing Resource Types i just made it work for the few types that exist with the existing go-passbolt Resource struct.

Since Encrypted metadata is going to change the Format of a Resource again i don't think we can just Expand the Struct anymore.

The current plan i have in mind up would be to have just the few unaffected fields Directly in the Resource Struct (id, folder_parent_id, ???) and have one map for the Metadata and one map for the Secret data which we both check against the Resource Type Json Schema from the Server.

We could then have Generic Resource Type variants which expose helper functions to get the Various values of the fields the Resource Type in Question has.

This would allow for Ergonomic programming with the usual Resource Types and still allow for all custom or new and currently unknown (to go-passbolt) Resource Types to be used.

Since all of this has just been in the Back of my head since the v5 announcement i would appreciate any suggestion / improvements on these ideas or alternative solutions.

speatzle avatar Feb 17 '25 23:02 speatzle

I think your idea is sound. When you are ready to start just create issues and we will tackle them together

Nelwhix avatar Feb 18 '25 11:02 Nelwhix

@speatzle Hi man, what v5 instance do we use to test. Community edition's last release is v4.11 https://github.com/passbolt/passbolt_api/releases/tag/v4.11.1

Nelwhix avatar Mar 04 '25 15:03 Nelwhix

@Nelwhix since v4.10 there should be a flag PASSBOLT_V5_ENABLED to enable the new API Version, you can test against that.

I managed to get some time approved at work to get this moving forward. I also had a meeting with the Passbolt team Today since i had some Questions after reading the google doc.

After Talking with them, i have decided to delay the more Generic approach since Custom Resource Type are still a while of and the more Hard coded approach is currently preferred by them due to downgrade security concerns which will be solved in the future.

Ill make a v5 Branch for now, all PR's should go against that till v5 support is ready for main.

I plan to work on #26 tomorrow and maybe even #27 which are fundamental for most of the Work.

If you want to work on something in the next few days, you could take a look at #28

speatzle avatar Mar 04 '25 15:03 speatzle

Okay thanks

Nelwhix avatar Mar 04 '25 15:03 Nelwhix

@Nelwhix @stripthis I have made the Integration tests run only if the Environment Variable REG_URL is set, This way you can test all other unit tests locally if you don't have a Passbolt instance running on your system.

Also Github Actions have been enabled for the v5 branch and pull's targeting v5.

Output should look like this:

go-passbolt/helper took 4s ➜  go test -v
REG_URL Env Variable Empty, Skipping integration tests
=== RUN   TestResourceCreate
--- SKIP: TestResourceCreate (0.00s)
=== RUN   TestGenerateOTPCode
=== RUN   TestGenerateOTPCode/generates_otpcode_from_token_with_padding
=== RUN   TestGenerateOTPCode/generates_otpcode_from_token_without_padding
=== RUN   TestGenerateOTPCode/invalid_token_format
--- PASS: TestGenerateOTPCode (0.00s)
    --- PASS: TestGenerateOTPCode/generates_otpcode_from_token_with_padding (0.00s)
    --- PASS: TestGenerateOTPCode/generates_otpcode_from_token_without_padding (0.00s)
    --- PASS: TestGenerateOTPCode/invalid_token_format (0.00s)
PASS
ok  	github.com/passbolt/go-passbolt/helper	0.004s

speatzle avatar Mar 05 '25 16:03 speatzle

Well done

Nelwhix avatar Mar 05 '25 16:03 Nelwhix

Done with #28, can I pick any or are we doing them in order?

Nelwhix avatar Mar 05 '25 16:03 Nelwhix

Hi @speatzle, sorry for going offline for a bit. my main work has been overwhelming. Please what is the current status and what can I help with?

Nelwhix avatar May 21 '25 22:05 Nelwhix

@Nelwhix Currently i am Working on Multiple Metadatakeys, Trusting new ones and so on.

You could take a look at #51 and #30

speatzle avatar May 22 '25 08:05 speatzle

Hello,

First of all, thank you for the incredible work you do. As a sysadmin, I rely heavily on your project to administrate my passbolt CE.

I recently upgraded to Passbolt v5 without realizing that the project doesn't support it. I now have nearly 200 passwords to distribute across various teams.

Could you please let me know if there's a timeline or roadmap for when the sharing feature will be supported in v5? This would help me plan accordingly or consider temporary workarounds if needed.

Thanks again for your continued efforts.

Best regards,

GalactikDrkCoder avatar Jul 28 '25 10:07 GalactikDrkCoder

Hi, go-passbolt can work with v5, only the new Resources types are unsupported, you can re enable v4 Resources (Cleartext Metadata) for creation in your servers settings:

Image

If you also want listing/modifying Resources to work then you need to disable The Encrypted Metadata Content Types and Migrate back the Resources to the v4 Ones from the Settings.

There is no timeline. I believe i have already done most of the work. The Only Blocker for getting the basics working is the metadata key management / trusting and how that should work (skipping all caching and folder things).

Ill have some time to work on this again next Week but no guaranties.

speatzle avatar Jul 28 '25 11:07 speatzle

Ah ok, very good news ! Because when I launch this command .\passbolt.exe share resource --id 8d -g 9ae -t 1 I got this error Error: Sharing Resource: Validating Secret Data: Validating Secret Data: invalid character 1 after top-level value For more context, I download the Release v0.3.2 for Windows, I configure the tool and I'm able to list users and resources without error but the share command doesn't work. Thanks again for your work 👍 PS : I don't know if I need to create another issue, tell me and I execute

GalactikDrkCoder avatar Jul 28 '25 12:07 GalactikDrkCoder

Can you run that command with the --debug flag and post the output (Needs some censoring) in a new issue?

speatzle avatar Jul 28 '25 12:07 speatzle

Once this is ready (released or pre-release), I'd be happy to test this on our Passbolt database with around two thousand secrets, all with encrypted meta data.

ravage84 avatar Sep 04 '25 16:09 ravage84

@ravage84 There is a beta release of the CLI https://github.com/passbolt/go-passbolt-cli/releases/tag/v0.4.0-beta.2

Testing would be much appreciated!

speatzle avatar Sep 09 '25 13:09 speatzle

@speatzle using that beta version, I get the following errors over and over (I think for pretty much all of the records):

Decryping Resources [1872/1933] █████████████████████████████████████  97% | 10s
Skipping Export of Resource e0709ab8-0344-41d6-89ef-91f3f8c20b2c  Because of: Get Resource e0709ab8-0344-41d6-89ef-91f3f8c20b2c: Validate Secret Data: Compiling Json Schema: "file:///home/user/secret.json#" is not valid against metaschema: jsonschema validation failed with 'https://json-schema.org/draft/2020-12/schema#'
- at '': 'allOf' failed
  - at '/properties/custom_fields': 'allOf' failed
    - at '/properties/custom_fields/properties/items': 'allOf' failed
      - at '/properties/custom_fields/properties/items/items': 'allOf' failed
        - at '/properties/custom_fields/properties/items/items/properties/secret_key': got array, want boolean or object

ravage84 avatar Sep 09 '25 14:09 ravage84

@ravage84 Sorry, i think i missed your notification. Do those errors still happen with the latest build? You can get it from the Artifacts here: https://github.com/passbolt/go-passbolt-cli/actions/runs/19029616319

speatzle avatar Nov 07 '25 19:11 speatzle

This time it went through. Though, only about half of the secrets needed to be exported since we did some clean-up of very old, expired secrets in the mean time.

passbolt export keepass -f backup_2025-11-11.kdbx
Enter Password:
Enter Keepass Password:
Getting Resources...
Decryping Resources [1049/1049] ███████████████████████████████████ 100% | 1m31s
Done

From the file name alone, it was the same binary I used last time.

Thus, I can't really fully verify whether anything was fixed.

🤷‍♂️ 🙇‍♂️

ravage84 avatar Nov 11 '25 16:11 ravage84

That looks good, there where fixes in the validation area since then.

speatzle avatar Nov 11 '25 16:11 speatzle

Hello, i just tried the new build against our V5 passbolt server and i still get the same jsonschema error:

~/Downloads/go-passbolt-cli-artifacts/go-passbolt-cli_darwin_arm64/passbolt get resource --id f31ec17e-acbc-4507-96ca-46024be2d93c -j     
Error: Getting Resource: Validate Secret Data: Compiling Json Schema: "file://xxxxxxxxx/secret.json#" is not valid against metaschema: jsonschema validation failed with 'https://json-schema.org/draft/2020-12/schema#'
- at '': 'allOf' failed
  - at '/properties/custom_fields': 'allOf' failed
    - at '/properties/custom_fields/properties/items': 'allOf' failed
      - at '/properties/custom_fields/properties/items/items': 'allOf' failed
        - at '/properties/custom_fields/properties/items/items/properties/secret_key': got array, want boolean or object

This is really annoying, i have to enter the same passwords 350 times a day when applying playbooks or whatnot

earzur avatar Nov 14 '25 09:11 earzur

are there some options to set in the server to ensure compatibility with go-passbolt-cli ?

earzur avatar Nov 14 '25 09:11 earzur

@earzur Is your Passbolt server up to date? There where some issues with the Server side Schema in older versions.

speatzle avatar Nov 14 '25 10:11 speatzle

@earzur Is your Passbolt server up to date? There where some issues with the Server side Schema in older versions.

We're running 5.4.1, upgrade to latest is planned for this weekend, i'ill report if fixes the issue.

Thanks

earzur avatar Nov 14 '25 15:11 earzur