Use context bindings in REST API (and other custom queries)

[mjashanks]

We should add the ability to use the following bindings in all custom queries.

• {{ Current User....}} : logged here already https://github.com/Budibase/budibase/issues/3884
• Access headers on incoming requests: https://github.com/Budibase/budibase/discussions/3934
• Properties from the SSO token: https://github.com/Budibase/budibase/issues/3827
• Eventually, we would like to put environmental things in here - e.g. config and secrets for dev/staging/prod

I thought it would be better to collect these into one issue.

We would need to think about how these work in automations - i.e. is the "User" available, or does it makes sense?

Commented on here: https://github.com/Budibase/budibase/discussions/4120#discussioncomment-2007650

[mike12345567]

Just a point, the first one using Current User in datasources/automations could be quite difficult as they are server-side, its possible we could enrich it from the user session for datasources, but automations are ran outside the context of a request so won't have a current user for context.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[thehowl]

Bumping this so that stalebot doesn't close this, as I think it's quite important to have.

While writing this comment, I've realised many others have commented on the importance of a similar feature on #3827. So I'll just collapse it here:

Explanation of use case and argument on the importance of this feature

So is there currently no way of ensuring some kind of "workspace" access to a data source server-side? My use case is building an app for a service which is being sold to different clients. We want each client to have access to their data exclusively, without being able to see data of other clients. However, from what I understand, this is currently only possible to send as a parameter of a query to a data source? So client-side?

It seems to me that the only current way would be to duplicate the app and making sure that the requests to the backend always specify the correct workspace, which is of course undesirable as I don't want to maintain an app for each client and have to export/import/change the datasource queries each time. I'd like to make the case that this is quite critical in any scenario where we want access to data to be secured in some kind of way which is outside of the "access level" system, because otherwise the queries to datasources are just waiting for any techie to come by and realise that access is unsecured server-side.

I understand you have your own priorities and roadmap for the project where this has not yet come up as an important issue -- judging from the fact that this issue has been open for the past 6 months and was about to be closed in five days' time. I'd be willing to work on this to address at least the {{ Current User }} variable in the REST datasource - which is my own itch - and properties from the SSO token which I see has been highly requested and without having seen the source code I'd intuitively think is a trivial change to add if I'm also adding data of the current user.

However, as this is a bit more complicated than the other issue I've created, I'd like to have a few pointers to the source code where I should be looking at if possible? And any security considerations I should keep in mind developing this?

Thanks.

[thehowl]

Bumping this as this may have been missed since it was during the Easter break.

[mike12345567]

Hi @thehowl - we understand this is an important issue - it is something that we will look at - we just have a lot on our plate at the moment!

In terms of the source code/where to look for this feature, it is likely not something that can be implemented by an external contributor as it touches on automations. If an automation were to run a query that utilises the {{ Current User }} information it could not work, as automations aren't ran within a users context. Some thought is needed here as to how to protect against this scenario as it could be very confusing for the user as to why it is failing. As for the actual implementation for data sources/queries - the flow would go something like packages/server/src/api/controllers/query/index.js then updating the preview and execute API endpoints to extract the user details (if available) and passing them down as parameters to the query runner thread. For SSO information - this is another larger piece of work, right now when a user logs in via an SSO strategy we don't record any information about the SSO connection to the session/context - this information isn't readily available at the minute, it is just used in the auth services.

[thehowl]

Hey Mike, sure I did not want that to come across as offensive. (The original comment was written after quite a bit of frustration and thinking "How can this be in a 1.0 without this?" - I understand that like all software dev teams you must have a lot on your plate) Thank you for your reply! I'll make a note of what you wrote and will make an attempt at making the changes - as otherwise Budibase is working very well for our usecase. Have a nice day!

[SantoshSrinivas79]

Hi @mike12345567 ... It's been several months this has been open and wondering if any of the issues are in the roadmap.

I believe the REST API datasource was being revamped. Can {{ Current User }} atleast in the REST datasource be supported? (I understand automations may not be able to support it, which is perhaps ok in many use-cases)

[shogunpurple]

Hi @SantoshSrinivas79 - we have scheduled the SSO token work for June - so it will be done over the next few weeks. We could support {{ Current User }} in the REST datasource - while we are doing the SSO token work, we can add that binding too. As Michael says, it's not going to be possible for use in automations, but the REST connector is where it is more important.

Thanks!

[SantoshSrinivas79]

Can't wait! The REST datasource binding alone would be so useful. (does it have to be included alongside SSO tokens? .... just thinking if the REST connector can be modified separately and much more quickly)

[danielbangs]

If anyone needs a temporary (and quite hacky) workaround:

1. Create a new REST data source, and add a GET query with the url https://your.budibase.domain/api/self (e.g. https://example.budibase.app/api/self for the cloud version). Call it getSelf (or whatever)
2. Add the following headers:
   • x-budibase-api-version: 1
   • x-budibase-app-id: app_example_abc1234... (your production appId)
   • x-budibase-type: client
   • cookie: {{cookie}}
3. Add a binding:
   • cookie: budibase:oidc:config=... (your current cookie, which you can get by typing document.cookie in the Chrome console)
4. Click Send next to the request URL, and you should see your user details, including an oauth2 object, with an accessToken property
5. IMPORTANT: remove your cookie from the cookie binding and leave it blank, before clicking Save (lest you expose your personal token to unauthenticated users)
6. On any page with > Basic access, insert a new Data Provider with Data source getSelf. Call it Get Self Data Provider.
7. Click the gear icon to edit the query bindings, then click the lightning bolt next to the cookie binding
8. Choose JavaScript, enter return document.cookie and click Save, then Save again.

Now for any queries that need an auth token, you just need to add an authToken binding, and nest its Data Provider within the Get Self Data Provider, then edit its query binding to the JavaScript expression return $("Get Self Data Provider.Rows")[0].oauth2.accessToken (you could wrap it in a try/catch to avoid errors for email/password users).

Obviously this uses a private API so can't be recommended. But in our case it allows us to press on and build awesome things with Budibase now, instead of next month. And oh the awesome things we will build, I love this thing!

[shogunpurple]

@danielbangs I'm seriously impressed 😃 We can't wait to get native support for this shipped in the next release!

[mlomax]

@shogunpurple do you have any details of what the "native support" will allow us to do? We're currently stalled using Budibase because of the need to use the SSO token in the Rest database. We're certainly going to have a look at the work around but would like to understand what the June release will offer. I posted on #3827 but received no reposnse. Many thanks

[niggydotcom]

@danielbangs are you using AWS? I'm trying to use this method using Azure but my OIDC authentication URL is https://login.microsoftonline.com/..../v2.0/.well-known/openid-configuration and my authorize URL is https://login.microsoftonline.com/.../oauth2/v2.0/authorize Same appid and orgid, obviously. Is it possible to get this to work - ie sign in to Budi and use the same token to authorize the API?

[danielbangs]

@niggydotcom we're using Auth0 for authentication - by default with both Auth0 and Google Cloud Identity (the only two I've played with, so may be different on Azure) the oauth2 token that Budibase stores only works against the userinfo_endpoint defined in your OIDC config, and it's not a valid JWT. So to use it as is, you'd need to call the userinfo_endpoint from your API server with the users oauth2 token, then proceed with authorisation using the user you get back.

A much better solution though is to set an oauth audience (aud) in your OIDC config, or with your provider (here's how we do it on Auth0). When you set an audience, you get back a JWT which you can validate and decode server side, and will include the users info.

jwt.io is a good place to test if the token you have is a JWT or not, and what audience its for (you'll probably be using the RS256 algorithm).

[Kanary159357]

@danielbangs Your way works really welll.... Thx... But how can I refresh the authToken that I get from private API?

I'm looking for way to do this but I could not found it.....

[mlomax]

@danielbangs @deanhannigan is there any budibase documentation out there that will help us? we don't have the development skills to work this out and chose budibase because it was low-code.

We can log on to Budibase with openid connect We have an Oauth2 working in Insomnia to produce an access token in the format our external provider requires but can't translate it into budibase Since I can't get that I don't know if there's any option to use the openid OAuth token If I manually put that Insomnia access token into budibase and specify it as a dynamic variable I don't seem to be able to use it.

Anything you can help with would be most gratefully received. If we can't get to our external provider then I think we'll have to rethink using Budi.

Thanks Mary

[deanhannigan]

Hi @mlomax, thanks for getting in touch. If you have successfully been able to configure your Budibase to authenticate with Insomnia, then you should be good to go.

The current users OAuth2 token is made available for use within queries with the following binding: {{ Current User.OAuthToken }}.

The standard method to utilise an OAuth2 token to authorize your query requests is to:

1. Navigate to your REST datasource, open the desired query and select the header tab.
2. Add a new header called Authorization to your query.
3. Set the value of the header to Bearer {{ Current User.OAuthToken }}

In most scenarios, this should be all that is required to utilise the current users access token when making requests.

All available bindings for the query properties can be viewed by clicking the lightning icon next to the property you wish to edit. The user bindings now display in the binding drawer.

Some documentation on how to use the binding as a global header for your all your REST datasource queries can be found here. https://docs.budibase.com/docs/rest-authentication#using-sso-tokens-in-rest-requests

Further discussion on the feature is also available here. https://github.com/Budibase/budibase/pull/6315

If your need further assistance with this, I'm happy to help.

[mlomax]

Hi Dean

Thanks for your quick response. We’ve tried what you suggest but I think our external provider specifies a token format so it just spits out Bearer {{ Current User.OAuthToken }}

What I’ve managed to do in Insomnia is produce the token that the external provider requires. I can’t work out how to get Budi to replicate it - either using the SSO info or by reentering it again.

@.***

I’ve also got dynamic variables that just don’t appear in the binding drawer (Some of them aren’t correct but I can’t seem to delete them!)

@.***

I’d like to use schema_value_var in a different query (I’m temporarily pasting in the access token from Insomnia to get prove the concept of linked/variable queries) but all I’m getting as Bindings are the Current User fields.

Am I missing something on where a dynamic variable shows up? It’s a dynamic variable against a rest query that I want to use as a binding in a different rest.

@.***

Any help or insight you have would be most gratefully received

Thank you

Mary

Mary Lomax Business Change Manager T: 07592 104863 [cid:twitter_32x32_e1e3e76b-cc3f-4c41-85a7-e8b60d3ff9fa.png] @weare_pure [cid:facebook_32x32_f9e8fd91-aa50-4816-918c-9663104e27c2.png]https://www.facebook.com/PureInnovations/ /pureinnovations www.Pureinnovations.co.ukhttp://www.pureinnovations.co.uk/ Please note my usual working days are Tuesday - Thursday If you need help on Eclipse please contact @.*** [cid:newlogo3_c079c830-3375-4b01-861c-bdbded8f7685.jpg] [cid:chaspurplelogo_1681fe54-3124-4e4c-a858-b78831ac8690.png]

[cid:disabilitycondfidentlogo_97482504-fe03-41db-bf83-969412b6773e.png] [cid:cyberessentials_certificationmark_colour_8a7d5167-12ca-41c2-87da-439903a57446.png]

From: deanhannigan @.> Sent: 05 August 2022 09:54 To: Budibase/budibase @.> Cc: Mary Lomax @.>; Mention @.> Subject: Re: [Budibase/budibase] Use context bindings in REST API (and other custom queries) (Issue #4139)

Hi @mlomaxhttps://github.com/mlomax, thanks for getting in touch. If you have successfully been able to configure your Budibase to authenticate with Insomnia, then you should be good to go.

The current users OAuth2 token is made available for use within queries with the following binding: {{ Current User.OAuthToken }}.

The standard method to utilise an OAuth2 token to authorize your query requests is to:

1. Navigate to your REST datasource, open the desired query and select the header tab.
2. Add a new header called Authorization to your query.
3. Set the value of the header to Bearer {{ Current User.OAuthToken }}

In most scenarios, this should be all that is required to utilise the current users access token when making requests.

All available bindings for the query properties can be viewed by clicking the lightning icon next to the property you wish to edit. The user bindings now display in the binding drawer. [https://user-images.githubusercontent.com/5913006/183034612-e3d514f5-7cc4-4564-95f7-6176357cc46d.png]https://user-images.githubusercontent.com/5913006/183034612-e3d514f5-7cc4-4564-95f7-6176357cc46d.png

Some documentation on how to use the binding as a global header for your all your REST datasource queries can be found here. https://docs.budibase.com/docs/rest-authentication#using-sso-tokens-in-rest-requests

Further discussion on the feature is also available here. #6315https://github.com/Budibase/budibase/pull/6315

If your need further assistance with this, I'm happy to help.

— Reply to this email directly, view it on GitHubhttps://github.com/Budibase/budibase/issues/4139#issuecomment-1206210123, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AZHUJAOHUIJA5FLZ622SM63VXTJC5ANCNFSM5MPD4YSA. You are receiving this because you were mentioned.Message ID: @.@.>>

[deanhannigan]

Hi Mary,

Apologies, I may have misunderstood your workflow here. A few questions:

• Does your initial request require your OIDC authorization to request the token?
• In Budibase, are you getting a successful response back from your endpoint with the token visible in the response body?
• What structure does the value of the schema_value_var variable have in your query?
• What is the structure of the request that uses the schema_value_var variable? Where in the request do you need the token to be placed?

You may have uncovered a bug with the bindings drawer, I also cannot see my dynamic bindings in the list. However, I can still use my dynamic bindings if I add them manually e.g if you were to place {{ schema_value_var }} in your request it will use the value cached there. Give this a try and see if your token makes its way into your request.

To delete a dynamic variable, navigate to the query in which you wish to remove it and click send. This will open up the existing dynamic variables for deletion again.

Let me know if you make any progress with this and I'll have a look at the bindings drawer issue.

Thanks

[mlomax]

Hi Dean

With regard to the variable bindings - I can see them in the REST query that I created them from but nowhere else.

I also don't seem to be able to pull any dynamic variables manually into any other query - The schema_value_var is just simply a number or text string. The REST query I'm currently sending it to simply sends the same value back {{ data.0.[value] }} I can send you a video as it's not confidential data.

Mary

[deanhannigan]

Hi Mary, a video would be brilliant, thank you!

[mlomax]

Hi @deanhannigan

Here's a video of trying to use the dynamic variable https://user-images.githubusercontent.com/105858177/183638721-0a669c9b-f623-4f4a-80fa-0f222ad7bf34.mp4

This is the original query with the binding drawer

And here's the duplicated query with the binding drawer - and no dynamic variable available

Mary

[mlomax]

And in reposnse to your other questions

Does your initial request require your OIDC authorization to request the token? In Budibase, are you getting a successful response back from your endpoint with the token visible in the response body?

I log in to Budi using Azure and it authenticates fine, logs me in and there is a CurrentUser.OAuthtoken available. The difficulty happens when I try and send that OAuth to my external API as they are specifying certain fields in their JWT.

This is the jwt from the OAuthtoken

Using Insomnia I can specify some different things to be in the token and this is what I need

as you can see the jwt are different.

So we tried building a query to replicate this Insomnia query (so that we could then specify the token as a dynamic variable) but we haven't succeeded. Do you think this should be possible - either using the OAuthtoken or by requesting the user to re-login?

Thanks Mary

[deanhannigan]

Hi Mary, many thanks for the video, it was very helpful!

In your second query could you attempt to use the dynamic variable directly without the token binding? If you update the code url parameter to use {{ schema_value_var2 }} this should map the dynamic parameter correctly.

Let me know if this changes the behaviour for your request.

The contents from your first token seems to be a V1 access token. Has the API been explicitly configured to return a V2 response? I believe there is some configuration required in Azure to set this.

Have you specified any particular scopes in order to return the V2 response in your second example?

[mlomax]

Hi Dean I've update the code parameter and it has managed to bring something through - except I think it must be caching somewhere This is now my value of query 1 It used to be 27 (see above) And this is the newly bound query 2

[mlomax]

Interesting on the token. This is our login on budibase and we will check azure endpoints although I thought we'd followed the bud docs for set up.

There is a specific permission and scope in Azure for the second example. You can see it on the Insomnia request too.

Mary

[mlomax]

Hi Our Azure looks like the budi docs with both the v1 and v2 endpoints I'm not aware of anywhere else to look. Mary

[mlomax]

there is definitely a lag on the dynamic variable. I've changed query1 to "something" but query 2 is now showing "fdsd4689dhf133g2r56e4r6a" my value from earlier

[deanhannigan]

Hi Mary, a dynamic variable will only update when the query utilising it fails.

Regarding your token issue, I believe this is indeed related to the auth scopes being passed to Azure. My understanding is that the scope for your custom API must also be included during authentication to receive the V2 auth token.

We don't currently allow users to customise their authentication scopes but work on adding that functionality will begin shortly.

This issue can be tracked here: https://github.com/Budibase/budibase/issues/7267

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity.