DMPRoadmap
Add API V2 from DMPonline
PR for issue #3586
This issue branch uses the doorkeeper gem to implement the Authorization Code Grant Flow (ACGF) part of the v2 API. (The Client Credentials Flow (CCF) part of the v2 API (which is what the org-admins will use) will be implemented separately in a future piece of work.)
| 1 Error | |
|---|---|
| :no_entry_sign: | Please include a CHANGELOG entry. |
| 2 Warnings | |
|---|---|
| :warning: | There are code changes, but no corresponding tests. Please include tests if this PR introduces any modifications in behavior. \n Ignore this warning if the PR ONLY contains translation.io synced updates. |
| :warning: | This PR is too big! Consider breaking it down into smaller PRs. |
Generated by :no_entry_sign: Danger
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Thank you for this PR.
During today's DMPRoadmap Monthly all-team meeting I mentioned how there are also related DMPTool v2 API test files. Would you like me to add them to this PR?
I'm seeing a lot of rubocop errors in here. This might be a difference in the configuration of Rubocop between the Roadmap repo and DMP Online. Let me know how you'd want to approach this. If possible, I could start a PR attempting to fix these.
I'm seeing a lot of rubocop errors in here. This might be a difference in the configuration of Rubocop between the Roadmap repo and DMP Online. Let me know how you'd want to approach this. If possible, I could start a PR attempting to fix these.
@momo3404 apologies we missed this. I can do this and push it back next week, if that's ok.
These two docs are more for developers support. It might be useful [Authorization_Code_Grant_Flow.pdf](https://github.com/user-attachments/files/23827 doorkeeper_gem_for_OAuth_2.0_flows.pdf 178/Authorization_Code_Grant_Flow.pdf)
These two docs are more for developers support. It might be useful [Authorization_Code_Grant_Flow.pdf](https://github.com/user-attachments/files/23827 doorkeeper_gem_for_OAuth_2.0_flows.pdf 178/Authorization_Code_Grant_Flow.pdf)
Awesome, thank you. Sorry, the links got a little bungled up in the comment. :P https://github.com/user-attachments/files/23827180/doorkeeper_gem_for_OAuth_2.0_flows.pdf works. Could you just provide the Authorization_Code_Grant_Flow.pdf link again?
These two docs are more for developers support. It might be useful [Authorization_Code_Grant_Flow.pdf](https://github.com/user-attachments/files/23827 doorkeeper_gem_for_OAuth_2.0_flows.pdf 178/Authorization_Code_Grant_Flow.pdf)
Awesome, thank you. Sorry, the links got a little bungled up in the comment. :P https://github.com/user-attachments/files/23827180/doorkeeper_gem_for_OAuth_2.0_flows.pdf works. Could you just provide the Authorization_Code_Grant_Flow.pdf link again?
@aaronskiba Here you go: Authorization_Code_Grant_Flow.pdf
Let us know if this doesn't work.
I'm encountering the following when I test the GET /api/v2/heartbeat endpoint (I can also see that the endpoint is returning a 500 for DMPOnline):
$ curl http://127.0.0.1:3000/api/v2/heartbeat
NoMethodError at /api/v2/heartbeat
==================================
undefined method `name' for nil:NilClass
> To access an interactive console with this error, point your browser to: /__better_errors
app/views/api/v2/_standard_response.json.jbuilder, line 18
----------------------------------------------------------
``` ruby
13 json.ignore_nil!
14
15 json.server @server
16 json.source "#{request.method} #{request.path}"
17 json.time Time.now.to_formatted_s(:iso8601)
> 18 json.client @client.name
19 json.code response.status
20 json.message Rack::Utils::HTTP_STATUS_CODES[response.status]
21
22 if response.status == 200
23
App backtrace
- app/views/api/v2/_standard_response.json.jbuilder:18:in `_app_views_api_v___standard_response_json_jbuilder__1455696549825964062_81440'
- app/views/api/v2/error.json.jbuilder:3:in `_app_views_api_v__error_json_jbuilder__3591388735008983784_81420'
- app/controllers/api/v2/base_api_controller.rb:69:in `handle_internal_server_error'
- app/controllers/api/v2/base_api_controller.rb:58:in `handle_exception'
Full backtrace ...
Please let me know if I've done something wrong here, but I keep getting 302s when executing the GET command from the Authorization Code Grant Flow.
curl -i -X GET "http://127.0.0.1:3000/oauth/authorize?response_type=code&client_id=[my_client_id]&redirect_uri=[redirect_uri]" HTTP/1.1 302 Found x-frame-options: SAMEORIGIN x-xss-protection: 0 x-content-type-options: nosniff x-permitted-cross-domain-policies: none referrer-policy: strict-origin-when-cross-origin location: http://127.0.0.1:3000/users/sign_in content-type: text/html; charset=utf-8 cache-control: no-cache
But the command still worked, because after calling Doorkeeper::AccessGrant.first I found my authorization which I used for the POST command, and that worked perfectly. So is this a known error, or something I'm missing? I was also signed in when making the request.
I'm encountering the following when I test the
GET /api/v2/heartbeatendpoint (I can also see that the endpoint is returning a 500 for DMPOnline):$ curl http://127.0.0.1:3000/api/v2/heartbeat NoMethodError at /api/v2/heartbeat ================================== undefined method `name' for nil:NilClass > To access an interactive console with this error, point your browser to: /__better_errors app/views/api/v2/_standard_response.json.jbuilder, line 18 ---------------------------------------------------------- ``` ruby 13 json.ignore_nil! 14 15 json.server @server 16 json.source "#{request.method} #{request.path}" 17 json.time Time.now.to_formatted_s(:iso8601) > 18 json.client @client.name 19 json.code response.status 20 json.message Rack::Utils::HTTP_STATUS_CODES[response.status] 21 22 if response.status == 200 23App backtrace
* app/views/api/v2/_standard_response.json.jbuilder:18:in `_app_views_api_v___standard_response_json_jbuilder__1455696549825964062_81440' * app/views/api/v2/error.json.jbuilder:3:in `_app_views_api_v__error_json_jbuilder__3591388735008983784_81420' * app/controllers/api/v2/base_api_controller.rb:69:in `handle_internal_server_error' * app/controllers/api/v2/base_api_controller.rb:58:in `handle_exception'Full backtrace ...
@aaronskiba this is fixed and should be working now
Full backtrace ...
@aaronskiba this is fixed and should be working now
Cool. I can get the 200 via the heartbeat whether I'm signed in or signed in now.
Unfortunately, I think the fix may have introduced other bugs. I am testing the following curl request:
$ curl -H "Authorization: Bearer ${BEARER_TOKEN}" -H "Accept: application/json" "http://127.0.0.1:3000/api/v2/plans"
This works and returns the expected JSON response on https://github.com/DMPRoadmap/roadmap/pull/3589, which is not up-to-date with these lastest heartbeat changes. However, when I test on this PR, I get the following response:
{
"source": "GET /api/v2/plans",
"time": "2025-12-22T12:03:00-07:00",
"code": 500,
"message": [
"There was a problem in the server."
]
I also get the following in the rails console:
Processing by Api::V2::PlansController#index as JSON
Doorkeeper::AccessToken Load (1.3ms) SELECT "oauth_access_tokens".* FROM "oauth_access_tokens" WHERE "oauth_access_tokens"."token" = $1 LIMIT $2 [["token", "lp5znDvJAOZndbJRWAZ1eeZ_5NR8ZZs_2raswR-YsgQ"], ["LIMIT", 1]]
Exception message: undefined method `include?' for nil:NilClass
raise Pundit::NotAuthorizedError unless @scopes.include?('read')
^^^^^^^^^
Rendering api/v2/error.json.jbuilder
Rendered api/v2/_standard_response.json.jbuilder (Duration: 0.7ms | Allocations: 400)
Rendered api/v2/error.json.jbuilder (Duration: 1.8ms | Allocations: 728)
Completed 500 Internal Server Error in 10ms (Views: 3.8ms | ActiveRecord: 0.9ms | Allocations: 7697)
This must be related to @scopes = doorkeeper_token.scopes.to_a if doorkeeper_token being removed in commit 3645cae2f7dcfa36437dc618f46d12a132c2c9d5
I am also looking at the @application = ApplicationService.application_name change in app/controllers/api/v2/base_api_controller.rb. I like this renaming because it is consistent with app/controllers/api/v1/base_api_controller.rb. However, json.server @server still exists in app/views/api/v2/_standard_response.json.jbuilder. Changing it to json.application @application should fix things and again align with the api/v1 approach.
The assignments to @client and @resource_owner were removed in commit 3645cae2f7dcfa36437dc618f46d12a132c2c9d5. This will need to be addressed as well.
Full backtrace ...
@aaronskiba this is fixed and should be working now
Cool. I can get the 200 via the heartbeat whether I'm signed in or signed in now.
Unfortunately, I think the fix may have introduced other bugs. I am testing the following curl request:
$ curl -H "Authorization: Bearer ${BEARER_TOKEN}" -H "Accept: application/json" "http://127.0.0.1:3000/api/v2/plans"This works and returns the expected JSON response on #3589, which is not up-to-date with these lastest heartbeat changes. However, when I test on this PR, I get the following response:
{ "source": "GET /api/v2/plans", "time": "2025-12-22T12:03:00-07:00", "code": 500, "message": [ "There was a problem in the server." ]I also get the following in the rails console:
Processing by Api::V2::PlansController#index as JSON Doorkeeper::AccessToken Load (1.3ms) SELECT "oauth_access_tokens".* FROM "oauth_access_tokens" WHERE "oauth_access_tokens"."token" = $1 LIMIT $2 [["token", "lp5znDvJAOZndbJRWAZ1eeZ_5NR8ZZs_2raswR-YsgQ"], ["LIMIT", 1]] Exception message: undefined method `include?' for nil:NilClass raise Pundit::NotAuthorizedError unless @scopes.include?('read') ^^^^^^^^^ Rendering api/v2/error.json.jbuilder Rendered api/v2/_standard_response.json.jbuilder (Duration: 0.7ms | Allocations: 400) Rendered api/v2/error.json.jbuilder (Duration: 1.8ms | Allocations: 728) Completed 500 Internal Server Error in 10ms (Views: 3.8ms | ActiveRecord: 0.9ms | Allocations: 7697)This must be related to
@scopes = doorkeeper_token.scopes.to_a if doorkeeper_tokenbeing removed in commit 3645caeI am also looking at the
@application = ApplicationService.application_namechange inapp/controllers/api/v2/base_api_controller.rb. I like this renaming because it is consistent withapp/controllers/api/v1/base_api_controller.rb. However,json.server @serverstill exists inapp/views/api/v2/_standard_response.json.jbuilder. Changing it tojson.application @applicationshould fix things and again align with the api/v1 approach.The assignments to
@clientand@resource_ownerwere removed in commit 3645cae. This will need to be addressed as well.
Hey @aaronskiba, so sorry about this. I have changed @server to @application, to keep it consistent with V1. I've also put back @scopes and @resource_owner. That was removed unintentionally, so thanks for spotting that. You might still get other errors because we haven't fixed and tested them all, but I think this might be the last change I push before the long break to avoid causing more errors :D
Also, please feel free to commit any changes you feel are necessary (some of which you have already added as comments in this PR).
It looks like we have to sync the translations to get the Dookeeper views rendering properly (maybe this is only an issue in dev mode?). We usually have yaml translations disabled, but they'll have to be enabled in this instance.
https://github.com/DMPRoadmap/roadmap/commit/8befc57475126d2e2cc3f9fa11230ff8a3cd611b does exactly that and it looks like it fixes the rendering (see screenshots below). If you'd like, I could cherrypick the commit onto this PR?
BEFORE
AFTER