Update typing and API support to 15.x

[jdalrymple]

Summary

1. Updating library to match latest 14.x API
2. Updating typing to use explicit return types on all exposed functions
3. Removing 'version' as a Resource parameter. v4 will be hardcoded
4. Begin adding version support information with each release.

Supported Versions: 14.9 Excluding Alpha APIS (Debian group/project distributions, Dependencies)

fixes #2084 fixes #1897 fixes #2311 fixes #2296 fixes #2267 fixes #2295 fixes #2172 fixes #2276 fixes #2319 fixes #1449 fixes #1906 fixes #2072 fixes #2322 closes #1592 closes #2463 closes #2450

UPDATES - Updating unit tests and primary exports

[jdalrymple]

Update: Currently at the Issue Stats API. :smiling_face_with_tear: still a lot to go haha

[jdalrymple]

Update: At the Maven API

[jdalrymple]

Update: At the Notes API!

[jdalrymple]

Progress! At the protected environments API

[lunodan]

Hello! What's the deal with this MR? Do you need any help?

[jdalrymple]

@lunodan Im going to review the status of this over the next week and follow up. I finally have a bit of time to jump back on this, but honestly forget where I was in the PR. I know i was running through the testing, but where i stopped is beyond me haha.

[lunodan]

Amazing, thank you! I'm pretty excited for epic links :)

On Fri, 5 Aug 2022 at 13:21, Justin Dalrymple @.***> wrote:

@lunodan https://github.com/lunodan Im going to review the status of this over the next week and follow up. I finally have a bit of time to jump back on this, but honestly forget where I was in the PR. I know i was running through the testing, but where i stopped is beyond me haha.

— Reply to this email directly, view it on GitHub https://github.com/jdalrymple/gitbeaker/pull/2258#issuecomment-1206384064, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUWO7WNRTJ42VPP4VZPSLGLVXUBMVANCNFSM5JGH3SWQ . You are receiving this because you were mentioned.Message ID: @.***>

--

Dan Millwright-HughesEngineering Director London - Europe https://www.luno.com/en/ We're hiring! See our latest roles here https://www.luno.com/careers. https://www.glassdoor.co.uk/Overview/Working-at-Luno-EI_IE1027878.11,15.htm

[jdalrymple]

Update with where I'm at! Testing out the file streaming functions, but am running into issues with the native fetch function. For what ever reason when i try to stream the response into a file it throws me a http error. :thinking: So still trying to debug that!

[salim-runsafe]

@jdalrymple is there anything that contributors can help with to get this across the line? We've been using gitbeaker successfully for awhile, but soon need GroupHooks, GroupVariables, and other features that are in this MR.

[jdalrymple]

@salim-runsafe Hey! Yea actually. Though it may seem kinda trivial haha, i havent been able to get it to work and its blocking me atm. Basically, im trying to figure out how to use native nodejs fetch to stream to a file from Gitlab. For example:

import { createWriteStream } from 'node:fs'
import { Readable } from 'stream';

const response = await fetch('https://gitlab.com/api/v4/projects/21015560/repository/archive.zip', {
  method: 'GET',
  headers: {
    'private-token': [some token],
  },
})


Readable.fromWeb(response.body)
  .pipe(createWriteStream('test.zip'))

For what ever reason the file saved is a corrupt zip file and i cant seem to figure out why. The goal here is to test that a proper stream is returned so i can ensure the library is actually returning streams when requested.

[salim-runsafe]

I think I can help with that @jdalrymple. Are you targeting a particular version of Node?

[jdalrymple]

Yup, NodeJS 18+ since its the latest LTS and it supports the fetch API. The major benefit for the fetch API is that the library no longer needs to depend on platform specific request API (right now we use got for node and ky for browser), so the browser and nodejs distributions will just be the same distribution which reduces complexity, upkeep, testing and package size. Everything else seems to work, but than i ran into this stream issue

[salim-runsafe]

@jdalrymple on the current node:latest Docker image I use this:

import { createWriteStream } from 'node:fs'
import { Readable } from 'stream';

const response = await fetch('https://gitlab.com/api/v4/projects/21015560/repository/archive.zip', {
  method: 'GET',
})


const stream = Readable.from(response.body);
const writeable = createWriteStream('test.zip');
stream.pipe(writeable);

test.zip is a valid file, but its contents are {"message":"406 Not Acceptable"}, which would be from an Accept or Accept-Encoding header mismatch. I'm hoping that if you drop in the stream updates it will work in your environment, but if it doesn't I can keep working on it.

[jdalrymple]

I've tinkered with the Accept header, but didnt have much time to keep testing through it. I didnt think of the "Accept-Encoding" header though, ill give it a look!

Update: I did a blanket test with accept: * and accept-encoding:*, but no luck. Fetch much be setting some header value but i cant seem to determine what it is. Testing the same url in other requesters like the browser, curl, insomnia etc works as expected, so its definitely linked to some value internally being set by default

Update2: Tinkering with request bin. Follow up when i know more!

Update3: Using: mode: 'same-origin', for fetch solved the problem. Ill continue testing later today and update the progress information above!

[salim-runsafe]

I'm able to download the zip just fine with postman, curl, python, and powershell; it's just node giving this issue.

This is the code that causes it to return a 406 Unacceptable and I'm about to start digging into what is causing it.

Oh, I just saw that you updated your comment with success, great! I'll post this anyway for posterity.

[sebsgr]

@jdalrymple Hey, do you have an approximate schedule for this release? We also need some features from this release for our project.

[jdalrymple]

@jdalrymple Hey, do you have an approximate schedule for this release? We also need some features from this release for our project.

As soon as i get these passing: https://gitlab.com/jdalrymple/gitbeaker/-/pipelines :crossed_fingers: Its my main focus for the week

[jdalrymple]

Update: Down to one test failure in the pipeline. Basically when the cli code is compiled, the json import assertion line is stripped causing it to crash.

// From this
import API_MAP from '@gitbeaker/core/map.json' assert { type: 'json' }; // eslint-disable-line import/no-unresolved

// To this
import API_MAP from '@gitbeaker/core/map.json'

Not sure why its being stripped. I've tested with other non-local json files and it worked fine. I think it may have something to do with the path being rewritten in the tsconfig to point to a build file

{
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "paths": {
      "@gitbeaker/core/map.json": ["../core/dist/map.json"], //<<<Here
      "@gitbeaker/*": ["../*/src"],
    }
  },
  "include": ["./**/*.ts"]
}

but im not sure. trying to narrow it down to a repeatable sample

[jdalrymple]

Debug Update: Looks related to the imports @ symbol. If i remove it, it works fine. Got to put together a sample project

[jdalrymple]

WOO, ready for review. If anyone has the time to pick at this, all feedback is welcome! Ill release an rc version for live usage for the next month or so to iron out any kinks that may be found during initial usage.

[jdalrymple]

Still need to iron out some issues with the release pipeline, but rc versions are now available on npm. Ill be working on this on and off for the next month or so to fix up any issues i notice along the way in addition to streamlining the end of this ci pipeline

[salim-runsafe]

@jdalrymple I've found a bug with at least ProjectAccessTokens and GroupAccessTokens, but it looks like it impacts other resources as well.

Every request 404s because the URL is not being constructed properly. For instance, when I attempt to create a ProjectAccessToken for project 12345678 it attempts to reach https://gitlab.com/api/v4/projects12345678/access_tokens with no / between projects and the project ID.

I think the proper fix is to build the url with a / included between ${prefixUrl} and ${endpoint}, but I'm not sure if this would have repercussions I'm not aware of:

const url = `${prefixUrl}/${endpoint}${searchParams ? `?${searchParams}` : ""}`

[jdalrymple]

Hmm good catch. I should change it such that the slash isn't important 🤔

[salim-runsafe]

Hmm good catch. I should change it such that the slash isn't important 🤔

You could use the built-in URL.

[jdalrymple]

Hmm good catch. I should change it such that the slash isn't important thinking

You could use the built-in URL.

Exactly what i was thinking! Implemented and being deployed now!

[salim-runsafe]

@jdalrymple at least with ProjectAccessToken, that drops projects from the URL.

https://gitlab.com/api/v4/12345678/access_tokens should be https://gitlab.com/api/v4/projects/12345678/access_tokens

[jdalrymple]

@jdalrymple at least with ProjectAccessToken, that drops projects from the URL.

https://gitlab.com/api/v4/12345678/access_tokens should be https://gitlab.com/api/v4/projects/12345678/access_tokens

100p, something is off. Ill give it a look! Thanks for following up

[jdalrymple]

Apparently, the URL instance doesnt like the input:

let a = new URL('21015560/access_tokens','https://gitlab.com/api/v4/projects')

a.toString()

//"https://gitlab.com/api/v4/21015560/access_tokens"

​ Gonna reread the docs to see where i went wrong. They have a similar example, but im not sure...why haha

UPDATE: I need to append a slash at the end of the base url to make it capture it correctly. sigh

[salim-runsafe]

@jdalrymple it looks like we didn't get a build with the latest fix because of a pipeline failure. Is that something you can address?

[jdalrymple]

@jdalrymple it looks like we didn't get a build with the latest fix because of a pipeline failure. Is that something you can address?

Ill sort it!