[BUG] Fails to download jars

[ventsyv]

⚠️ Important Notice

Please differentiate the bug

---

🐛 Bug Report:

Describe the bug

When we run the generator for the first time expecting to get resolved from the PATH (openapi-generator-cli version) it hangs and after a while gives the following error:

node_modules/@openapitools/openapi-generator-cli/main.js:1045
            error.response.data.on('data', data => this.logger.log(data.toString('utf8')));
                                ^
TypeError: error.response.data.on is not a function
    at VersionManagerService.printResponseError (/GMSS/node_modules/@openapitools/openapi-generator-cli/main.js:1045:33)
    at /GMSS/node_modules/@openapitools/openapi-generator-cli/main.js:924:18
    at /GMSS/node_modules/rxjs/dist/cjs/internal/operators/catchError.js:13:51
    at OperatorSubscriber._this._error (/GMSS/node_modules/rxjs/dist/cjs/internal/operators/OperatorSubscriber.js:43:21)
    at Subscriber.error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)
    at Subscriber.error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)
    at Subscriber.error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/GMSS/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)

Steps to Reproduce

Install like this

cd ~
mkdir testdir
cd ~/testdir/
npm install -g npm@latest
npm install @openapitools/openapi-generator-cli

sudo ln -s /home/$USER/testdir/node_modules/@openapitools/openapi-generator-cli/main.js /usr/bin/openapi-generator-cli

sudo chmod 755 node_modules/@openapitools/openapi-generator-cli/main.js

Now try running:

~/testdir/$ which openapi-generator-cli
/usr/bin/openapi-generator-cli
~/testdir$ openapi-generator-cli version
Unable to query repository, because of: "Request failed with status code 504"

Response:
date Thu, 12 Jan 2023 17:45:56 GMT
content-type text/html
content-length 160
connection keep-alive
server nginx

/home/user/testdir/node_modules/@openapitools/openapi-generator-cli/main.js:1045
            error.response.data.on('data', data => this.logger.log(data.toString('utf8')));
                                ^

TypeError: error.response.data.on is not a function
    at VersionManagerService.printResponseError (/home/user/testdir/node_modules/@openapitools/openapi-generator-cli/main.js:1045:33)
    at /home/user/testdir/node_modules/@openapitools/openapi-generator-cli/main.js:924:18
    at /home/user/testdir/node_modules/rxjs/dist/cjs/internal/operators/catchError.js:13:51
    at OperatorSubscriber._this._error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/operators/OperatorSubscriber.js:43:21)
    at Subscriber.error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)
    at Subscriber.error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)
    at Subscriber.error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18)
    at Subscriber._error (/home/user/testdir/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)

Strangely enough, changing the directory seems to work

cd ..
~$ openapi-generator-cli version
Download 6.2.1 ...
Downloaded 6.2.1
Did set selected version to 6.2.1
6.2.1

You can break it again:

cd testdir
rm -rf node_modules/@openapitools/openapi-generator-cli/versions/
openapi-generator-cli version

The above will give you the same error

Expected behavior

openapi-generator-cli version
Download 6.2.1 ...
Downloaded 6.2.1
Did set selected version to 6.2.1
6.2.1

Screenshots

If applicable, add screenshots to help explain your problem.

Operation System (please complete the following information):

• OS: Ubuntu
• Version: 22

Package System (please complete the following information):

• Node.js v19.3.0
• npm - 9.2.0

Additional context

Add any other context about the problem here.

[JoePotentier]

Seeing the same. Blocking deployments for my team.

[saivamsilolla]

Even we are blocked

[risotto-master]

Blocked as well when installing through NPM per instructions from https://openapi-generator.tech/docs/installation/

[githubeto]

Is there an alternative plan?

[githubeto]

I confirmed that it works fine when using 'openapitools/openapi-generator-cli' docker images.

[PSchubertMHP]

Hi, we're having the same issue. Also blocks deployments. Seems like the repo is offline.

[talesAtTilt]

Same here too.

[DavidBiesack]

We use

npx openapi-generator-cli version-manager set 5.4 stable

as there are bugs in openapi-generator 6 that affect us, but now we cannot download 5.4 due to this outage

Unable to query repository, because of: "Request failed with status code 502"

Response:

server awselb/2.0
date Fri, 13 Jan 2023 13:08:10 GMT
content-type text/html
content-length 122
connection close

and openapi-generator uses 6.2.1 instead of 5.4

As a workaround, use a static copy of the 5.4.0 openapi-generator jar and access that directly via

--custom-generator=/path/to/openapi-generator-5.4.0.jar

One can download the needed jar from maven, such as https://repo1.maven.org/maven2/org/openapitools/openapi-generator/5.4.0/openapi-generator-5.4.0.jar or https://repo1.maven.org/maven2/org/openapitools/openapi-generator/6.2.1/openapi-generator-6.2.1.jar

[ventsyv]

Creating an openapitools.json file in the current working directory with the following content seems to fix it for me:

{
  "$schema": "./node_modules/@openapitools/openapi-generator-cli/config.schema.json",
  "spaces": 2,
  "generator-cli": {
    "version": "6.2.1"
  }
}

It appears that sometimes the file is not generated for some reason. When it is being generated everything works fine. I can't find any obvious reason (permissions, etc) for the file not to be generated. Another strange thing is that the schema path is not valid from the current working directory - so I can't see a reason why it would fix the issues, but it seems that it does. I tried 4 - 5 times and every time I added the file (different working dirs) it works like a charm.

This started happening in the last couple of days.

[dougkeen]

I looked into this a bit, and it seems related to the not-so-great uptime of search.maven.org:

This will cause failures when:

1. Defaulting to using latest openapi-generator version, since that search.maven.org endpoint is called to figure out what the latest published version is.
2. Explicitly specifying the openapi-generator version, since the set method on the version manager calls the search.maven.org endpoint to validate the version param.

In other words, you're in trouble in all cases if search.maven.org is down.

It seems like validation via the search step while doing a openapi-generator-cli version-manager set <versionTags...> may be causing more problems than it solves, given the coupling to search.maven.org.

[dougkeen]

For contrast, here's the uptime of the CDN from where the openapi-generator jars are actually downloaded:

It would be great if there was a way to specify the exact openapi-generator version to use, and skip the query step so your usage of the tool is decoupled from search.maven.org.

[GustavoKatel]

the situation is aggravated because the command does not return a non-zero code, so it does not stop in CI and it silently fails in some cases.

[alanwguo]

We use

npx openapi-generator-cli version-manager set 5.4 stable

as there are bugs in openapi-generator 6 that affect us, but now we cannot download 5.4 due to this outage

Unable to query repository, because of: "Request failed with status code 502"

Response:

server awselb/2.0
date Fri, 13 Jan 2023 13:08:10 GMT
content-type text/html
content-length 122
connection close

and openapi-generator uses 6.2.1 instead of 5.4

As a workaround, use a static copy of the 5.4.0 openapi-generator jar and access that directly via

--custom-generator=/path/to/openapi-generator-5.4.0.jar

One can download the needed jar from maven, such as https://repo1.maven.org/maven2/org/openapitools/openapi-generator/5.4.0/openapi-generator-5.4.0.jar or https://repo1.maven.org/maven2/org/openapitools/openapi-generator/6.2.1/openapi-generator-6.2.1.jar

@DavidBiesack,

The --custom-generator param doesn't seem to work for me. It still tries to use the version set via npx openapi-generator-cli version-manager set 5.4 stable. Is this the way you used it?

$ wget https://repo1.maven.org/maven2/org/openapitools/openapi-generator/4.3.0/openapi-generator-4.3.0.jar -O /tmp/openapi-generator-4.3.0.jar
$ npx openapi-generator-cli generate \
		--custom-generator=/tmp/openapi-generator-4.3.0.jar \
		-i ../../backend/gen/openapi.json -g typescript-fetch \
		-o src/gen/openapi -c openapiconfig.yaml

[enceem]

we are encountering same issue "Request failed with status code 502" in our corporate network. I think that it is a common scenario to haven nexus configured as mirror for maven. So i would suggest: If maven is installed then maven should be used for the download the jar file instead of hard-coded url. mvn dependency:copy -Dartifact=org.openapitools:openapi-generator-cli:6.2.1 -DoutputDirectory=./

[DavidBiesack]

alanwguo yes, that is how I'm using it. I'm using this instead of calling npx openapi-generator-cli version-manager set 5.4 stable. The other solution, creating the openapitools.json tool, also worked for me. I use

openapiGeneratorVersion=5.4.0
cat <<EOF >| openapitools.json
{
  "\$schema": "./node_modules/@openapitools/openapi-generator-cli/config.schema.json",
  "spaces": 2,
  "generator-cli": {
    "version": "${openapiGeneratorVersion}"
  }
}
EOF
That is, I think the `openapi-generator-cli` will use the version in the `openapitools.json` file, 
but calling `openapi-generator-cli version-manager set 5.4 stable` will still force it to
lookup versions in Maven repo to verify it, so to fix this issue, you need to not use that command.

[LKreutzer]

Seeing the same issue https://github.com/magma/magma/issues/14860, but for us it seems to be flaky, see workflow runs. It started ca. 1 week ago.

[stephanecodes]

I have the same problem when validating an openapi schema in an Azure pipeline. The --custom-generator option is only supported for the generate command, not for the validate command

On the other hand, it is possible to specify a storageDir which openapi-generator-cli uses to download the JAR and check if it is already downloaded. The solution I have chosen is to store a recent version of the openapi-generator-x.y.z.jar JAR in my Git repository and use it if search.maven.org is not responding.

The openapitools.json file is not committed in my repo, it forces openapi-generator-cli to download the latest version available at each build.

Here is my validate:openapi script in package.json

{
  "scripts": {
    "validate:openapi": "openapi-generator-cli validate -i ../path/to/openapi.yaml",
  },
}

And the part of the CI shell script in the pipeline that validates the openapi schema

# If search.maven.org is down the openapi-generator-cli-x.y.z.jar 
# won't be downloaded and the command will fail
# The issue is described at https://github.com/OpenAPITools/openapi-generator-cli/issues/680

set +e;
# The next command will automatically check and download the latest version of openapi-generator-cli
# Let's capture the command exit status. 
# If exit status is not zero then fallback on a static version of the jar (may not be the latest version)
npm run validate:openapi
exit_status=$?

# Exit immediately if a command returns a non-zero status
set -e;

if (( exit_status != 0 )); then
    echo ""
    echo "Failed to fetch version of openapi-generator-cli (exit status=$exit_status)"
    # Use the last modified openapi-generator jar stored in static dir as fallback
    fallback_jar_version=$(basename -s .jar $(ls -t ./static/openapi-generator-cli/*.jar | head -n1))
    echo "Using fallback library openapi-generator-cli-$fallback_jar_version.jar"
    # Create or replace openapitools.json file and force the use of the fallback version

cat <<EOF >| openapitools.json
{
  "\$schema": "./node_modules/@openapitools/openapi-generator-cli/config.schema.json",
  "spaces": 2,
  "generator-cli": {
    "storageDir": "./static/openapi-generator-cli",
    "version": "${fallback_jar_version}"
  }
}
EOF

    # Retry
    npm run validate:openapi
fi

# Remove openapitools.json
rm -f openapitools.json

[talesAtTilt]

It is back.

openapi-generator-cli version-manager set 6.4.0 Unable to query repository, because of: "Request failed with status code 502" Response: date Tue, 27 Jun 2023 14:53:24 GMT content-type text/html content-length 150 connection close server nginx /usr/local/lib/node_modules/@openapitools/openapi-generator-cli/main.js:1036 error.response.data.on('data', data => this.logger.log(data.toString('utf8'))); ^ TypeError: error.response.data.on is not a function at VersionManagerService.printResponseError (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/main.js:1036:33) at /usr/local/lib/node_modules/@openapitools/openapi-generator-cli/main.js:915:18 at /usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/operators/catchError.js:13:51 at OperatorSubscriber._this._error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/operators/OperatorSubscriber.js:43:21) at OperatorSubscriber.Subscriber.error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18) at OperatorSubscriber.Subscriber._error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30) at OperatorSubscriber.Subscriber.error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18) at OperatorSubscriber.Subscriber._error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30) at OperatorSubscriber.Subscriber.error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:60:18) at OperatorSubscriber.Subscriber._error (/usr/local/lib/node_modules/@openapitools/openapi-generator-cli/node_modules/rxjs/dist/cjs/internal/Subscriber.js:84:30)

[arnaud-deprez]

When going to https://search.maven.org, I get this:

It is probably related...

[swcm-mnestler]

We're seeing this issue very often in the last few days, a rerun of the build mostly works. Would it make sense to implement a basic retry of the maven search call in the openapi-generator-cli?

[rettgerst]

happening again today. this project needs to stop depending on search.maven.org, which is described as "legacy" by its maintainers and has shown a history of being unreliable.

the offending function is here.

I'm unfamiliar with the java ecosystem - would if be bad practice to bypass the search and make a request directly to https://repo1.maven.org/maven2/org/openapitools/openapi-generator/ to download the jar?

[swcm-mnestler]

@rettgerst what helped us was running npx openapi-generator-cli version directly before the generation - it will download the version that is set in the openapitools.json without a maven search lookup

[heyline]

Anyone struggling with this, I chose to submit the jar package to git-lfs and manually copy the jar to the corresponding folder after npm install, so that I can skip the download. Here is a gradle task that can help me copy the file there.

tasks.register<Copy>("openapi-generate-preprocessing") {
  from(layout.projectDirectory.file("lib/openapi-generator-cli/7.5.0.jar"))
  into(layout.projectDirectory.dir("node_modules/@openapitools/openapi-generator-cli/versions"))
}

At the same time, I also have openapitools.json in the project root directory to fix the openapi-generator-cli version.

{
  "$schema": "./node_modules/@openapitools/openapi-generator-cli/config.schema.json",
  "spaces": 2,
  "generator-cli": {
    "version": "7.5.0"
  }
}

[nabsource]

{ "$schema": "./node_modules/@openapitools/openapi-generator-cli/config.schema.json", "spaces": 2, "generator-cli": { "version": "6.2.1" } }

Thank you so much, this fixed our issue as well