jenkinsci
[JENKINS-50154] Webhook delivery failed as X-Hub-Signature did not match calculated
Our users were reporting that several PR's across a number of different repositories were not being built by Jenkins. Our Webhooks are setup manually and we use Organization Folders as documented here and Freestyle/Pipeline Jobs as documented here
We noticed that Github was showing a number of failures in both our organization and in individual repositories with messages like so:
"Content-Type" content="text/html;charset=utf-8"/>Error 400 Provided signature [139e89df4530d9827a6a8f32c1ee28564fff178f] did not match to calculated HTTP ERROR 400
Problem accessing /github-webhook/. Reason:
Provided signature [139e89df4530d9827a6a8f32c1ee28564fff178f] did not match to calculated
"http://eclipse.org/jetty">Powered by Jetty:// 9.4.z-SNAPSHOT
This appeared to be limited to three repositories at first but it soon spread to other repositories. It's also intermittent and also tied to specific pull requests (i.e once a PR has had a failed webhook, subsequent updates to that PR will also fail).
We had upgraded from Jenkins 2.64.3 and Github Plugin 1.27.0 and migrated to new instances in AWS over the weekend and had wondered if this may have made the issue worse - we've seen similar failures before now but less frequently.
So far we have tried:
- Rolling back to Github Plugin 1.28.1 which completely broke webhooks
- Clearing the Github Plugin Cache (setting to 0, and back to a positive integer).
- Changing the secret used for the webhook on Github and Jenkins
- Changing the hook content type to `application/json`
- Deleting and re-creating the hook
The only workaround we have right now has been to:
- Give our bot user Admin permissions on the affected repository
- Enable 'Manage Hooks' and re-create hooks on all repositories
- The created hook on the repository is, so far, delivering with 100% success
Originally reported by dave_tucker, imported from: Webhook delivery failed as X-Hub-Signature did not match calculated
- assignee:
lanwen
- status: Open
- priority: Major
- component(s): github-plugin
- resolution: Unresolved
- votes: 5
- watchers: 14
- imported: 2025-12-08
Raw content of original issue
Our users were reporting that several PR's across a number of different repositories were not being built by Jenkins. Our Webhooks are setup manually and we use Organization Folders as documented here and Freestyle/Pipeline Jobs as documented here
We noticed that Github was showing a number of failures in both our organization and in individual repositories with messages like so:
<html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 400 Provided signature [139e89df4530d9827a6a8f32c1ee28564fff178f] did not match to calculated</title> </head> <body><h2>HTTP ERROR 400</h2> <p>Problem accessing /github-webhook/. Reason: <pre> Provided signature [139e89df4530d9827a6a8f32c1ee28564fff178f] did not match to calculated</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.z-SNAPSHOT</a><hr/> </body> </html>This appeared to be limited to three repositories at first but it soon spread to other repositories. It's also intermittent and also tied to specific pull requests (i.e once a PR has had a failed webhook, subsequent updates to that PR will also fail).
We had upgraded from Jenkins 2.64.3 and Github Plugin 1.27.0 and migrated to new instances in AWS over the weekend and had wondered if this may have made the issue worse - we've seen similar failures before now but less frequently.
So far we have tried:
- Rolling back to Github Plugin 1.28.1 which completely broke webhooks
- Clearing the Github Plugin Cache (setting to 0, and back to a positive integer).
- Changing the secret used for the webhook on Github and Jenkins
- Changing the hook content type to `application/json`
- Deleting and re-creating the hook
The only workaround we have right now has been to:
- Give our bot user Admin permissions on the affected repository
- Enable 'Manage Hooks' and re-create hooks on all repositories
- The created hook on the repository is, so far, delivering with 100% success
- environment:
Jenkins 2.89.4, Github Plugin 1.29.0
1 attachment
recampbell:
- Original comment link
Raw content of original comment:
Dave, do you have any log messages with the category org.jenkinsci.plugins.github.webhook?
Dave, do you have any log messages with the category org.jenkinsci.plugins.github.webhook?
dave_tucker:
- Original comment link
Raw content of original comment:
Yes we see lots of instances of:
Mar 14, 2018 3:12:58 PM FINEST org.jenkinsci.plugins.github.webhook.RequirePostWithGHHookPayload$Processor shouldProvideValidSignatureTrying to verify sign from header sha1=8e290f17632932bed4138eb632dd2e761f529bcfMar 14, 2018 3:12:58 PM FINEST org.jenkinsci.plugins.github.webhook.GHWebhookSignature matchesSignature: calculated=deff2fe4d0173b1f2c39d086fa6cb6ee4f85b1fd provided=8e290f17632932bed4138eb632dd2e761f529bcf
Yes we see lots of instances of:
Mar 14, 2018 3:12:58 PM FINEST org.jenkinsci.plugins.github.webhook.RequirePostWithGHHookPayload$Processor shouldProvideValidSignatureTrying to verify sign from header sha1=8e290f17632932bed4138eb632dd2e761f529bcfMar 14, 2018 3:12:58 PM FINEST org.jenkinsci.plugins.github.webhook.GHWebhookSignature matchesSignature: calculated=deff2fe4d0173b1f2c39d086fa6cb6ee4f85b1fd provided=8e290f17632932bed4138eb632dd2e761f529bcf
nusnewob:
- Original comment link
Raw content of original comment:
We are seeing the same problem, but it wasn't tied to specific PR/branch, completely random and then just disappear.
We are seeing the same problem, but it wasn't tied to specific PR/branch, completely random and then just disappear.
joshm:
- Original comment link
Raw content of original comment:
I'm also seeing this issue this morning but only for a specific repo (1 out of 20+). Looking at the webhook history, this was working fine and then this same repo starting showing failed webhooks as of 2018-08-09 14:36:05.
Note that we do not have repo-level hooks, just a single org-level hook. All other builds are working fine!
I'm also seeing this issue this morning but only for a specific repo (1 out of 20+). Looking at the webhook history, this was working fine and then this same repo starting showing failed webhooks as of 2018-08-09 14:36:05.
Note that we do not have repo-level hooks, just a single org-level hook. All other builds are working fine!
joshm:
- Original comment link
Raw content of original comment:
BUMP! This is getting worse for us. We have more failed webhooks than succeeded now. We recently moved to multi-branch config, and now most of our webhook requests from GitHub are showing the "Provided signature [...] did not match to calculated" message.
BUMP! This is getting worse for us. We have more failed webhooks than succeeded now. We recently moved to multi-branch config, and now most of our webhook requests from GitHub are showing the "Provided signature [...] did not match to calculated" message.
medianick:
- Original comment link
Raw content of original comment:
In my case I was getting this randomly – for some requests, not others – but it seems to have been the result of choosing "application/x-www-form-urlencoded" as the Content Type on the GitHub side. Once I switched this to "application/json", the webhook payloads started consistently delivering successfully.
In my case I was getting this randomly – for some requests, not others – but it seems to have been the result of choosing "application/x-www-form-urlencoded" as the Content Type on the GitHub side. Once I switched this to "application/json", the webhook payloads started consistently delivering successfully.
lmat:
- Original comment link
Raw content of original comment:
All webhooks fail with 400 from Jenkins with a message like this.
<tr><th>URI:</th><td>/github-webhook/</td></tr> <tr><th>STATUS:</th><td>400</td></tr> <tr><th>MESSAGE:</th><td>Provided signature [2ea5940025571f3314b15e7d1283a3c258f70912] did not match to calculated</td></tr> <tr><th>SERVLET:</th><td>Stapler</td></tr>
All webhooks fail with 400 from Jenkins with a message like this.
URI: /github-webhook/ STATUS: 400 MESSAGE: Provided signature [2ea5940025571f3314b15e7d1283a3c258f70912] did not match to calculated SERVLET: Stapler
oxygenxo:
- Original comment link
Raw content of original comment:
We have two repositories in our organization, for one webhook is delivered successfully every time, for other we always receive "Provided signature did not match calculated" error.
I've taken a look at the code and this line is looking suspicious to me https://github.com/jenkinsci/github-plugin/blob/830219a7e696f9274acf3f8ced9e1c5ea865d3a3/src/main/java/org/jenkinsci/plugins/github/webhook/GHWebhookSignature.java#L57
I've added Unicode character (™) to the description on the first repo the issue was reproduced on it.
We have two repositories in our organization, for one webhook is delivered successfully every time, for other we always receive "Provided signature did not match calculated" error.
I've taken a look at the code and this line is looking suspicious to me https://github.com/jenkinsci/github-plugin/blob/830219a7e696f9274acf3f8ced9e1c5ea865d3a3/src/main/java/org/jenkinsci/plugins/github/webhook/GHWebhookSignature.java#L57
I've added Unicode character (™) to the description on the first repo the issue was reproduced on it.
oxygenxo:
- Original comment link
Raw content of original comment:
I've created a PR with possible solution to this https://github.com/jenkinsci/github-plugin/pull/242
Reviewers are appreciated
I've created a PR with possible solution to this https://github.com/jenkinsci/github-plugin/pull/242
Reviewers are appreciated
enys:
- Original comment link
Raw content of original comment:
It looks like since this PR was merged and released as version 1.33 there are quite a few people with triggers failing.
<title>Error 400 Provided signature [c0180c723bbe3ddb266be493f5c9c1977896ea15] did not match to calculated</title> </head>
It looks like since this PR was merged and released as version 1.33 there are quite a few people with triggers failing.
Error 400 Provided signature [c0180c723bbe3ddb266be493f5c9c1977896ea15] did not match to calculated
oxygenxo:
- Original comment link
Raw content of original comment:
enys could you please provide an example JSON payload for me to investigate this?
enys could you please provide an example JSON payload for me to investigate this?
enys:
- Original comment link
Raw content of original comment:
Sure, will prepare an annonimised payload.
I've more or less drilled it down to 2 types.
A commit containing unicode char, appearing in the payload as :
"message": "Ë << unicode",
Or merging a PR with multi line description :
"Message\r\n\r\n message continued"
Sure, will prepare an annonimised payload.
I've more or less drilled it down to 2 types.
A commit containing unicode char, appearing in the payload as :
"message": "Ë ,
Or merging a PR with multi line description :
"Message\r\n\r\n message continued"
enys:
- Original comment link
Raw content of original comment:
I can confirm that above payloads cause the issue.
I can confirm that above payloads cause the issue.
vinutha_garimella:
- Original comment link
Raw content of original comment:
this issue exits in git plugin version 1.33.0
this issue exits in git plugin version 1.33.0
oxygenxo:
- Original comment link
Raw content of original comment:
enys, vinutha_garimella thanks for your reports. I've created a PR to revert this changes I've made https://github.com/jenkinsci/github-plugin/pull/246 I apologize to everyone who suffered from this change - looks like the bug is not in the plugin. I've tried to setup webhooks on Jenkins instance behind corporate firewall using this instruction https://www.jenkins.io/blog/2019/01/07/webhook-firewalls/, but I've taken pysmee (https://github.com/Akrog/pysmee) instead of official smee.io client written in JavaScript. If the bug exists, it is in pysmee - official client passes webhook payload correctly.
enys, vinutha_garimella thanks for your reports. I've created a PR to revert this changes I've made https://github.com/jenkinsci/github-plugin/pull/246
I apologize to everyone who suffered from this change - looks like the bug is not in the plugin.
I've tried to setup webhooks on Jenkins instance behind corporate firewall using this instruction https://www.jenkins.io/blog/2019/01/07/webhook-firewalls/, but I've taken pysmee (https://github.com/Akrog/pysmee) instead of official smee.io client written in JavaScript. If the bug exists, it is in pysmee - official client passes webhook payload correctly.
enys:
- Original comment link
Raw content of original comment:
oxygenxo Thanks for all the information, and trying to fix a bug to start with!
The only thing that would have been useful is an entry in the plugin changelog for 1.33 release.
Thanks,
oxygenxo Thanks for all the information, and trying to fix a bug to start with!
The only thing that would have been useful is an entry in the plugin changelog for 1.33 release.
Thanks,
nh2:
- Original comment link
Raw content of original comment:
I am still suffering from this problem.
It occurs when I force-push a commit that has the message:
mycommit*
Here the thing that breaks it seems to be the star in the commit message. In the Github hook JSON body this is:
... "commits": [ ... "message": "mycommit*",
If I use a + instead, of the *** the issue does not occur and Jenkins accepts the webhook.
The star seems to be forbidden anywhere in the message, e.g. on a new line as part of a Markdown-style list.
Equally breaking are commit messages like
a*
or even just
*
as the commit message which results in JSON being
"message": "*",
I am still suffering from this problem.
It occurs when I force-push a commit that has the message:
mycommit*
Here the thing that breaks it seems to be the star in the commit message. In the Github hook JSON body this is:
... "commits": [ ... "message": "mycommit*",
If I use a + instead, of the *** the issue does not occur and Jenkins accepts the webhook.
The star seems to be forbidden anywhere in the message, e.g. on a new line as part of a Markdown-style list.
Equally breaking are commit messages like
a*
or even just
*
as the commit message which results in JSON being
"message": "*",
nh2:
- Original comment link
Raw content of original comment:
I think I found a solution for my case:
In the "Manage webhook" settings on Github, I had Content-type as application/x-www-form-urlencoded. Changing it to application/json fixes the issue with the stars in commit messages.
I'm struggling to find documentation that tells me which "Content type" I should be using for the Github plugin.
On https://support.cloudbees.com/hc/en-us/articles/224543927-GitHub-Integration-Webhooks#manualmode it says
Content type should be set up as application/json (application/x-www-form-urlencoded for PR events in Freestyle Jobs)
So this suggests that either is OK, which is evidently not the case if you use stars in your commit messages.
Could a plugin developer clarify?
I think I found a solution for my case:
In the "Manage webhook" settings on Github, I had Content-type as application/x-www-form-urlencoded. Changing it to application/json fixes the issue with the stars in commit messages.
I'm struggling to find documentation that tells me which "Content type" I should be using for the Github plugin.
On https://support.cloudbees.com/hc/en-us/articles/224543927-GitHub-Integration-Webhooks#manualmode it says
Content type should be set up as application/json (application/x-www-form-urlencoded for PR events in Freestyle Jobs)
So this suggests that either is OK, which is evidently not the case if you use stars in your commit messages.
Could a plugin developer clarify?