WebHooks against specific builds are not shown in the project webhooks list

[YoungjaeKim]

Expected Behavior

Shown properly

Current Behavior

If we select as below, webhooks are only assigned to certain buids within a project (eg, not all builds).

Then, the webhooks not shown in Project view as below.

If we select All projects and Sub-projects, Then, the webhooks are shown properly as below.

Steps to Reproduce (for bugs)

1. Create webhook in a Project (it would be good to have builds on itself and to have Subprojects and their builds)
2. Apply template partially to Project.
3. In Project view/Webhook tab, it did not shown.
4. Apply template to All by checking all chekboxes.
5. In Project view/Webhook tab, it shows as expected.

Your Environment

• tcWebHooks Version:
• TeamCity Version: 2018.1.1 (build 58406)
• TeamCity server Operating System: Ubuntu 16 Docker
• Are you using a WebHook Template?: 1.1-alpha17.235.354

Example Configuration (xml)

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" uuid="218fa8dd-7042-496b-963a-e299d1d748a7" xsi:noNamespaceSchemaLocation="http://www.jetbrains.com/teamcity/schemas/2018.1/project-config.xsd">
  <name>Display Client App</name>
  <description>Android Apps</description>
  <parameters>
    <param name="BuildFlavor" value="" />
    <param name="app.package.id" value="" />
    <param name="build.version" value="0" />
    <param name="images.android" value="android/build:av26_bt27.0.3_gr4.4_ndr16b_file" />
    <param name="wiki.parentid" value="024" />
  </parameters>
  <project-extensions>
    <extension id="PROJECT_EXT_5" type="IssueTracker">
      <parameters>
        <param name="host" value="http://sample.com" />
        <param name="idPrefix" value="DIS" />
        <param name="name" value="YouTrack" />
        <param name="secure:password" value="zcd05d" />
        <param name="type" value="youtrack" />
        <param name="username" value="bot" />
      </parameters>
    </extension>
  </project-extensions>
  <cleanup />
</project>

[netwolfuk]

I think the best way to solve this is to get rid of the "WebHooks configured for projectName > buildName" and instead just list the Enabled Builds like it does on the WebHooks tab of the Project Settings, and the WebHook edit page.

eg,

The other feature on the roadmap (but quite a way down the list) is tags, and it could be possible to list the builds a webhook is associated with via a tag. I'm not sure that scales very well to the All builds & Sub-Projects one though.

[netwolfuk]

I've implemented a mix of both solutions. As you can see, on the build tab the webhooks associated to the build are listed separately (and excluded from the project webhook list).

But viewing the project tab, shows the all the webhooks, even if they are only relevant to a specific build or set of builds.

And the list of builds that the webhook is associated with is available when hovering over the build count (if a specific set of builds is selected)

[netwolfuk]

Also, these are now available when editing a project, as it seems more logical to find them there.

[netwolfuk]

I'll post here when the next alpha 1.2.0 is released.

[jan0sch]

Hi,

maybe this is related. If I specify a webhook and/or webhook parameters on a build level then they are not associated to the build but to the parent project. The same happens if I define them on a sub-project (they are associated with the parent project).

Project A
  Project A1 <-- Webhooks or parameters defined here are shown at Project A
    Build B1 <-- Webhooks or parameters defined here are shown at Project A1
  Project A2
Project B
  Build B2 <-- Webhooks or parameters defined here are shown at Project B
  Build B3

[netwolfuk]

Which tcWebhooks version are you running?

[netwolfuk]

BTW. internally all the webhook configurations are stored against a project.. It's not possible to store this sort of config against a build. That is why they appear at the project level.

I'll look into what you say about the project details being stored against the parent project though.

[jan0sch]

I am using 1.2.0-alpha.8-build.425

It would be cool though if the custom webhook parameters could be stored on the build config level. Or is this not possible?

[netwolfuk]

If I understand the use case, you want the same variable to resolve to different values for different build configs and want to reference them in the template?

If so, I think it's possible to create a TeamCity parameter in the build configuration and use that.

[netwolfuk]

If you're using the velocity template engine, it's also possible to define a JSON map as the text of a parameter, and then use jsontool to convert it to a Java Map in the template.

If you used the buildTypeId as the key in the map you could reference the value in your template. I've done this before to map git usernames to slack handles.

[jan0sch]

Thanks for the info.

If so, I think it's possible to create a TeamCity parameter in the build configuration and use that.

As mentioned the problem is also with parameters. They as well get attached to the parent. :-(

Also I noticed that deleted webhooks are still shown in the overview list even when I cannot click on them. The global settings page for webhooks show the correct count though, so maybe that is some caching issue? To be more concrete the still shown ones are the hooks I defined which were then wrongly attached to the parent project. Hooks I define on a "top level" project are fine.

[netwolfuk]

Thanks for all the feedback. I'll setup that version on a machine somewhere and test. I've broken my current dev environment as I'm in the middle of a huge refactor to reduce a lot of duplication in the JavaScript UI code

[netwolfuk]

Just to confirm. Did you create the parameter in teamcity, or tcwebhooks?

I was intending you to create it on the TeamCity Build Config itself, rather than in the project webhook parameters.

[jan0sch]

Just to confirm. Did you create the parameter in teamcity, or tcwebhooks?

Oh, I missed that one, thanks. But how can I reference them in the webhook? I need some of them to construct the proper URL. After creating a build parameter foo and webhook.foo which references (with %foo%) using it in the URL like ${foo} just gives an UNRESOLVED upon testing.

[netwolfuk]

if I understand correctly, you want to:

1. Define a per build config parameter.
2. Force resolve it early.
3. Refer to it in the URL.

I think this might work (untested).

1. Create a TeamCIty build config parameter called foo.
2. Create a tcWebHooks Project WebHook parameter named bar, choose force resolve, and set the value as %foo%
3. Add ${bar} to your URL.

When the force resolve code runs, it takes an instance of the build (SRunningBuild), and asks it to resolve the parameter. It will find the build config paramater %foo% and attempt to resolve it into a webhook variable named bar which can be referenced as ${bar}. I hope that makes sense, and I hope it works. :-)

Edit: Sorry about the above screenshot. The presence of the webhook. was misleading. I was testing it for something else. webhook prefixes are historical, and not really needed since template support was added.

[jan0sch]

This works but not for nested projects and also not if we have multiple build configurations under one project which whould need different variables. Because the variables are also assigned to the parent project we can only define a variable once.

We could circumvent the build issue problem by wrapping the build configs into sub projects but for that the issue of everything being attached to the parent project instead of the current one remains a problem.

Last but not least: Many thanks for your work and response.

[netwolfuk]

I must be missing something. I'm sorry I'm not understanding correctly.

If you define bar once for all projects, eg on the root project, and define foo as a parameter on every build you need it on.

This should work because bar is resolved at runtime against the instance of the build that is running.

[jan0sch]

Here is a workflow with screenshots.

1. edit webhooks of a project -> define a webhook
2. define a variable on a project because that will be used across all build configs/sub projects (CommitHash)
3. edit webhooks of a build configuration -> define a variable
4. The build config variable is not shown on the build config but on the parent project instead. :-(

Does this help? I am prepared to record a screencast. ;-)

[netwolfuk]

Yep. Thanks. If I understand correctly, the owner should be unique per build config.

If so, create it as a TeamCity configuration parameter, and remove it from the Webhook parameters.

tcWebhooks will still find it with the name ${owner}

[netwolfuk]

Hi @jan0sch.

I just wanted to say thank you for taking the time to submit your comments. For me it seems obvious how this works because I wrote all internals, but I'll try to explain a bit more.

The WebHooks and WebHook Parameters are stored against the project in which they live. This is simply because that's the easiest place to store them (and the only place they could be stored when I first started the project over a decade ago). Creating a webhook against a specific build configuration is really just a webhook in a project, but with just the specific build configuration explicitly selected.

The confusion comes from the way they are displayed in the UI. I present them as if they are webhooks for a specific build, and at the time this bug was first raised, they were hidden when you viewed the project webhooks. This now been fixed in the version you are using and you can see webhooks in the project, even if they only execute for a specific set of build configurations.

However, when viewing webhooks for a specific build, this shows webhooks that are specifically configured for this build config. There could in fact be many webhooks that this build also "inherits" from the project it is in, and any parent projects that have "sub-projects" enabled but they are not currently shown.

For this reason, I have been working on the search page (/webhooks/search.html). It's still a work in progress, but internally there is now a central service that manages and indexes the webhooks across all projects (although they are still stored per project). This allows searching by a build configuration, and the search walks the tree and finds any webhook that is configured to run on this build. They could be specific to that build, in the associated project, or in any parent project.

I feel like this gives a more visible indication of what webhooks will actually be executed for a specific build. It is this indexed list of webhooks that also allows viewing the webhooks for parent projects when you view the webhooks tab on the Project Configuration (named WebHooks and Templates).

The search page tries to be clever and not show you the full URL of webhooks that live in a project for which you don't have edit access. For example, if there was a parent project with a webhook for https://hooks.slack.com/hooks/my-secret-token but your user does not have edit access to that project, the URL you will see in the search results will be slack.com.

Also, webhooks in projects that you don't have view access to will not appear in the search. This is more relevant when you search by template or tag because the results could be in any project across the whole teamcity instance.

Currently, the edit webhook code (jQuery and javascript), is very specific to the edit page only. The large refactor I am currently working through, is to extract all the relevant components and enable re-use so that the UI code (eg, that dialog and all the fetching and saving, etc) can be used in other pages. This will allow editing of webhooks and parameters directly from the project configuration page, and also the search results page.

[netwolfuk]

BTW, here are some example searches.

• /webhooks/search.html?buildTypeId=TcPlugins_TcWebhooksGitHub
• /webhooks/search.html?projectId=TcPlugins
• /webhooks/search.html?tag=enabled
• /webhooks/search.html?templateId=gitea
• /webhooks/search.html?show=all

[jan0sch]

Defining the parameters as regular ones on the build configuration works. :tada: :fireworks:

Again thank you very much for your work and time! :heart:

[netwolfuk]

You're welcome. I'm glad you got it all working.