Make Jenkinsfile Runner accessible for non-Java users

[shadycuz]

Hey everyone,

I'm a big fan of the Jenkinsfile-runner, I have been for years when I accidentally stumbled upon it in early development. Unfortunately I have never been able to get it to work. I revisit often and from time to time I see updated instructions and so I try again and again but never am I successful. Well not on my own, see I have actually used the project for a year now but only because I steal working solutions, like the dockerfile from here.

I'm sure the project is easy to build and run if your very intimate with either Jenkins/Java or both. But I feel this project would be much more successful if you could make it so anyone can run the Jenkinsfile-runner. I think a good first step would be to provide instructions to build a Jenkinsfile-runner based on the Jenkins docker image. Lots of people are already using this image at 500million pulls. This would make it extremely easy to take a companies production Jenkins and then put it in a CI/CD pipeline. Also the steps to configure the jenkins:jenkins:lts image is already well defined. So it should make customizing a Jenkinsfile-runner even easier.

I'm not 100% on the implementation details but I think you could provide two routes.

FROM jenkins/jenkins:LTS

// some steps to configure the Jenkins container with plugins etc

// some steps that install the Jenkinsfile-runner

and/or

FROM jenkins/jenkins:LTS

FROM Jenkinsfile-Runner as runner

COPY --from runner jenkinsfile-runner /usr/bin/

// steps to configure the Jenkins container with plugins etc.

That to me would make it so much easier for me to move people to Jenkinsfile-runner or get them started 🚀

Thanks!

[oleg-nenashev]

As we discussed, https://github.com/jenkinsci/custom-war-packager is a way to do it. To be updated

[shadycuz]

I think the custom-war-packager is a great project and makes a lot of sense to use for people wanting to manage their docker container while getting benefits from tools like dependabot.

But I don't think it is the solution for this issue, which is how do I make it easier for non java folks to use JFR. I do think providing a dockerfile using one or both templates I outlined above would close this issue.

Last time I tried to do this I failed, but It's been a year and I know a lot more about java, so maybe I will find time to try again.

[oleg-nenashev]

I agree @shadycuz . Note that both issues for Tekton and GitHub actions target such integrated experience. I think the same should be done for the Docker images, we can indeed integrate the Custom WAR Packager mode there right away. I will add it as a separate item to the roadmap if you do not mind

FTR Issues

• Custom WAR Packager #570
• Tekton #572
• GitHub Actions #576

[shadycuz]

@oleg-nenashev So I spent a couple hours today trying to tackle this issue. I made progress but eventually got stuck.

Here is the Dockerfile I am trying to make work;

ARG baseImage=jenkins/jenkins:lts

FROM jenkins/jenkinsfile-runner:latest as jfr

FROM ${baseImage}

COPY --from=jfr /app /app

ENV JENKINS_HOME="/usr/share/jenkins/ref/"
ENV JAVA_OPTS="-Djenkins.model.Jenkins.slaveAgentPort=50000 -Djenkins.model.Jenkins.slaveAgentPortEnforce=true -Dhudson.model.LoadStatistics.clock=1000"

ENTRYPOINT ["/app/bin/jenkinsfile-runner",\
            "-w", "/usr/share/jenkins/"]

CMD ["-f /workspace/Jenkinsfile"]

At first I tried just copying over /app/bin/jenkinsfile-runner but that binary doesn't seem to be complete with out the rest of the /app contents.

Once I had that figured out I built the image and tagged it as jsl_jfr:

docker build -t jsl_jfr -f docker/jfr/Dockerfile .

When I ran it I got a very strange error:

shadycuz-> docker run --rm -u root -v $(pwd)/jobs/logging/logging_example.groovy:/workspace/Jenkinsfile jsl_jfr
no Jenkinsfile in current directory.

This is strange because the Jenkinsfile is present:

shadycuz-> docker run --rm -u root -v $(pwd)/jobs/logging/logging_example.groovy:/workspace/Jenkinsfile --entrypoint /bin/bash -it jsl_jfr
root@c8ef72baba98:/# cat /workspace/Jenkinsfile 

node() {
  println('working')
}
root@c8ef72baba98:/# 

Then I tried to run it manually and things got weirder:

root@c8ef72baba98:/# /app/bin/jenkinsfile-runner -w /usr/share/jenkins/ -f /workspace/Jenkinsfile 
java.lang.RuntimeException: Unhandled exception
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:69)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:71)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:21)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.main(Bootstrap.java:46)
Caused by: java.lang.NoClassDefFoundError: hudson/model/RootAction
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:757)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:757)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
        at java.lang.Class.getDeclaredConstructors0(Native Method)
        at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
        at java.lang.Class.getConstructor0(Class.java:3075)
        at java.lang.Class.newInstance(Class.java:412)
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.runJenkinsfileRunnerApp(JenkinsLauncherCommand.java:226)
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:67)
        ... 10 more
Caused by: java.lang.ClassNotFoundException: hudson.model.RootAction
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
        ... 38 more
root@c8ef72baba98:/# 

At this point I thought maybe it was a difference in Java?

Java included in jenkins:lts

root@c8ef72baba98:/# java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

java included with jenkinsfile-runner

shadycuz-> docker run --rm -u root -v $(pwd)/jobs/logging/logging_example.groovy:/workspace/Jenkinsfile --entrypoint /bin/bash -it jenkins/jenkinsfile-runner:latest
root@4c6eeb3bb2e1:/# java -version
openjdk version "1.8.0_262"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_262-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.262-b10, mixed mode)

Seems like the java thing is probably the issue?

[oleg-nenashev]

Looks like the unpacked Jenkins core is not present in the class path. No idea why, need to take a look

[shadycuz]

@oleg-nenashev Maybe you can direct me? Are you talking about /app or /usr/share/jenkins

This is what I see:

root@7c9240dff52f:/usr/share/jenkins# ls -la
total 64700
drwxr-xr-x 1 root    root     4096 May 28  2020 .
drwxr-xr-x 1 root    root     4096 May 28  2020 ..
-rw-r--r-- 1 root    root 66239216 May 28  2020 jenkins.war
drwxr-xr-x 1 jenkins root     4096 May 28  2020 ref

Note that in my Dockerfile, I never exploded any war.

When I do explode it... I don't get very much

jar -xvf jenkins.war
  created: META-INF/
 inflated: META-INF/MANIFEST.MF
  created: ref/
  created: ref/init.groovy.d/

It seemed strange for it to be that empty, maybe I ran the jar command with the wrong flags?

Anyways, he is how the .war file gets put their by the upstream Dockerfile:

# Can be used to customize where jenkins.war get downloaded from
ARG JENKINS_URL=https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war

# could use ADD but this one does not check Last-Modified header neither does it allow to control checksum
# see https://github.com/docker/docker/issues/8331
RUN curl -fsSL ${JENKINS_URL} -o /usr/share/jenkins/jenkins.war \
  && echo "${JENKINS_SHA}  /usr/share/jenkins/jenkins.war" | sha256sum -c -

So it seems like it should be the correct war?

But I'm still getting the same error, even after exploding the war.

[shadycuz]

Also the contents of the /app directory, which look correct.

root@7c9240dff52f:/usr/share/jenkins# ls /app/*
/app/bin:
jenkins-plugin-manager.jar  jenkinsfile-runner  jenkinsfile-runner-launcher  jenkinsfile-runner.bat

/app/jenkins:
ColorFormatter.class  JNLPMain.class  LogFileOutputStream$1.class  LogFileOutputStream$2.class  LogFileOutputStream.class  META-INF  Main$FileAndDescription.class  Main.class  MainDialog$1$1.class  MainDialog$1.class  MainDialog.class  WEB-INF  bootstrap  executable  favicon.ico  robots.txt  winstone.jar

/app/lib:
payload  setup

/app/repo:
com  commons-collections  commons-io  info  io  org

[shadycuz]

Okay so the contents for /app/jenkins is the exploded war from the JFR docker image.

RUN mkdir /app && unzip /jenkinsfile-runner/vanilla-package/target/war/jenkins.war -d /app/jenkins && \
  rm -rf /app/jenkins/scripts /app/jenkins/jsbundles /app/jenkins/css /app/jenkins/images /app/jenkins/help /app/jenkins/WEB-INF/detached-plugins /app/jenkins/WEB-INF/jenkins-cli.jar /app/jenkins/WEB-INF/lib/jna-4.5.2.jar \

but what I can't figure out is why is the war from the jenkins:lts image so empty?

[shadycuz]

I don't know what I did wrong... but I deleted the container and spun it back up fresh and I could properly explode /usr/share/jenkins/jenkins.war and it had tons of files in it.

But I still get error:

root@a809acf3a116:/usr/share/jenkins# /app/bin/jenkinsfile-runner -w /usr/share/jenkins -f /workspace/Jenkinsfile 
java.lang.RuntimeException: Unhandled exception
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:69)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:71)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:21)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.main(Bootstrap.java:46)
Caused by: java.lang.NoSuchFieldError: SYSTEM2
        at io.jenkins.jenkinsfile.runner.JenkinsfileRunnerLauncher.doLaunch(JenkinsfileRunnerLauncher.java:30)
        at io.jenkins.jenkinsfile.runner.JenkinsLauncher.launch(JenkinsLauncher.java:121)
        at io.jenkins.jenkinsfile.runner.App.run(App.java:32)
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.runJenkinsfileRunnerApp(JenkinsLauncherCommand.java:226)
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:67)
        ... 10 more

Is this because my container is using OpenJDK and JFR is expecting AdoptOpenJDK?

[shadycuz]

Strangely, when switching from jenkins:lts which is jdk8 to jenkins/jenkins:latest-jdk11, I get a different error 🤔 :

root@e239ad31accb:/# /app/bin/jenkinsfile-runner -w /usr/share/jenkins -f workspace/Jenkinsfile 
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/usr/share/jenkins/WEB-INF/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
java.lang.RuntimeException: Unhandled exception
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:69)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:71)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:21)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.main(Bootstrap.java:46)
Caused by: java.lang.NoClassDefFoundError: com/cloudbees/plugins/credentials/CredentialsUnavailableException
        at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
        at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
        at java.base/java.lang.Class.getMethodsRecursive(Class.java:3307)
        at java.base/java.lang.Class.getMethod0(Class.java:3293)
        at java.base/java.lang.Class.getMethod(Class.java:2106)
        at io.jenkins.jenkinsfile.runner.JenkinsfileRunnerLauncher.doLaunch(JenkinsfileRunnerLauncher.java:34)
        at io.jenkins.jenkinsfile.runner.JenkinsLauncher.launch(JenkinsLauncher.java:121)
        at io.jenkins.jenkinsfile.runner.App.run(App.java:32)
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.runJenkinsfileRunnerApp(JenkinsLauncherCommand.java:226)
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:67)
        ... 10 more
Caused by: java.lang.ClassNotFoundException: com.cloudbees.plugins.credentials.CredentialsUnavailableException
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        ... 20 more
root@e239ad31accb:/# java -version
openjdk version "11.0.13" 2021-10-19
OpenJDK Runtime Environment Temurin-11.0.13+8 (build 11.0.13+8)
OpenJDK 64-Bit Server VM Temurin-11.0.13+8 (build 11.0.13+8, mixed mode)

My job doesn't use credentials? If that plugin required for use by JFR?

[shadycuz]

Last update for the night. The credentials plugin is installed, I have no clue why I am getting that error 😦.

I did fix the no Jenkinsfile in current directory. error. I tried passing a single string with a space in it, but it needed to be broken up like this:

CMD ["-f", "/workspace/Jenkinsfile"]

I can now call it from the cmd line with out having to enter the container and run the command manually.

shadycuz-> docker run --rm -u root -v $(pwd)/jobs/logging/logging_example.groovy:/workspace/Jenkinsfile jsl_jfr
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/usr/share/jenkins/WEB-INF/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
java.lang.RuntimeException: Unhandled exception
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:69)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:71)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.call(Bootstrap.java:21)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at io.jenkins.jenkinsfile.runner.bootstrap.Bootstrap.main(Bootstrap.java:46)
Caused by: java.lang.NoClassDefFoundError: com/cloudbees/plugins/credentials/CredentialsUnavailableException
        at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
        at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
        at java.base/java.lang.Class.getMethodsRecursive(Class.java:3307)
        at java.base/java.lang.Class.getMethod0(Class.java:3293)
        at java.base/java.lang.Class.getMethod(Class.java:2106)
        at io.jenkins.jenkinsfile.runner.JenkinsfileRunnerLauncher.doLaunch(JenkinsfileRunnerLauncher.java:34)
        at io.jenkins.jenkinsfile.runner.JenkinsLauncher.launch(JenkinsLauncher.java:121)
        at io.jenkins.jenkinsfile.runner.App.run(App.java:32)
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.runJenkinsfileRunnerApp(JenkinsLauncherCommand.java:226)
        at io.jenkins.jenkinsfile.runner.bootstrap.commands.JenkinsLauncherCommand.call(JenkinsLauncherCommand.java:67)
        ... 10 more
Caused by: java.lang.ClassNotFoundException: com.cloudbees.plugins.credentials.CredentialsUnavailableException
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        ... 20 more

Still not sure why I'm getting a ton of weird errors, but I'm getting the closest I have been is with jdk11.

[shadycuz]

SERIOUSLY THE LAST UPDATE TONIGHT.

Do you think if I passed the -p flag... then this would work correctly 👀

shadycuz-> docker run --rm -u root -v $(pwd)/jobs/logging/logging_example.groovy:/workspace/Jenkinsfile jsl_jfr
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/usr/share/jenkins/WEB-INF/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Started
Resume disabled by user, switching to high-performance, low-durability mode.
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /tmp/jenkinsfileRunner.tmp/jfr11682560948436011876.run/workspace/job
[Pipeline] {
[Pipeline] echo
working
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

We will never speak of this again 🙄

So this is the final Dockerfile:

ARG baseImage=jenkins/jenkins:latest-jdk11

FROM jenkins/jenkinsfile-runner:latest as jfr

FROM ${baseImage}

USER root

COPY --from=jfr /app /app

# These are optional? kinda? Not sure what the bare min plugins are for JFR
RUN jenkins-plugin-cli --plugins workflow-aggregator filesystem_scm git

RUN cd /usr/share/jenkins && jar -xvf jenkins.war

ENV JENKINS_HOME="/usr/share/jenkins/ref/"
ENV JAVA_OPTS="-Djenkins.model.Jenkins.slaveAgentPort=50000 -Djenkins.model.Jenkins.slaveAgentPortEnforce=true -Dhudson.model.LoadStatistics.clock=1000"

ENTRYPOINT ["/app/bin/jenkinsfile-runner", "-w", "/usr/share/jenkins/", "-p", "/usr/share/jenkins/ref/plugins", "-f"]

CMD ["/workspace/Jenkinsfile"]

Expect a PR from me in the next day or two.

[shadycuz]

@oleg-nenashev with the PR merged, I think this is closable.

[oleg-nenashev]

@shadycuz maybe. In my case I was rather thinking about broader story as mentioned in the title: "Make Jenkinsfile Runner accessible for non-Java users". I believe there are many other enhancements that could be done towards this goal, e.g. #570 or #572. Probably I should create another issue for the roadmap and rename this ticket, WDYT?

[shadycuz]

I think a GitHub project around user experience would work best.

That way each story gets it's own issue and it's easy to group things and track progress.

@oleg-nenashev