burndown

burndown copied to clipboard

#+STARTUP: org-startup-with-inline-images inlineimages

• burndown -- Charting for agile projects using org-mode and Git

  Uses [[https://github.com/mbostock/d3]] to generate burn-down charts from org-mode project Effort estimates and CLOCKSUM time log data. The historical "Effort" (estimate) and "CLOCKSUM" (work) data are collected from the Git repository, and rendered as Burndown charts, as per:

  http://www.mountaingoatsoftware.com/scrum/alt-releaseburndown
  

  Normal burndown charts don't differentiate between changes in project scope (adding or removing task), and progress on the project (converting tasks from "todo" to "done"). The "alternative" burndown chart shows project progress by changes along the top of the bar chart, and changes in project scope on the bottom of the bar chart -- where the best-fit lines meet is where the project is likely to be finished.

  By separating the two concepts, it is absolutely clear where changes in the project finish originate; either due to changes in work progress, or changes in project scope.

** Finish Day/Date Projection

The project is going to finish A) after a certain number of project Days of effort, and B) on a certain calendar Date. These are somewhat independent, since the amount of work applied to the project changes over time (holidays, changes in staffing, etc.)

To compute the Project's finish Day, we use a best-fit curve to project the intersection of the Project's progress and change lines. Based on the burndown style selected, this reflects units of effort, units of elapsed time, or sprints.

Once the finish Day is computed, we also use a best-fit line through each sample's actual calendar date, to predict the finish Date. This gives us a correlation between each burndown unit, and calendar time.

** Burndown Styles

Three styles of burndown chart are available. All of these change the meaning of the X (time) axis. The Y (estimate) axis always represents the amount of task "Effort" (estimate) remaining in the project. By changing the X axis, we can see how the project is progressing as we track:

*** effort

Tracks project "CLOCKSUM" (work) time.  Each graph X axis sample represents
a certain number of hours of actual work on the project.  The default is 8
hours.  This computes the project finsh "Day" based on effort-days, and a
finish Date based on a projection of historical work Days per calendar Day.

*** elapsed

Tracks calendar elapsed time.  This produces one X axis sample per calendar
day.  Therefore, the computed project finish Day and calendar Date are
identical.

*** sprint (not yet implemented)

Tracks project Sprints, one sprint per sample.  Works best if your sprints
represent roughly equal work effort, but will respond to changes in Sprint
effort over time, in a best-fit fashion.  Predicts the number of Sprints
remaining.

** Maintaining org-mode Project Data

Prepare an org-mode file containing a project, in the form of a tree of TODO entries. Ensure that the top-level TODO entry for the project contains a date entry, and put an org-mode table within it:

#+BEGIN_EXAMPLE * TODO Project Test <2012-03-18 Sun>
#+BEGIN: columnview :hlines 1 :id local | Task | Effort | CLOCKSUM | |--------------------------------------+--------+----------| | * TODO Project Test <2012-03-18 Sun> | 46:00 | | | ** TODO Setup | 10:00 | | #+END: ** TODO Setup ... #+END_EXAMPLE

Each day, update the CLOCKSUM (time worked) and/or Effort (TODO estimates) data in the org file. Update the top-level Project's date. Finally, refresh the table (hit C-c C-c in the BEGIN: line of the data summary table), save the file and commit it to the Git repository.

The orgserver will parse the project data from the specified project's .org files in the Git repository. By scanning the historical contents in the Git commit history, a burn-down chart will be rendered. We use d3, so make sure your browser supports SVG (eg. Chrome, Firefox or IE9+).

** orgserver -- A web.py based HTML and JSON API server

Serves org-mode data from the specified org directory, for the specified project(s). May be configured to update from git, and kill running server(s) if changes to the burndown chart implementation itself are detected.

The running orgserver starts the python orgserver.py and puts its PID in the org directory in an orgserver.pid.##### file ending with the orgserver's own PID. This is used by subsequent orgserver invocations, to locate and terminate the other running orgserver.py instances.

The options are:

#+BEGIN_EXAMPLE orgserver [-- [...]] [ [...]] --help This help --verbose Log activity --restart Restart any running server (default: no) --refresh Refresh Git repositories (default: no) --address <i[:p]> Bind to interface:port (default: :8080) Performs a "git pull origin master" on both the org data directory, and the orgserver directory. In either case, if the master branch's commit changes, the orgserver will forcibly restart any running orgserver.py. --org

The following environment variables may be set, instead of specifying the Org data directory and the Projects list on the command line:

#+BEGIN_EXAMPLE ORG_PROJECTS -- The list of project names (default: project) ORG_DIRECTORY -- The org directory (default: ~/org) #+END_EXAMPLE

*** EXAMPLE

Run a orgserver bound to *:8080 on the host, serving org-mode data from
~/org, for the project 'test' (ie. from org-data in ~/org/test.org):

#+BEGIN_EXAMPLE ./orgserver --verbose --org=~/org test #+END_EXAMPLE

Start (or restart) the orgserver's underlying webserver, refreshing the Org
data (~/org) and orgserver (~/src/burndown) Git repositories.  Exits quietly
if the orgserver is already running, and restarts it if the orgserver
repository 'master' branch has had new commits:

   ~/src/burndown/orgserver --refresh --org=~/org some project names

This can be set up to occur automatically (say, every 5 minues) using crontab -e:

#+BEGIN_EXAMPLE
    # minute hour mday month wday command
      */5    *    *    *     *    $HOME/src/burndown/orgserver --refresh --log orgserver.log some project names
#+END_EXAMPLE

** orgserver.py -- A Python module for process org-mode data in Git repositories

When run as a command (as by orgserver), orgserver.py starts a web.py webserver, by default bound to port 8080 on all interfaces.

*** HTTP JSON API

The HTTP API respects the Accept: header, and generally responds to
"text/html" requests with HTML, and to "text/javascript", "application/json"
and "text/plain" requests with JSON.  Alternatively, any HTTP request may
force JSON by appending ".json" to any URI.

**** /api/projects[.json]

 Returns a list of all projects.
 #+BEGIN_EXAMPLE
 [
     {
         "project": "burndown", 
         "styles": [
             "effort", 
             "elapsed", 
             "sprint"
         ]
     },
 ]
 #+END_EXAMPLE    

**** /api/data/[/

 Returns the request project's data in the specified style.  The "list"
 attribute contains an entries with non-empty "estimated" and "work" data
 for every day actual data, and computed data (inserted in-between existing
 data points, to ensure that the data points are consistently spaced for the
 requested X axis style.)  All project data is supplied in 2 forms: "xxxx":
 <textual> and "xxxx#": <seconds>.  Relative time values (eg. task "added#")
 are in seconds from 0 (when the project started), and absolute time values
 (eg. the sample "date#") are in seconds since Jan 1, 1970 UTC (the unix
 Epoch).

 If burndown trend lines can be computed, they are included; the "change"
 and "progress" lines are in seconds since the project started, and "date"
 in seconds since the Epoch.

 Empty data points are appended to "list" between the final day with project
 data and the computed finish point of the project (or some maximum limit,
 if the computed finish date is far in the future).  These have everything
 except "lines", "work" and "estimated" are null.
 
 #+BEGIN_EXAMPLE    
 {
     "list": [
         {
             "blob": "6c4a8197f1efc91dbb197a0e210ad7b38a03a6b1", 
             "date": "2012-03-01", 
             "date#": 1330585200.0, 
             "estimated": {
                 "added": "0:00", 
                 "added#": 0, 
                 "addedTotal": "0:00", 
                 "addedTotal#": 0, 
                 ...
             }, 
             "label": "Day 1", 
             "lines": {
                 "change": {
                     "x1": 0, 
                     "x2": 1, 
                     "y1": 0, 
                     "y2": 0
                 }, 
                 "date": {
                     "x1": 0, 
                     "x2": 1, 
                     "y1": 1330585200, 
                     "y2": 1330671600
                 }, 
                 "progress": {
                     "x1": 0, 
                     "x2": 1, 
                     "y1": 61200, 
                     "y2": 61200
             "sprint": 0, 
             "work": {
                 "added": "0:00", 
                 "added#": 0, 
                 "addedTotal": "0:00", 
                 "addedTotal#": 0, 
                 ...
             }
         }, 
         ...
         {
             "blob": null, 
             "date": "2012-04-02", 
             "date#": 1333425728.2857144, 
             "estimated": null, 
             "finish": "2012-03-25 (Day 15)", 
             "label": "Day 19", 
             "lines": null, 
             "sprint": 0, 
             "work": null
         }, 
     ],
     "project": "burndown", 
     "style": "effort"
 }
 #+END_EXAMPLE

*** REQUIREMENTS

If you are on a Mac, you might look at https://github.com/pjkundert/setup to
see detailed build and installation automation and instructions for these
(and other) packages.

The following Python modules are required (all these assume your
PYTHONPATH=/usr/local/lib/python2.7/site-packages):

**** git-python 0.3; requires nose and mock for tests

 Obtain source from https://github.com/gitpython-developers/GitPython
 Assuming your PYTHONPATH=/usr/local/lib/python2.7/site-packages, this might
 work:
 
 #+BEGIN_EXAMPLE
 git clone git://github.com/gitpython-developers/GitPython.git git-python
 cd git-python
 git checkout origin 0.3
 git submodule update --init --recursive
 python setup.py install --prefix=/usr/local
 #+END_EXAMPLE

**** web.py

 Obtain source from https://github.com/webpy/webpy

 #+BEGIN_EXAMPLE
 git clone git://github.com/webpy/webpy.git
 cd webpy
 git checkout origin master
 python setup.py install --prefix=/usr/local
 #+END_EXAMPLE

**** wsgilog

 Obtain source from https://bitbucket.org/lcrees/wsgilog/src

 #+BEGIN_EXAMPLE
 hg clone https://bitbucket.org/lcrees/wsgilog
 cd wsgilog
 hg pull -u
 cd wsgilog # yes, again...
 python setup.py install --prefix=/usr/local
 #+END_EXAMPLE

**** argparse For pre-2.6 Python, you'll need to install argparse and simplejson:

 #+BEGIN_EXAMPLE
 sudo easy_install argparse
 sudo easy_install simplejson
 #+END_EXAMPLE

*** SCREENSHOT

The burndown project itself, in "effort" style:
[[file:static/resources/screenshot-burndown.png]]