rnotebook
rnotebook copied to clipboard
yihui
alternative JSON structure
I really like this idea and I always wanted to include all related information in my reproducible reports -- that's why I came up with the rapport package, where the generated markdown report not only includes the results, but also the raw R object with all messages etc. Maybe some parts of this rather old structure might be interesting for a quick review, otherwise please feel free to close this ticket:
> cat(RJSONIO::toJSON(rapport::tpl.example('example', 1), pretty = TRUE))
{
"meta" : {
"title" : "Example template",
"author" : "Gergely Daróczi",
"packages" : [
"ggplot2",
"xtable"
],
"example" : [
"rapport(\"example\", ius2008, v='age')",
"rapport(\"example\", ius2008, v='gender', pacman=FALSE)",
"rapport(\"example\", ius2008, v='age', s='FOO BAR')"
],
"description" : "This template demonstrates the basic features of rapport. We all hope you will like it!"
},
"inputs" : [
{
"name" : "v",
"label" : "Variable",
"description" : "A variable",
"length" : {
"min" : 1,
"max" : 1
},
"value" : null,
"required" : true,
"standalone" : false
},
{
"name" : "pacman",
"label" : "Pacman",
"description" : "Show Pacman in the results?",
"class" : "logical",
"length" : {
"min" : 1,
"max" : 1
},
"value" : true,
"required" : false,
"standalone" : true
},
{
"name" : "s",
"label" : "A string",
"description" : "Any character value to be printed at the end of the report",
"class" : "character",
"length" : {
"min" : 1,
"max" : 1
},
"value" : "Bye!",
"nchar" : {
"min" : 1,
"max" : 256
},
"required" : false,
"standalone" : true
}
],
"report" : [
{
"type" : "text",
"text" : {
"raw" : "\n",
"eval" : "\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
},
{
"type" : "heading",
"text" : {
"raw" : "Hello, world!\n",
"eval" : "Hello, world!\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
},
"level" : 1
},
{
"type" : "text",
"text" : {
"raw" : "\nThis template is a tutorial on how to write [rapport](http://rapport-package.info) templates.\n\nYou have already learned a lot: above we [say hello to the world](http://en.wikipedia.org/wiki/Hello_world_program) in a header (of level `1`) in [ATX-style](http://johnmacfarlane.net/pandoc/README.html#atx-style-headers). So the first lesson of all programming tutorials is accomplished :)\n\nA rapport template is a regular [Pandoc's markdown](http://johnmacfarlane.net/pandoc/) file with two exceptions:\n\n * the file starts with a special `header` which is documented on the (rapport homepage](http://rapport-package.info/#custom). In short: you need to specify some metadata (like: author, title and description of the template, what kind of parameters would you like to use *in* the template and some examples of the template)\n * and text between `brew` tags are recognized as R commands. Details about this can be found in [`pander`'s documentation](http://rapporter.github.com/pander/#brew-to-pandoc).\n\nLet us check out this latter (short descriptives about _<%=v.name%>_):\n\n",
"eval" : "\nThis template is a tutorial on how to write [rapport](http://rapport-package.info) templates.\n\nYou have already learned a lot: above we [say hello to the world](http://en.wikipedia.org/wiki/Hello_world_program) in a header (of level `1`) in [ATX-style](http://johnmacfarlane.net/pandoc/README.html#atx-style-headers). So the first lesson of all programming tutorials is accomplished :)\n\nA rapport template is a regular [Pandoc's markdown](http://johnmacfarlane.net/pandoc/) file with two exceptions:\n\n * the file starts with a special `header` which is documented on the (rapport homepage](http://rapport-package.info/#custom). In short: you need to specify some metadata (like: author, title and description of the template, what kind of parameters would you like to use *in* the template and some examples of the template)\n * and text between `brew` tags are recognized as R commands. Details about this can be found in [`pander`'s documentation](http://rapporter.github.com/pander/#brew-to-pandoc).\n\nLet us check out this latter (short descriptives about _age_):\n\n"
},
"chunks" : {
"raw" : "<%=v.name%>",
"eval" : "age"
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
},
{
"type" : "block",
"robject" : {
"src" : "if (is.numeric(v)) {\n summary(v)\n} else {\n table(v)\n}",
"result" : {
"Min." : 16,
"1st Qu." : 20,
"Median" : 23,
"Mean" : 24.57,
"3rd Qu." : 26,
"Max." : 58,
"NA's" : 32
},
"output" : [
"",
"--------------------------------------------------------",
" Min. 1st Qu. Median Mean 3rd Qu. Max. NA's ",
"------ --------- -------- ------ --------- ------ ------",
" 16 20 23 24.57 26 58 32 ",
"--------------------------------------------------------",
""
],
"type" : [
"summaryDefault",
"table"
],
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
},
"stdout" : null
}
},
{
"type" : "text",
"text" : {
"raw" : "\n\nAbove we have a so called `chunk` between the special `brew` tags and the result is printed in the run template. Everything between the tags are evaluated after parsing the chunk line-by-line and are also printed if returned anything.\n\nTags can be inline too <%=\"of course\"%>, but multi line results (e.g. ",
"eval" : "\n\nAbove we have a so called `chunk` between the special `brew` tags and the result is printed in the run template. Everything between the tags are evaluated after parsing the chunk line-by-line and are also printed if returned anything.\n\nTags can be inline too of course, but multi line results (e.g. "
},
"chunks" : {
"raw" : "<%=\"of course\"%>",
"eval" : "of course"
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
},
{
"type" : "block",
"robject" : {
"src" : "mtcars[1:5, 1:2]",
"result" : {
"mpg" : [
21,
21,
22.8,
21.4,
18.7
],
"cyl" : [
6,
6,
4,
6,
8
]
},
"output" : [
"",
"-----------------------------------",
" mpg cyl ",
"----------------------- ----- -----",
" **Mazda RX4** 21 6 ",
"",
" **Mazda RX4 Wag** 21 6 ",
"",
" **Datsun 710** 22.8 4 ",
"",
" **Hornet 4 Drive** 21.4 6 ",
"",
" **Hornet Sportabout** 18.7 8 ",
"-----------------------------------",
""
],
"type" : "data.frame",
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
},
"stdout" : null
}
},
{
"type" : "text",
"text" : {
"raw" : ") are get separated.\n\n`brew` ninjas also know that another pair of tag is existing in the syntax, but please hold on. We really suggest to always use the so called `BRCATCODE` (ending in an equal sign), because only those are [cached](http://rapporter.github.com/pander/#caching) and the robust error handling does not happen in \"simple\" `BRCODE`.\n\n`BRCODE` tags are useful when you want to loop through something or optionally filter out/conditionally format a part of the template. A really easy example of this:\n\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n * <%=l%>\n\n\nWe have just shown the alphabet to the users in a list.\n\nLoops and `if` conditionals without the curly braces do not work between the `BRCATCODE` tags, as everything between those are `parse`d. So in short: use `BRCODE` for loops and filtering, use `BRCATCODE` for the rest.\n\n",
"eval" : ") are get separated.\n\n`brew` ninjas also know that another pair of tag is existing in the syntax, but please hold on. We really suggest to always use the so called `BRCATCODE` (ending in an equal sign), because only those are [cached](http://rapporter.github.com/pander/#caching) and the robust error handling does not happen in \"simple\" `BRCODE`.\n\n`BRCODE` tags are useful when you want to loop through something or optionally filter out/conditionally format a part of the template. A really easy example of this:\n\n\n * a\n\n * b\n\n * c\n\n * d\n\n * e\n\n * f\n\n * g\n\n * h\n\n * i\n\n * j\n\n * k\n\n * l\n\n * m\n\n * n\n\n * o\n\n * p\n\n * q\n\n * r\n\n * s\n\n * t\n\n * u\n\n * v\n\n * w\n\n * x\n\n * y\n\n * z\n\n\nWe have just shown the alphabet to the users in a list.\n\nLoops and `if` conditionals without the curly braces do not work between the `BRCATCODE` tags, as everything between those are `parse`d. So in short: use `BRCODE` for loops and filtering, use `BRCATCODE` for the rest.\n\n"
},
"chunks" : {
"raw" : [
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>",
"<%=l%>"
],
"eval" : [
"a",
"b",
"c",
"d",
"e",
"f",
"g",
"h",
"i",
"j",
"k",
"l",
"m",
"n",
"o",
"p",
"q",
"r",
"s",
"t",
"u",
"v",
"w",
"x",
"y",
"z"
]
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
},
{
"type" : "heading",
"text" : {
"raw" : "Errors and warning\n",
"eval" : "Errors and warning\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
},
"level" : 1
},
{
"type" : "text",
"text" : {
"raw" : "\nWhat happens if you have an error in your document?\n\n",
"eval" : "\nWhat happens if you have an error in your document?\n\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
},
{
"type" : "block",
"robject" : {
"src" : "mean(foobar)",
"result" : null,
"output" : " **ERROR**^[object 'foobar' not found]",
"type" : "error",
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : "object 'foobar' not found"
},
"stdout" : null
}
},
{
"type" : "text",
"text" : {
"raw" : "\n\nWhich is possible even inline too: <%=foo%> and <%=bar%> beside a normal chunk showing $\\pi$ (<%=pi%>).\n\nAnd how do warnings show up?\n\n",
"eval" : "\n\nWhich is possible even inline too: **ERROR**^[object 'foo' not found] and **ERROR**^[object 'bar' not found] beside a normal chunk showing $\\pi$ (_3.142_).\n\nAnd how do warnings show up?\n\n"
},
"chunks" : {
"raw" : [
"<%=foo%>",
"<%=bar%>",
"<%=pi%>"
],
"eval" : [
" **ERROR**^[object 'foo' not found]",
" **ERROR**^[object 'bar' not found]",
"_3.142_"
]
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : [
"object 'foo' not found",
"object 'bar' not found"
]
}
},
{
"type" : "block",
"robject" : {
"src" : "chisq.test(mtcars$am, mtcars$gear)",
"result" : {
"statistic" : {
"X-squared" : 20.945
},
"parameter" : {
"df" : 2
},
"p.value" : 2.8309e-05,
"method" : "Pearson's Chi-squared test",
"data.name" : "mtcars$am and mtcars$gear",
"observed" : [
{
"3" : 15,
"4" : 4,
"5" : 0
},
{
"3" : 0,
"4" : 8,
"5" : 5
}
],
"expected" : [
{
"3" : 8.9062,
"4" : 7.125,
"5" : 2.9688
},
{
"3" : 6.0938,
"4" : 4.875,
"5" : 2.0312
}
],
"residuals" : [
{
"3" : 2.0419,
"4" : -1.1707,
"5" : -1.723
},
{
"3" : -2.4686,
"4" : 1.4153,
"5" : 2.083
}
],
"stdres" : [
{
"3" : 4.3953,
"4" : -2.3234,
"5" : -2.943
},
{
"3" : -4.3953,
"4" : 2.3234,
"5" : 2.943
}
]
},
"output" : [
"",
"---------------------------------------",
" Test statistic df P value ",
"---------------- ---- -----------------",
" 20.94 2 _2.831e-05_ * * *",
"---------------------------------------",
"",
"Table: Pearson's Chi-squared test: `mtcars$am` and `mtcars$gear`",
"",
" **WARNING**^[Chi-squared approximation may be incorrect]"
],
"type" : "htest",
"msg" : {
"messages" : null,
"warnings" : "Chi-squared approximation may be incorrect",
"errors" : null
},
"stdout" : null
}
},
{
"type" : "text",
"text" : {
"raw" : "\n\n",
"eval" : "\n\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
},
{
"type" : "heading",
"text" : {
"raw" : "Control-flow\n",
"eval" : "Control-flow\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
},
"level" : 1
},
{
"type" : "text",
"text" : {
"raw" : "\nWe almost forgot about our inputs! A quite detailed documentation can be found on [rapport homepage](http://rapport-package.info/#custom).\n\nLet us check if the template was called with `pacman` parameter set to TRUE and show some great stuff if so:\n\n\n",
"eval" : "\nWe almost forgot about our inputs! A quite detailed documentation can be found on [rapport homepage](http://rapport-package.info/#custom).\n\nLet us check if the template was called with `pacman` parameter set to TRUE and show some great stuff if so:\n\n\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
},
{
"type" : "block",
"robject" : {
"src" : "pie(c(2, 8), init.angle = -30, main = \"I was born 30 years ago!\")",
"result" : "plots/rapport-example-7-1.png",
"output" : "",
"type" : "image",
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
},
"stdout" : null
}
},
{
"type" : "text",
"text" : {
"raw" : "\nYeah, this plot is generated with `pie` from `graphics` package, no fancy `lattice` or `ggplot2` call :)\n\n\n",
"eval" : "\nYeah, this plot is generated with `pie` from `graphics` package, no fancy `lattice` or `ggplot2` call :)\n\n\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
},
{
"type" : "heading",
"text" : {
"raw" : "Evaluate multiple commands at once\n",
"eval" : "Evaluate multiple commands at once\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
},
"level" : 1
},
{
"type" : "text",
"text" : {
"raw" : "\nAll R code in the template body are parsed to separate expressions to grab all error/warning/simple message. This is really comfortable and useful most of the time, but not desired with e.g. complex plots (so multiple lines creates one plot after all) or in other situations when you want evaluate your commands at one run.\n\nOf course there is workaround: use `+` in the beginning of the line which you want to run locked with the prior one(s). For example let us create a plot with two histograms:\n\n",
"eval" : "\nAll R code in the template body are parsed to separate expressions to grab all error/warning/simple message. This is really comfortable and useful most of the time, but not desired with e.g. complex plots (so multiple lines creates one plot after all) or in other situations when you want evaluate your commands at one run.\n\nOf course there is workaround: use `+` in the beginning of the line which you want to run locked with the prior one(s). For example let us create a plot with two histograms:\n\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
},
{
"type" : "block",
"robject" : {
"src" : [
"par(mfrow = c(2, 1))",
"hist(mtcars$wt)",
"hist(mtcars$hp)"
],
"result" : "plots/rapport-example-7-2.png",
"output" : "",
"type" : "image",
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
},
"stdout" : null
}
},
{
"type" : "text",
"text" : {
"raw" : "\n\n",
"eval" : "\n\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
},
{
"type" : "heading",
"text" : {
"raw" : "H3 tag\n",
"eval" : "H3 tag\n"
},
"chunks" : {
"raw" : null,
"eval" : null
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
},
"level" : 3
},
{
"type" : "text",
"text" : {
"raw" : "\nOh, and by the way: <%=s%>\n",
"eval" : "\nOh, and by the way: Bye!\n"
},
"chunks" : {
"raw" : "<%=s%>",
"eval" : "Bye!"
},
"msg" : {
"messages" : null,
"warnings" : null,
"errors" : null
}
}
],
"call" : {
"" : "rapport",
"fp" : "example",
"data" : "ius2008",
"v" : "age"
},
"time" : 0.623,
"file.name" : "/tmp/RtmpLcRGdZ/rapport-example-7-%n"
}
Thanks for the reference! It is definitely interesting. As I said in README.md, this package was only a quick and dirty experiment (I spent about two afternoons on it), and chances are it may end up nowhere eventually. I do not have strong opinions on whether source and output should be in one file or not. I'm fine with either way. JSON is certainly more human-readable than XML, but still less human-readable than R Markdown. It is easy to come up with a JSON spec for notebooks, but it is more important and challenging to get the editor support (be it RStudio or Emacs or any other editors). Without a nice (graphical) user interface, I do not think anybody will be happy with editing a large JSON file. I can add some simple GUI support for this package, but I do not think I can do it well.
The above list was generated from an R object with rapport class, which prints nicely in the R console (as markdown). And we also have some helper functions to edit/remove parts of it, even via GUI at http://rapporter.net
Not sure if it would be interesting or not, but I hope to get some time to clean up the codebase and open-source the stuff -- although it depends on bunch of other projects as being a webapp (AppArmor, CouchDB etc)
Anyway, I wish you great luck with the project!