xts

xts copied to clipboard

Description

A program that successfully created a PDF plot with xts 0.9-7 generates a (nearly) empty PDF file with xts 10.0-0. The file has non-zero length, but it is much smaller than the expected one, and Acrobat Reader says that it is empty.

Expected behavior

See above. I am attaching a .zip file with a sample program, input data, two versions of the PDF output files (using two versions of xts), and a file 00-README.txt with more information.

Minimal, reproducible example

xts-plot-problem.zip

Session Info

> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server release 6.9 (Santiago)

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

Thanks for the report and reproducible example! I also encounter issues when I try to run your code using the most recent commit in the repo. We'll investigate.

This seems related to #156.

While you wait on a fix, here's a work-around that should build a plot similar to what the old version of xts would have created.

pdf(file="data.pdf", paper="letter", width=7.5, height=10)
par(mfrow=c(3,1))
for (name in names(data_xts)) {
    d <- data_xts[,name]
    ylim <- range(d)
    if (ylim[1] == ylim[2])
      ylim <- ylim * c(0.98, 1.02)
    print(xts::plot(d,main=name,ylim=ylim))
}
graphics.off()

Note that you now need to explicitly call print() on the plot.xts() result if you call it from inside a loop (for or while) or inside function. Also note that you generally should not call methods directly; just call the generic plot() function and let S3 method dispatch handle the rest.

I'm able to run your code after re-installing the most current development version of xts and adding the print() around the plot() call. Not sure what version I was running when I wrote my previous comment. Can you install the most current development version (remotes::install_github("joshuaulrich/xts")) and add the print() call to see if that solves the problem?

That works for the sample program and data set I sent you. However, the sample program was a stripped-down version of a larger program that had options for generating other graphics formats, and at least one of those still fails. Here is a variation of my original example that generates SVG output files:

#!/usr/bin/env Rscript
library("xts")
data <- read.csv(file="data.csv", stringsAsFactors=FALSE, header=TRUE, sep="\t")
data[[1]] <- as.POSIXct(data[[1]], tz="", "%FT%T%z")
data_xts <- as.xts(data[,2:ncol(data)], data[[1]], dateFormat="POSIXct")
for (name in names(data_xts)) {
    filename=paste(name, ".svg", sep="")
    svg(filename=filename, width=7.5, height=3.3)
    print(xts::plot.xts(data_xts[,name],main=name))
    graphics.off()
}

Again, this works with 0.9-7 (generating one .svg file per data column) but still fails with your new 0.10-1, generating the following messages:

$ ./plot_ts-svg-1.R
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric


 *** caught segfault ***
address 0x4, cause 'memory not mapped'

Traceback:
 1: clip(par("usr")[1], par("usr")[2], ylim[[frame]][1], ylim[[frame]][2])
 2: set_window(clip)
 3: cs$set_frame(1)
 4: xts::plot.xts(data_xts[, name], main = name)
 5: print(xts::plot.xts(data_xts[, name], main = name))
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault (core dumped)

I tried adding the workaround statements that you provided earlier, but that did not work.

I did not mean to close the issue. I apologize; I am new at GitHub.

No worries about he the close/re-open. It happens. No harm done.

Thanks for the updated example... which is an exceptionally worse error than the original. :( I cannot replicate that crash using the data in the .zip file you provided in your initial report. Is that what you used? If you used a different data set, is there any way you can provide it to me? If you don't want it to be public, you can email it to me.

Sorry to be the bearer of bad news. I did use the same data set. Just to be sure, I tried again; here are the results:

$ more plot_ts-svg.R
#!/usr/bin/env Rscript
library("xts")
data <- read.csv(file="data.csv", stringsAsFactors=FALSE, header=TRUE, sep="\t")
data[[1]] <- as.POSIXct(data[[1]], tz="", "%FT%T%z")
data_xts <- as.xts(data[,2:ncol(data)], data[[1]], dateFormat="POSIXct")
for (name in names(data_xts)) {
    filename=paste(name, ".svg", sep="")
    svg(filename=filename, width=7.5, height=3.3)
    print(xts::plot.xts(data_xts[,name],main=name))
    graphics.off()
}

$ ./plot_ts-svg.R
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric


 *** caught segfault ***
address 0x4, cause 'memory not mapped'

Traceback:
 1: clip(par("usr")[1], par("usr")[2], ylim[[frame]][1], ylim[[frame]][2])
 2: set_window(clip)
 3: cs$set_frame(1)
 4: xts::plot.xts(data_xts[, name], main = name)
 5: print(xts::plot.xts(data_xts[, name], main = name))
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault (core dumped)

$ R

R version 3.3.3 (2017-03-06) -- "Another Canoe"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-redhat-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Server release 6.9 (Santiago)

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     
> quit("no")

For good measure, I am attaching another .zip file with the script and the data set used above. xts-svg-problem.zip

Here is a quick note. I copied (tar-ed) the build of xts 0.9-7 from the older RHEL6 system with R 3.2.3 to the newer RHEL6 system with R 3.3.3, where I found the problem with xts 0.10-0 and that now has 0.10-1. I replaced the newer build directory of xts with the 0.9-7 directory, and my original program now works as do the two test programs (PDF, SVG) I have sent you. So, that is my workaround on this system, but that has limited utility, obviously. If I can provide you any further useful information, please ask.

I could not replicate this even when run with gctorture(TRUE), which usually triggers memory protection issues. I'm running a newer version of R on a slightly different platform, however:

R version 3.4.1 (2017-06-30)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.2 LTS

Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_3.4.1 tools_3.4.1

My working hypothesis is that this is an issue with your local machine(s). This is because the traceback shows clip() as the last function called, and the clip() function immediately calls compiled code:

R> clip
function (x1, x2, y1, y2) 
invisible(.External.graphics(C_clip, x1, x2, y1, y2))
<bytecode: 0x3025698>
<environment: namespace:graphics>

I just upgraded xts to version 0.10-0 on my Macintosh PowerBook, and I reproduce the problem:

> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.6

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

MC02SD6B1G8WN$ ./plot_ts-svg.R
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric


 *** caught segfault ***
address 0x4, cause 'memory not mapped'

Traceback:
 1: clip(par("usr")[1], par("usr")[2], ylim[[frame]][1], ylim[[frame]][2])
 2: set_window(clip)
 3: cs$set_frame(1)
 4: xts::plot.xts(data_xts[, name], main = name)
 5: print(xts::plot.xts(data_xts[, name], main = name))
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault: 11

In common with the RHEL 6 server I have but different from you, I am running R 3.3.x. I will upgrade R on my Mac to 3.4.1 and see what that does.

I just remembered I have access to a machine running R-3.3.2, and I'm able to reproduce the crash on it. I'll investigate further.

I upgraded R on my Mac to 3.4.1 and installed xts-0.10-0. The SVG test program runs without crashing, but it produces empty SVG plot files. I am attaching two .zip files:

• output from RHEL6 running R-3.3.3 and xts-0.9-7 (works)
• output from macOS 10.12.6 running R-3.4.1 and xts-0.10-0 (fails) Each .zip file contains the R test script, data file, and resulting SVG output files; also, each file includes output from sessionInfo() and installed.packages(). Both systems have zoo-1.8-0. RHEL6_R-3.3.3_xts-0.9-7_works.zip macOS-10.12.6_R-3.4.1_xts-0.10-0_fails.zip

Thanks for the additional example. I'll try to replicate. I have confirmed that the crash was caused by a bug in base R, not xts:

julrich@computer:~? R
R version 3.3.2 (2016-10-31)
R> svg(); clip(1502769757.0, 1502862745.0, 0, 1); dev.off()

 *** caught segfault ***
address 0x4, cause 'memory not mapped'

Traceback:
 1: clip(1502769757, 1502862745, 0, 1)

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: 2
julrich@computer:~?

And it was likely fixed as part of this bugfix mentioned in the NEWS file:

R> print(news(grepl("svg", Text), db = db), doBrowse = FALSE)
Changes in version 3.4.0:

BUG FIXES

    o   Fix for cairo_pdf() (and svg() and cairo_ps()) when replaying a
         saved display list that contains a mix of grid and graphics output.
         (Report by Yihui Xie.)

Changes in version 3.3.0:

NEW FEATURES

    o   R now uses a new version of the logo (donated to the R Foundation by
         RStudio).  It is defined in .svg format, so will resize without
         unnecessary degradation when displayed on HTML pages-there is also
         a vector PDF version.  Thanks to Dirk Eddelbuettel for producing
         the corresponding X11 icon.

I tried running plot_ts-svg-1.R from macOS-10.12.6_R-3.4.1_xts-0.10-0_fails.zip and starts to create the mem_total.svg plot but throws an error before completion:

R> source("plot_ts-svg-1.R")
Error in segments(xlim[1], y_grid_lines(get_ylim()[[2]]), xlim[2], y_grid_lines(get_ylim()[[2]]),  : 
  cannot mix zero-length and non-zero-length coordinates
R> traceback()
13: segments(xlim[1], y_grid_lines(get_ylim()[[2]]), xlim[2], y_grid_lines(get_ylim()[[2]]), 
        col = theme$grid, lwd = grid.ticks.lwd, lty = grid.ticks.lty)
12: eval(aob, attr(aob, "env"))
11: eval(aob, attr(aob, "env"))
10: FUN(X[[i]], ...)
9: lapply(x$Env$actions, fo)
8: plot.replot_xts(x, ...)
7: plot(x, ...)
6: print.replot_xts(xts::plot.xts(data_xts[, name], main = name))
5: print(xts::plot.xts(data_xts[, name], main = name)) at plot_ts-svg-1.R#9
4: eval(ei, envir)
3: eval(ei, envir)
2: withVisible(eval(ei, envir))
1: source("plot_ts-svg-1.R")

I apologize for the delay in replying; we have been distracted in Houston the past few days.

Yes, I see the same thing when running on my Mac:

> source("plot_ts-svg-1.R")
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Error in segments(xlim[1], y_grid_lines(get_ylim()[[2]]), xlim[2], y_grid_lines(get_ylim()[[2]]),  : 
  cannot mix zero-length and non-zero-length coordinates
> source("plot_ts-svg-1.R")
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Error in segments(xlim[1], y_grid_lines(get_ylim()[[2]]), xlim[2], y_grid_lines(get_ylim()[[2]]),  : 
  cannot mix zero-length and non-zero-length coordinates
> traceback()
13: segments(xlim[1], y_grid_lines(get_ylim()[[2]]), xlim[2], y_grid_lines(get_ylim()[[2]]), 
        col = theme$grid, lwd = grid.ticks.lwd, lty = grid.ticks.lty)
12: eval(aob, env)
11: eval(aob, env)
10: FUN(X[[i]], ...)
9: lapply(x$Env$actions, function(aob) {
       if (attr(aob, "frame") > 0) {
           x$set_frame(attr(aob, "frame"), attr(aob, "clip"))
           env <- attr(aob, "env")
           if (is.list(env)) {
               env <- unlist(lapply(env, function(x) eapply(x, eval)), 
                   recursive = FALSE)
           }
           eval(aob, env)
       }
   })
8: plot.replot_xts(x, ...)
7: plot(x, ...)
6: print.replot_xts(xts::plot.xts(data_xts[, name], main = name))
5: print(xts::plot.xts(data_xts[, name], main = name)) at plot_ts-svg-1.R#9
4: eval(ei, envir)
3: eval(ei, envir)
2: withVisible(eval(ei, envir))
1: source("plot_ts-svg-1.R")

I hope it wasn't very bad for you in Houston...

I just re-ran your script using the HEAD of master and the .svg files appear to be created correctly. I think the issue was fixed in late July... I was able to replicate the error if I used the version of xts on CRAN (0.10-1), not the last commit on GitHub.

Please install the latest from GitHub and see if you still have any issues.

Thank you for asking. I had only minor water seepage in a corner of my house. Certainly, I have no complaint compared with what others have suffered.

I am on vacation this week, which is the reason for the delay in replying, but I had some free time today to work on this. I installed the latest version from GitHub by doing remotes::install_github("joshuaulrich/xts") on a RHEL6 system running R 3.4.1 and on the RHEL6 system running R 3.3.3 where all this started. The new version identifies itself as xts-0.10-1, and both systems have zoo-1.8.0. Here are the results. On the system running R 3.4.1, the test program for SVG files works, but the test program for PDF files generates a file that cannot be opened. On the system running R 3.3.3, the test program for SVG files fails with a core dump, and the test program for PDF files generates a file that cannot be opened. On both systems, restoring xts-0.9-7 allows both test programs to work without problem.

(blundered on the `Close and comment' button again)

This was caused by auto-printing being suppressed inside for loops. It happens with ggplot also. You need to explicitly call print(xts::plot(...)) for the plot to be rendered.