fastr icon indicating copy to clipboard operation
fastr copied to clipboard

Published 20 hours ago • verified publisher icon oracle

sprintf triggers FormatFlagsConversionMismatchException

Open rmartinsanta opened this issue 3 years ago • 2 comments

While debugging a possible bug in one R library, found a potential issue in the sprintf function. The formatter seems to not be a valid in Java, while being valid in R.

Minimum code to reproduce that works on standard R:

sprintf("%#15.10g", 2870.0)

Result in GraalVM:

❯ ./R --R.PrintErrorStacktracesToFile=true
R version 4.0.3 (FastR)
Copyright (c) 2013-21, Oracle and/or its affiliates
Copyright (C) 2020 The R Foundation for Statistical Computing
Copyright (c) 2012-4 Purdue University
All rights reserved.

FastR is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information.

Type 'q()' to quit R.
> sprintf("%#15.10g", 2870.0)
An internal error occurred: "java.util.FormatFlagsConversionMismatchException: Conversion = g, Flags = #"
Please report an issue at https://github.com/oracle/fastr including the commands and the error log file '/Users/rmartin/.sdkman/candidates/java/21.3.0.r17-grl/bin/fastr_errors_pid90947.log'.
> sessionInfo()
FastR version 4.0.3 (2020-10-10)
Platform: x86_64-apple-darwin20.6.0 (64-bit)
Running under: macOS  11.6

Matrix products: default
BLAS:   /Users/rmartin/.sdkman/candidates/java/21.3.0.r17-grl/languages/R/lib/libRblas.dylib
LAPACK: /Users/rmartin/.sdkman/candidates/java/21.3.0.r17-grl/languages/R/lib/libRlapack.dylib

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

Result in R:

❯ R

R version 4.1.1 (2021-08-10) -- "Kick Things"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin20.6.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> sprintf("%#15.10g", 2870.0)
[1] "    2870.000000"

Include the folloing info

  • If FastR produced fastr_errors_pid{xyz}.log or fastr_errors.log, attach them to the issue.
❯ cat /Users/rmartin/.sdkman/candidates/java/21.3.0.r17-grl/bin/fastr_errors_pid90947.log
Sun Nov 07 22:42:48 CET 2021
com.oracle.truffle.r.runtime.RInternalError: java.util.FormatFlagsConversionMismatchException: Conversion = g, Flags = #
	at com.oracle.truffle.r.nodes.function.FunctionDefinitionNode.execute(FunctionDefinitionNode.java:342)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:650)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:622)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:555)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:539)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callDirect(OptimizedCallTarget.java:481)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedDirectCallNode.call(OptimizedDirectCallNode.java:71)
	at com.oracle.truffle.r.nodes.function.call.CallRFunctionNode.execute(CallRFunctionNode.java:63)
	at com.oracle.truffle.r.nodes.function.RCallNode$DispatchedCallNode.execute(RCallNode.java:1233)
	at com.oracle.truffle.r.nodes.function.RCallNode$FunctionDispatch.dispatch(RCallNode.java:920)
	at com.oracle.truffle.r.nodes.function.RCallNodeGen$FunctionDispatchNodeGen.executeAndSpecialize(RCallNodeGen.java:745)
	at com.oracle.truffle.r.nodes.function.RCallNodeGen$FunctionDispatchNodeGen.execute(RCallNodeGen.java:708)
	at com.oracle.truffle.r.nodes.function.RCallNode.call(RCallNode.java:295)
	at com.oracle.truffle.r.nodes.function.RCallNodeGen.executeAndSpecialize(RCallNodeGen.java:236)
	at com.oracle.truffle.r.nodes.function.RCallNodeGen.execute(RCallNodeGen.java:211)
	at com.oracle.truffle.r.runtime.nodes.RNode.visibleExecute(RNode.java:74)
	at com.oracle.truffle.r.engine.REngine$AnonymousBodyNode.visibleExecute(REngine.java:741)
	at com.oracle.truffle.r.engine.REngine$AnonymousRootNode.execute(REngine.java:668)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:650)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:622)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:555)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:539)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callDirect(OptimizedCallTarget.java:481)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedDirectCallNode.call(OptimizedDirectCallNode.java:71)
	at com.oracle.truffle.r.engine.EngineRootNode$EngineBodyNode.execute(EngineRootNode.java:150)
	at com.oracle.truffle.r.engine.EngineRootNode.execute(EngineRootNode.java:88)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:650)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:622)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:555)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:539)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callIndirect(OptimizedCallTarget.java:463)
	at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.call(OptimizedCallTarget.java:444)
	at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.eval(PolyglotContextImpl.java:1218)
	at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextDispatch.eval(PolyglotContextDispatch.java:62)
	at org.graalvm.sdk/org.graalvm.polyglot.Context.eval(Context.java:379)
	at com.oracle.truffle.r.launcher.REPL.lambda$readEvalPrint$1(REPL.java:148)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.util.FormatFlagsConversionMismatchException: Conversion = g, Flags = #
	at java.base/java.util.Formatter$FormatSpecifier.failMismatch(Formatter.java:4438)
	at java.base/java.util.Formatter$FormatSpecifier.checkBadFlags(Formatter.java:3176)
	at java.base/java.util.Formatter$FormatSpecifier.checkFloat(Formatter.java:3187)
	at java.base/java.util.Formatter$FormatSpecifier.<init>(Formatter.java:2902)
	at java.base/java.util.Formatter.parse(Formatter.java:2747)
	at java.base/java.util.Formatter.format(Formatter.java:2671)
	at java.base/java.util.Formatter.format(Formatter.java:2625)
	at java.base/java.lang.String.format(String.java:4140)
	at com.oracle.truffle.r.nodes.builtin.base.Sprintf.processFormat(Sprintf.java:369)
	at com.oracle.truffle.r.nodes.builtin.base.Sprintf.format(Sprintf.java:318)
	at com.oracle.truffle.r.nodes.builtin.base.Sprintf.sprintf(Sprintf.java:131)
	at com.oracle.truffle.r.nodes.builtin.base.SprintfNodeGen.executeAndSpecialize(SprintfNodeGen.java:483)
	at com.oracle.truffle.r.nodes.builtin.base.SprintfNodeGen.executeObject(SprintfNodeGen.java:363)
	at com.oracle.truffle.r.nodes.builtin.base.Sprintf.sprintfOneElement(Sprintf.java:262)
	at com.oracle.truffle.r.nodes.builtin.base.SprintfNodeGen.execute(SprintfNodeGen.java:177)
	at com.oracle.truffle.r.nodes.builtin.RBuiltinNode$Arg2.call(RBuiltinNode.java:201)
	at com.oracle.truffle.r.nodes.builtin.InternalNode$InternalCallNode.execute(InternalNode.java:260)
	at com.oracle.truffle.r.runtime.nodes.RNode.visibleExecute(RNode.java:74)
	at com.oracle.truffle.r.nodes.function.FunctionBodyNode.visibleExecute(FunctionBodyNode.java:67)
	at com.oracle.truffle.r.nodes.function.FunctionDefinitionNode.execute(FunctionDefinitionNode.java:292)
	... 39 more
Caused by: Attached Guest Language Frames (3)

Frame(d=1): sprintf (called as: sprintf("%#15.10g", 2870))
Frame(d=0): <repl wrapper> (called as: <invalid call>)
<empty frame>

with frame slot contents:

Frame(d=1): sprintf (called as: sprintf("%#15.10g", 2870))
      fmt = [4, org.graalvm.compiler.truffle.runtime.FrameWithoutBoxing@69f461d3, expr=ConstantObjectNode@1bdfc2d2, %#15.10g]
      ... = RArgsValuesAndNames: null = [4, org.graalvm.compiler.truffle.runtime.FrameWithoutBoxing@69f461d3, expr=ConstantDoubleScalarNode@6ac95e2c, 2870.0]
      Visibility = true
Frame(d=0): <repl wrapper> (called as: <invalid call>)
      FunctionEvalCallNode-argsIdentifier = null
      FunctionEvalCallNode-funIdentifier = null
      Visibility = true
      .Random.seed = active binding
      RExplicitCall-argsIdentifier = null
<empty frame>
  • Use $GRAALVM_HOME/bin/R --vm.version and include the full output:
❯ ./R --vm.version
openjdk version "17.0.1" 2021-10-19
OpenJDK Runtime Environment GraalVM CE 21.3.0 (build 17.0.1+12-jvmci-21.3-b05)
OpenJDK 64-Bit Server VM GraalVM CE 21.3.0 (build 17.0.1+12-jvmci-21.3-b05, mixed mode, sharing)
  • Output of R built-in function sessionInfo(): Provided in the command output
  • OS name and version: MacOS Big Sur

rmartinsanta avatar Nov 07 '21 21:11 rmartinsanta

The bug seems to a discrepancy of the meaning of the '#' symbol in Java and C, relevant parts in bold.

Java

'g' | '\u0067' | Requires the output to be formatted in general scientific notation as described below. The localization algorithm is applied. After rounding for the precision, the formatting of the resulting magnitude m depends on its value. If m is greater than or equal to 10-4 but less than 10precision then it is represented in decimal format. If m is less than 10-4 or greater than or equal to 10precision, then it is represented in computerized scientific notation. The total number of significant digits in m is equal to the precision. If the precision is not specified, then the default value is 6. If the precision is 0, then it is taken to be 1. If the '#' flag is given then an FormatFlagsConversionMismatchException will be thrown. -- | -- | --

Source: https://docs.oracle.com/javase/7/docs/api/java/util/Formatter.html

C

The character % is followed by zero or more of the following flags: The value should be converted to an "alternate form". For o conversions, the first character of the output string is made zero (by prefixing a 0 if it was not zero already). For x and X conversions, a nonzero result has the string "0x" (or "0X" for X conversions) prepended to it. For a, A, e, E, f, F, g, and G conversions, the result will always contain a decimal point, even if no digits follow it (normally, a decimal point appears in the results of those conversions only if a digit follows). For g and G conversions, trailing zeros are not removed from the result as they would otherwise be. For other conversions, the result is undefined.

Source: https://linux.die.net/man/3/sprintf

rmartinsanta avatar Nov 07 '21 22:11 rmartinsanta

Hi @rmartinsanta, thank you for the precise description. We will look into this issue.

Unfortunately, these kinds of incompatibilities with standard R (GNU-R) are inevitable. In this case, it is just the difference between the standard formatter in Java and in C, but in some cases, GNU-R has corner cases that does not comply to any standard, e.g., GNU-R has its own regular expression engine that handles some cases in a way different from all other standard engines (see this PR https://github.com/r-lib/testthat/pull/1377)

Akirathan avatar Nov 16 '21 09:11 Akirathan