libfyaml icon indicating copy to clipboard operation
libfyaml copied to clipboard

failing test testsuite-json.test 256 UGM3 with jq 1.7

Open MarDiehl opened this issue 9 months ago • 4 comments

when building the AUR package, I encounter a failing test (testsuite-json.test 256 UGM3). Log file is attached.

There are also a few compiler warning:

Making all in src
  CC    tool/fy_tool-fy-tool.o
  CC    lib/fy-parse.o
  CC    lib/fy-types.o
  CC    lib/fy-diag.o
lib/fy-diag.c: In function 'fy_diag_error_atom_display':
lib/fy-diag.c:755:51: warning: 'cols' may be used uninitialized [-Wmaybe-uninitialized]
  755 |                         while (col8 < (line_shift + cols) && (c = fy_utf8_get(s, (e - s), &w)) >= 0 && rbs < rbe) {
      |                                       ~~~~~~~~~~~~^~~~~~~
lib/fy-diag.c:583:19: note: 'cols' was declared here
  583 |         int pass, cols, min_col, max_col, max_line_count, max_line_col8, max_width;
      |                   ^~~~
lib/fy-diag.c:746:29: warning: 'rowbufsz' may be used uninitialized [-Wmaybe-uninitialized]
  746 |                         rbe = rowbuf + rowbufsz;
      |                         ~~~~^~~~~~~~~~~~~~~~~~~
lib/fy-diag.c:594:16: note: 'rowbufsz' was declared here
  594 |         size_t rowbufsz;
      |                ^~~~~~~~
  CC    lib/fy-dump.o
  CC    lib/fy-atom.o
  CC    lib/fy-token.o
  CC    lib/fy-input.o
  CC    lib/fy-docstate.o
  CC    lib/fy-doc.o
  CC    lib/fy-docbuilder.o
  CC    lib/fy-emit.o
  CC    lib/fy-event.o
  CC    lib/fy-accel.o
  CC    lib/fy-walk.o
  CC    lib/fy-path.o
  CC    lib/fy-composer.o
lib/fy-composer.c:348:1: warning: conflicting types for 'fy_composer_process_event' due to enum/integer mismatch; have 'enum fy_composer_return(struct fy_composer *, struct fy_event *)' [-Wenum-int-mismatch]
  348 | fy_composer_process_event(struct fy_composer *fyc, struct fy_event *fye)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~
In file included from lib/fy-parse.h:40,
                 from lib/fy-composer.c:21:
lib/fy-composer.h:51:5: note: previous declaration of 'fy_composer_process_event' with type 'int(struct fy_composer *, struct fy_event *)'
   51 | int fy_composer_process_event(struct fy_composer *fyc, struct fy_event *fye);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~
  CC    xxhash/xxhash.o
  CC    util/fy-ctype.o
  CC    util/fy-utf8.o
  CC    util/fy-utils.o
  CC    util/fy-blob.o
  CC    thread/fy-thread.o
  CC    blake3/blake3_host_state.o
  CC    blake3/blake3_backend.o
  CC    blake3/blake3_be_cpusimd.o
  CC    blake3/fy-blake3.o
  CC    blake3/blake3_portable.o
  CC    blake3/blake3.o
  LINK  libb3portable.la
  CC    blake3/blake3_sse2.o
  AS    blake3/blake3_sse2_x86-64_unix.o
  CC    blake3/blake3.o
  LINK  libb3sse2.la
  CC    blake3/blake3_sse41.o
  AS    blake3/blake3_sse41_x86-64_unix.o
  CC    blake3/blake3.o
  LINK  libb3sse41.la
  CC    blake3/blake3_avx2.o
  AS    blake3/blake3_avx2_x86-64_unix.o
  CC    blake3/blake3.o
  LINK  libb3avx2.la
  CC    blake3/blake3_avx512.o
  AS    blake3/blake3_avx512_x86-64_unix.o
  CC    blake3/blake3.o
  LINK  libb3avx512.la
  LINK  libfyaml.la
  LINK  fy-tool
  CC    internal/libfyaml_parser-libfyaml-parser.o
  LINK  libfyaml-parser
  CC    internal/fy_thread-fy-thread.o
  LINK  fy-thread
  CC    internal/fy_b3sum-fy-b3sum.o
  LINK  fy-b3sum

test-suite.log

MarDiehl avatar Sep 25 '23 15:09 MarDiehl

the error is

--- /tmp/tmp.3VOEwfva1W 2023-09-26 10:45:04.296399978 +0200                                         
+++ /tmp/tmp.nIOU0yBdZ3 2023-09-26 10:45:04.299733412 +0200                                         
@@ -15,13 +15,13 @@                                                                                 
   "product": [                                                                                     
     {                                                                                              
       "description": "Basketball",                                                                 
-      "price": 450.00,                                                                             
+      "price": 450,                                                                                
       "quantity": 4,                                                                               
       "sku": "BL394D"                                                                              
     },                                                                                             
     {                                                                                              
       "description": "Super Hoop",                                                                 
-      "price": 2392.00,                                                                            
+      "price": 2392,                                                                               
       "quantity": 1,                                                                               
       "sku": "BL4438H"                                                                             
     }                                                                                              
not ok 256 UGM3 - Spec Example 2.27. Invoice (JSON)                                                 
FAIL: testsuite-json.test 256 UGM3 - Spec Example 2.27. Invoice (JSON)   

so somehow floats with trailing zeros are changed to integers.

I only see this error if jq (https://jqlang.github.io/jq/) is installed. The version that fails is 1.7 from https://archlinux.org/packages/extra/x86_64/jq/

MarDiehl avatar Sep 26 '23 09:09 MarDiehl

The issue seems to be related to the new jq release (https://github.com/jqlang/jq/releases/tag/jq-1.7). the test fails for libfyaml 0.8 and 0.9

MarDiehl avatar Sep 26 '23 09:09 MarDiehl

Yes, I would guess this would be a problem. I need to think about how to control jq output.

pantoniou avatar Oct 21 '23 17:10 pantoniou

The expected JSON file contains integers, the yaml file doesn't:

  "product": [
    {
      "sku": "BL394D",
      "quantity": 4,
      "description": "Basketball",
      "price": 450
    },
    {
      "sku": "BL4438H",
      "quantity": 1,
      "description": "Super Hoop",
      "price": 2392
    }
  ],
product:
    - sku         : BL394D
      quantity    : 4
      description : Basketball
      price       : 450.00
    - sku         : BL4438H
      quantity    : 1
      description : Super Hoop
      price       : 2392.00

This looks like a problem with the test itself. Libfyaml behaves correctly by not changing the type -- at least that's what I'd expect. Maybe jq converted floats to ints during canonicalization, which has been changed in newer versions? That would mask that the difference in outputs was always present. I'm running into the same failure with jq 1.6, btw.

E: It's probably this PR 1752. From the changes to the manual:

jq will also try to maintain the original decimal precision of the provided number literal. See below for examples. [...]

         - program: 'map([., . == 1]) | tojson'
           input: '[1, 1.000, 1.0, 100e-2]'
           output: ['"[[1,true],[1.000,true],[1.0,true],[1.00,true]]"']

rwirth avatar Dec 14 '23 14:12 rwirth