'make check' fails after the recent update on 2022/07/11

[heons]

Hi!

We were building the libpostal in a docker image and it suddenly started failing on make check step of the following Dockerfile after the recent change on 2022/07/11 This PR: https://github.com/openvenues/libpostal/pull/578/files

---

My country is

Canada

---

Here's how I'm using libpostal

Geocoding, address parsing.

---

Here's what I did

FROM golang:1.12

# libpostal apt dependencies
# note: this is done in one command in order to keep down the size of intermediate containers
RUN apt-get update && \
    apt-get install -y autoconf automake libtool pkg-config python && \
    rm -rf /var/lib/apt/lists/*

# install libpostal
RUN git clone https://github.com/openvenues/libpostal /code/libpostal
WORKDIR /code/libpostal
# RUN git checkout a97717f2b9f8fba03d25442f2bd88c15e86ec81b <--2022/04/18 ver. It is working.
RUN ./bootstrap.sh

RUN ./configure --datadir=/usr/share/libpostal
RUN make -j4
# it failed at this stage -------------------
RUN make check
RUN make install
RUN ldconfig

---

Here's what I got

root@e256a47994ae:/code/libpostal# make check
Making check in src
make[1]: Entering directory '/code/libpostal/src'
./libpostal_data download all /usr/share/libpostal/libpostal
Checking for new libpostal data file...
libpostal data file up to date
Checking for new libpostal parser data file...
libpostal parser data file up to date
Checking for new libpostal language classifier data file...
libpostal language classifier data file up to date
make[1]: Leaving directory '/code/libpostal/src'
Making check in test
make[1]: Entering directory '/code/libpostal/test'
make  check-TESTS
make[2]: Entering directory '/code/libpostal/test'
make[3]: Entering directory '/code/libpostal/test'
../test-driver: line 107:   350 Aborted                 (core dumped) "$@" > $log_file 2>&1
FAIL: test_libpostal
============================================================================
Testsuite summary for libpostal 1.1.0
============================================================================
# TOTAL: 1
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See test/test-suite.log
============================================================================
make[3]: *** [Makefile:1001: test-suite.log] Error 1
make[3]: Leaving directory '/code/libpostal/test'
make[2]: *** [Makefile:1109: check-TESTS] Error 2
make[2]: Leaving directory '/code/libpostal/test'
make[1]: *** [Makefile:1182: check-am] Error 2
make[1]: Leaving directory '/code/libpostal/test'
make: *** [Makefile:459: check-recursive] Error 1
root@e256a47994ae:/code/libpostal# 

---

Here's what I was expecting

---

For parsing issues, please answer "yes" or "no" to all that apply.

• Does the input address exist in OpenStreetMap?
• Do all the toponyms exist in OSM (city, state, region names, etc.)?
• If the address uses a rare/uncommon format, does changing the order of the fields yield the correct result?
• If the address does not contain city, region, etc., does adding those fields to the input improve the result?
• If the address contains apartment/floor/sub-building information or uncommon formatting, does removing that help? Is there any minimum form of the address that gets the right parse?

---

Here's what I think could be improved

[missinglink]

Are you using docker build or docker buildx?

[heons]

@missinglink I'm using docker build

[koertkuipers]

i am seeing same thing building libpostal on my ubuntu 22.04 laptop (no docker). error is:

* Suite libpostal_trie_tests:                                                                                                                                         
.free(): invalid next size (fast)                                                                                                                                     
FAIL test_libpostal (exit status: 134)

[brianmacy]

Turn off sse2. It creates access violations when enabled.

[pymaxion]

I am facing this same issue and seeing the same error as @koertkuipers. Disabling sse2 (via ./configure --disable-sse2 && make) unfortunately does not change the result.

Reverting libpostal to a version before PR #578 was merged (e.g., git reset --hard a97717f2b9f8fba03d25442f2bd88c15e86ec81b) allows make check to pass, but this is obviously not a good long-term solution.

[missinglink]

I can confirm this issue on an x86 Ubuntu 18.04.3 machine. More specifically make check fails in libpostal_trie_tests:

* Suite libpostal_trie_tests:
.free(): invalid next size (fast)

cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 158
model name	: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
stepping	: 13
microcode	: 0xde
cpu MHz		: 800.031
cache size	: 16384 KB
physical id	: 0
siblings	: 16
core id		: 0
cpu cores	: 8
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 22
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d arch_capabilities
bugs		: spectre_v1 spectre_v2 spec_store_bypass swapgs taa itlb_multihit
bogomips	: 7200.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:
...

[missinglink]

Worth adding that compiling & make check succeed on an modern M2 Mac (with an ARM 8.5-A chipset).

[brianmacy]

If it is the same issue as with SSE, it is likely just random. It will only error if it crosses a page boundary into an area not allocated for the process or if the memory heap gets overwritten in a way free/malloc notices.

A great thing to try is the gcc sanitize capabilities.

On Wed, Jul 5, 2023 at 06:52 Peter Johnson @.***> wrote:

Worth adding that compiling & make check succeed on an modern M2 Mac (with an ARM 8.5-A chipset).

— Reply to this email directly, view it on GitHub https://github.com/openvenues/libpostal/issues/592#issuecomment-1621514223, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF6OZVBLPGYOEFQK4LEBVWLXOVBPHANCNFSM53P3GZ4Q . You are receiving this because you commented.Message ID: @.***>

[missinglink]

This issue should be resolved by https://github.com/openvenues/libpostal/pull/632, please confirm

[heons]

Yes. I confirmed this is has resolved. Thank you! 👍 😄