kmeans-postgresql icon indicating copy to clipboard operation
kmeans-postgresql copied to clipboard

signal 11: Segmentation fault created when certain values are passed into kmeans

Open pthough opened this issue 11 years ago • 3 comments

First off, I'd like to say that kmeans is spectacular and is doing an awesome job at clustering our data points! We are using kmeans to cluster coordinates on Google Maps and have come across a rather odd error. It seems that whenever we pass in exactly 40 or 44 points to the kmeans function and ask it to cluster based on 65 or greater points it will force a complete shutdown of Postgres, killing any and all active queries. Here's an output of the log: 2014-02-04 16:03:49 UTC LOG: statement: with sites as ( select latitude, longitude, testsite_id from locator.testsites order by longitude desc limit 40 ), clusterd as ( select kmeans,count(*) ct,array_agg(testsite_id) as site_id FROM ( select kmeans(array[longitude,latitude], 75) over (), testsite_id from sites ) as wakka group by kmeans )

    select * from clusterd

2014-02-04 16:03:49 UTC LOG: server process (PID 11900) was terminated by signal 11: Segmentation fault 2014-02-04 16:03:49 UTC LOG: terminating any other active server processes 2014-02-04 16:03:49 UTC WARNING: terminating connection because of crash of another server process 2014-02-04 16:03:49 UTC DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2014-02-04 16:03:49 UTC HINT: In a moment you should be able to reconnect to the database and repeat your command. 2014-02-04 16:03:49 UTC LOG: all server processes terminated; reinitializing 2014-02-04 16:03:49 UTC LOG: database system was interrupted; last known up at 2014-02-04 16:02:24 UTC 2014-02-04 16:03:49 UTC LOG: database system was not properly shut down; automatic recovery in progress

The oddest thing that I find about this is the query returns perfectly fine when the limit is decreased to any other number or increased to any other number (aside from 44, that seems to be the other magical number). Since kmeans is doing such a fantastic job outside of this strange oddity we're currently working around it by simply adjusting the cluster count based on the number of sites passed in, but I thought I'd bring this to your attention.

Please let me know if you need any more information from me!

EDIT : Forgot to mention, we're running PostgreSQL 9.1.11 on Ubuntu 12.04.4 LTS and can replicate this error 100% of the time on PostgreSQL 9.1.10 and Ubuntu 12.04.3 LTS (haven't tried it on any other major versions)

Thanks, Paul

pthough avatar Feb 04 '14 16:02 pthough

+1

thiagomata avatar Aug 19 '15 19:08 thiagomata

+1

marcosmlopes avatar Aug 19 '15 19:08 marcosmlopes

I have seen this issue as well. In our case it occurred when we passed in 2 points and asked for a cluster of 2000 points. Any other value worked as expected.

I note the following in logs when this happens:

*** glibc detected *** postgres: postgres nautoguide 172.16.0.150(54899) SELECT: corrupted double-linked list: 0x00007f6740b60a20 ***

citizenfish avatar Mar 11 '16 10:03 citizenfish