this-might-be-offensive comment search

Let's face it, comment search still sucks. I don't really want to deal with it, but it's a real problem so here's a bug to annoy me.

Sep 23 '11 05:09 numist

cor recommends checking out http://www.elasticsearch.org/

Nov 25 '11 04:11 numist

http://www.rallydev.com/engblog/2011/03/18/search-that-sizzles-experiences-transitioning-from-sql-like-to-solr/

Nov 25 '11 07:11 plytro

we found out tonight that xapian isn't in debian anymore. that makes this even more important.

I'm really liking what I'm seeing with elasticsearch, I'm going to see if it will work for us next week.

Jan 22 '12 07:01 numist

we use elasticsearch in our company now to search through hundreds of millions of log entries. It's got some scaling issues towards that size, but for tmbo it should be great.

Jan 29 '12 12:01 corbosman

not that it matters at our scale, but are you clustering?

also, in case anyone's curious:

mysql> SELECT COUNT(*) FROM offensive_comments WHERE comment <> "";
+----------+
| COUNT(*) |
+----------+
|  2344658 |
+----------+
1 row in set (4.88 sec)

Jan 29 '12 16:01 numist

I dont think we're clustering. I havent set it up but as far as I can see we have 1 machine (although a kickass one)

Jan 29 '12 16:01 corbosman

I can't make progress on this unless we get a memory upgrade or something. back to the back burner for now.

Feb 05 '12 22:02 numist

Oh hey. I might have fixed it. Take a look at b63a8486e0d30e72ae2b697ed372cdb0819adfbd. (I also handled empty results correctly with 7fb838c08174845d57068ff3d34737e1efb03114). Deployed to sandbox for testing.

Apr 12 '14 03:04 jaswope

XapianQueryParser::FLAG_DEFAULT was what was missing? D'oh.

Apr 22 '14 20:04 numist