Use the RandomizedTesting library for repeatable randomized tests

[dsmiley]

Randomized tests need to be repeatable. Use this: http://labs.carrotsearch.com/randomizedtesting.html

Lucene/Solr uses this, the Spatial4j lib uses this, the SolrTextTagger uses this. It's wonderful and simple.