spark-solr
spark-solr copied to clipboard
Error when using "sample_seed" parameter
CODE:
val collection="PM_signals"
val zkhost="zk:2181" // Replace this with your cluster zkhost
val opts = Map("collection" -> collection, "zkhost" -> zkhost, "sample_seed" -> "5150")
val df = spark.read.format("solr").options(opts).load
df.show
ERROR:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://172.18.0.10:8983/solr/PM_signals: sort param field can't be found: random_5150
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:656)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:245)
at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1128)
at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:897)
at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:829)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1019)
at com.lucidworks.spark.util.SolrQuerySupport$.getNumDocsFromSolr(SolrQuerySupport.scala:560)
at com.lucidworks.spark.rdd.SolrRDD.calculateSplitsPerShard(SolrRDD.scala:92)
at com.lucidworks.spark.rdd.SelectSolrRDD$$anonfun$1.apply$mcI$sp(SelectSolrRDD.scala:100)```
You need to have these two in your managed schema
<fieldType name="random" class="solr.RandomSortField" indexed="true"/>
<dynamicField name="random_*" type="random"/>