spark-solr icon indicating copy to clipboard operation
spark-solr copied to clipboard

Error when using "sample_seed" parameter

Open shahanesanket opened this issue 5 years ago • 1 comments

CODE:

val collection="PM_signals"
val zkhost="zk:2181" // Replace this with your cluster zkhost
val opts = Map("collection" -> collection, "zkhost" -> zkhost, "sample_seed" -> "5150")
val df = spark.read.format("solr").options(opts).load
df.show

ERROR:

org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://172.18.0.10:8983/solr/PM_signals: sort param field can't be found: random_5150
  at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:656)
  at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
  at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:245)
  at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
  at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
  at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1128)
  at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:897)
  at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:829)
  at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1019)
  at com.lucidworks.spark.util.SolrQuerySupport$.getNumDocsFromSolr(SolrQuerySupport.scala:560)
  at com.lucidworks.spark.rdd.SolrRDD.calculateSplitsPerShard(SolrRDD.scala:92)
  at com.lucidworks.spark.rdd.SelectSolrRDD$$anonfun$1.apply$mcI$sp(SelectSolrRDD.scala:100)```

shahanesanket avatar Jan 14 '20 20:01 shahanesanket

You need to have these two in your managed schema

  <fieldType name="random" class="solr.RandomSortField" indexed="true"/>
  <dynamicField name="random_*" type="random"/>

kiranchitturi avatar Mar 15 '20 05:03 kiranchitturi