spotlight

spotlight copied to clipboard

With a new deployment, I am unable to upload images neither single file or by CSV. The files are uploaded to the public/uploads/spotlight/featured_image/ directory and the rows are added to the spotlight_resources table. They are not however being indexed into Solr. A reindexing does not work either.

Everything worked with a previousl deployment a few months back. No changes have been made in the deployment process.

cat config/blacklight.yml

# = jetty_path key
# each environment can have a jetty_path with absolute or relative
# (to app root) path to a jetty/solr install. This is used
# by the rake tasks that start up solr automatically for testing
# and by rake solr:marc:index.  
#
# jetty_path is not used by a running Blacklight application
# at all. In general you do NOT need to deploy solr in Jetty, you can deploy it
# however you want.  
# jetty_path is only required for rake tasks that need to know
# how to start up solr, generally for automated testing. 

development:
  adapter: solr
  url: <%= ENV['SOLR_URL'] || "http://127.0.0.1:8983/solr/blacklight-core" %>
test: &test
  adapter: solr
  url: <%= ENV['SOLR_URL'] || "http://127.0.0.1:#{ENV['TEST_JETTY_PORT'] || 8983}/solr/blacklight-core" %>
production:
  adapter: solr
  url: <%= ENV['SOLR_URL'] || "http://127.0.0.1:8983/solr/blacklight-core" %>

echo $SOLR_URL

http://janus.evans.tamu.edu:8983/solr/blacklight-core

curl http://janus.evans.tamu.edu:8983/solr/blacklight-core/admin/ping

<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><null name="zkConnected"/><int name="status">0</int><int name="QTime">1</int><lst name="params"><str name="distrib">false</str><str name="ps">2</str><str name="spellcheck.dictionary">default</str><str name="echoParams">all</str><str name="subject_qf">
        subject_tesim
      </str><str name="fl">
        *, score
      </str><str name="tie">0.01</str><str name="defType">edismax</str><str name="qf">
        id full_title_tesim short_title_tesim alternative_title_tesim active_fedora_model_ssi title_tesim author_tesim subject_tesim all_text_timv
      </str><str name="title_qf">
        title_tesim full_title_tesim short_title_tesim alternative_title_tesim
      </str><str name="author_pf"/><str name="mm">2&lt;-1 5&lt;-2 6&lt;90%</str><str name="q.alt">*:*</str><str name="qs">1</str><arr name="facet.field"><str>active_fedora_model_ssi</str><str>subject_ssim</str></arr><str name="subject_pf"/><str name="spellcheck.extendedResults">true</str><str name="q">{!lucene}*:*</str><str name="facet.limit">10</str><str name="spellcheck">true</str><str name="pf">
        all_text_timv^10
      </str><str name="title_pf"/><str name="spellcheck.onlyMorePopular">true</str><str name="author_qf">
        author_tesim
      </str><str name="facet.mincount">1</str><str name="spellcheck.count">5</str><str name="facet">true</str><str name="spellcheck.collate">false</str></lst></lst><str name="status">OK</str>
</response>

gem list

*** LOCAL GEMS ***

actioncable (5.0.4, 5.0.1)
actionmailer (5.0.4, 5.0.1)
actionpack (5.0.4, 5.0.1)
actionview (5.0.4, 5.0.1)
activejob (5.0.4, 5.0.1)
activemodel (5.0.4, 5.0.1)
activerecord (5.0.4, 5.0.1)
activesupport (5.0.4, 5.0.1)
acts-as-taggable-on (4.0.0)
addressable (2.5.1)
almond-rails (0.0.3)
arel (7.1.4)
autoprefixer-rails (7.1.2.1)
babel-source (5.8.35)
babel-transpiler (0.7.0)
bcrypt (3.1.11)
bigdecimal (default: 1.2.8)
bindex (0.5.0)
blacklight (6.10.1)
blacklight-gallery (0.8.0)
blacklight-oembed (0.1.0)
bootstrap-sass (3.3.7)
bootstrap_form (2.7.0)
breadcrumbs_on_rails (2.3.1)
builder (3.2.3)
bundler (1.15.1)
bundler-unload (1.0.2)
byebug (9.0.6)
cancancan (2.0.0)
carrierwave (1.1.0)
charlock_holmes (0.7.3)
clipboard-rails (1.7.1)
coffee-rails (4.2.2)
coffee-script (2.4.1)
coffee-script-source (1.12.2)
concurrent-ruby (1.0.5)
declarative (0.0.9)
declarative-option (0.1.0)
deprecation (1.0.0)
devise (4.3.0)
devise-guests (0.6.0)
devise_invitable (1.7.2)
did_you_mean (1.0.0)
diffy (3.2.0)
erubis (2.7.0)
escape_utils (1.1.1)
execjs (2.7.0)
executable-hooks (1.3.2)
faraday (0.12.1)
faraday_middleware (0.11.0.1)
ffi (1.9.18)
friendly_id (5.2.1)
gem-wrappers (1.2.7)
github-linguist (5.0.11)
github-markup (1.6.0)
globalid (0.4.0)
i18n (0.8.6)
iiif-presentation (0.2.0)
iiif_manifest (0.2.0)
io-console (default: 0.4.5)
jbuilder (2.7.0)
jquery-rails (4.3.1)
json (2.1.0, default: 1.8.3)
jwt (1.5.6)
kaminari (1.0.1)
kaminari-actionview (1.0.1)
kaminari-activerecord (1.0.1)
kaminari-core (1.0.1)
legato (0.7.0)
listen (3.0.8)
loofah (2.0.3)
mail (2.6.6)
method_source (0.8.2)
mime-types (3.1)
mime-types-data (3.2016.0521)
mini_magick (4.8.0)
mini_portile2 (2.2.0)
minitest (5.10.2, 5.8.3)
multi_json (1.12.1)
multi_xml (0.6.0)
multipart-post (2.0.0)
net-telnet (0.1.1)
nio4r (2.1.0, 1.2.1)
nokogiri (1.8.0)
oauth2 (1.4.0)
openseadragon (0.3.3)
orm_adapter (0.5.0)
paper_trail (5.2.3)
pg (0.21.0)
power_assert (0.2.6)
psych (default: 2.0.17)
public_suffix (2.0.5)
puma (3.9.1)
rack (2.0.3)
rack-test (0.6.3)
rails (5.0.4, 5.0.1)
rails-dom-testing (2.0.3)
rails-html-sanitizer (1.0.3)
railties (5.0.4, 5.0.1)
rake (12.0.0, 10.4.2)
rb-fsevent (0.10.2)
rb-inotify (0.9.10)
rdoc (default: 4.2.1)
redcarpet (3.4.0)
representable (3.0.4)
request_store (1.3.2)
responders (2.4.0)
retriable (3.0.2)
riiif (1.4.4)
roar (1.1.0)
roar-rails (1.1.0)
rsolr (2.0.2)
ruby-oembed (0.12.0)
ruby-progressbar (1.8.1)
rubygems-bundler (1.4.4)
rubyzip (1.2.1)
rugged (0.26.0)
rvm (1.11.3.9)
sass (3.4.25)
sass-rails (5.0.6)
signet (0.7.3)
sir_trevor_rails (0.5.1)
sitemap_generator (5.3.1)
solr_wrapper (1.1.0)
spring (2.0.2)
spring-watcher-listen (2.0.1)
sprockets (3.7.1)
sprockets-es6 (0.9.2)
sprockets-rails (3.2.0)
sqlite3 (1.3.13)
test-unit (3.1.5)
test_xml (0.1.8)
thor (0.19.4)
thread_safe (0.3.6)
tilt (2.0.7)
tinymce-rails (4.6.4)
tophat (2.3.1)
turbolinks (5.0.1)
turbolinks-source (5.0.3)
twitter-text (1.14.7)
twitter-typeahead-rails (0.11.1.pre.corejavascript)
tzinfo (1.2.3)
uber (0.1.0)
uglifier (3.2.0)
underscore-rails (1.8.3)
unf (0.1.4)
unf_ext (0.0.7.4)
warden (1.2.7)
web-console (3.5.1)
websocket-driver (0.6.5)
websocket-extensions (0.1.2)

ruby -v

ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-linux]

Workaround:

gem uninstall --force riiif
gem uninstall --force rsolr

Gemfile

-gem 'rsolr', '>= 1.0'
+gem 'rsolr', '1.1.2'

-gem 'riiif', '~> 1.0'
+gem 'riiif', '1.4.0'

rm -rf Gemfile.lock
bundle install

Might be able to get away with bundle update

@wwelling ah, interesting. It looks like there may be a compatibility issue with the more recent versions of one of those gems. Did you receive any errors messages in the logs when trying to upload images or re-index content? We may need to update Spotlight's gemspec (if it's not immediately obvious why the newer versions fail).

@jkeck no error messages in logs, which made it a bit more difficult to debug. My last effort was to do a gem list diff to a running Spotlight instance. There where other gem version differences, but these two are the only required to specify version to allow Solr indexing functionality to work.

Riiif gives a deprecation warning during an upload to /upload_resources:

DEPRECATION WARNING: Initializing a file resolver without setting the base path is deprecated and will be removed in Riiif 2.0. (called from new at /Users/njfranck/git/spotlight_test/config/initializers/riiif.rb:1

and also the upload controller seems to be broken:

Started POST "/spotlight/test3/upload_resources" for 127.0.0.1 at 2017-11-13 11:14:54 +0100
..
  
Rack::QueryParser::ParameterTypeError (expected Hash (got Array) for param `data'):
  
rack (2.0.3) lib/rack/query_parser.rb:112:in `normalize_params'

Not sure whether this is related..

Hi. I ran into the same issue. Any updates?

FWIW, I just tested this on the latest Spotlight master and was able to upload an item w/o any issues.

It may be worthwhile to test this using a different background job queuing framework (like sidekiq) as that may be able to give you some better logging capabilities from the job.