enju_leaf
enju_leaf copied to clipboard
next-l
TSVファイルからのインポートで書誌を変更できない
- [x] mdマニュアル修正の必要なし #1609
@nabeta #1514 で manifestation_id をキーにすれば、TSVファイルから書誌を更新できました。
しかし、久しぶりに Enju Leaf 1.3.5で manifestation_id と pub_date からなるTSVファイルをインポートしたところ、以下のメッセージが出て更新されずに終わってしまいます。
line n: 書誌インポート省略: すでに登録されている書誌が見つかりました。
何かバージョンアップで変わりましたでしょうか。
Originally posted by @miyatatk in https://github.com/next-l/enju_leaf/issues/1514#issuecomment-628655855
@nabeta はい、「更新」を選択しています。文字コードは明示的に「UTF-8」を指定しています。
いろいろ試してみると、「line 2: バリデーションに失敗しました: 内容はすでに存在します」というエラーも出ます。
エラーメッセージは以下の通りです。
Exception
ActiveRecord::RecordInvalid
Error
バリデーションに失敗しました: 内容はすでに存在します
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/validations.rb:80:in `raise_validation_error'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/validations.rb:52:in `save!'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:315:in `block in save!'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:212:in `transaction'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:315:in `save!'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/suppressor.rb:48:in `save!'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/enju_biblio-0.3.12/app/models/resource_import_file.rb:786:in `block (3 levels) in fetch'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/enju_biblio-0.3.12/app/models/resource_import_file.rb:784:in `each'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/enju_biblio-0.3.12/app/models/resource_import_file.rb:784:in `block (2 levels) in fetch'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:212:in `transaction'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/enju_biblio-0.3.12/app/models/resource_import_file.rb:781:in `block in fetch'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `block in transaction'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/usr/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `transaction'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activerecord-5.2.6/lib/active_record/transactions.rb:212:in `transaction'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/enju_biblio-0.3.12/app/models/resource_import_file.rb:680:in `fetch'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/enju_biblio-0.3.12/app/models/resource_import_file.rb:353:in `block in modify'
/usr/lib/ruby/2.5.0/csv.rb:1764:in `each'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/enju_biblio-0.3.12/app/models/resource_import_file.rb:322:in `modify'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/enju_biblio-0.3.12/app/models/resource_import_file.rb:51:in `import_start'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/enju_biblio-0.3.12/app/jobs/resource_import_file_job.rb:5:in `perform'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/execution.rb:39:in `block in perform_now'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/callbacks.rb:109:in `block in run_callbacks'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/i18n-1.8.10/lib/i18n.rb:314:in `with_locale'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/callbacks.rb:118:in `instance_exec'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/logging.rb:26:in `block (4 levels) in <module:Logging>'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/notifications.rb:168:in `block in instrument'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/notifications.rb:168:in `instrument'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/logging.rb:25:in `block (3 levels) in <module:Logging>'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/logging.rb:46:in `block in tag_logger'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/tagged_logging.rb:71:in `block in tagged'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/tagged_logging.rb:28:in `tagged'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/tagged_logging.rb:71:in `tagged'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/logging.rb:46:in `tag_logger'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/logging.rb:22:in `block (2 levels) in <module:Logging>'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/callbacks.rb:118:in `instance_exec'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/callbacks.rb:136:in `run_callbacks'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/execution.rb:38:in `perform_now'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/execution.rb:24:in `block in execute'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/callbacks.rb:109:in `block in run_callbacks'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/railtie.rb:28:in `block (4 levels) in <class:Railtie>'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/execution_wrapper.rb:87:in `wrap'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/reloader.rb:73:in `block in wrap'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/execution_wrapper.rb:87:in `wrap'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/reloader.rb:72:in `wrap'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/railtie.rb:27:in `block (3 levels) in <class:Railtie>'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/callbacks.rb:118:in `instance_exec'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/callbacks.rb:118:in `block in run_callbacks'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activesupport-5.2.6/lib/active_support/callbacks.rb:136:in `run_callbacks'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/execution.rb:22:in `execute'
/home/USER/enju_leaf_13/vendor/bundle/ruby/2.5.0/gems/activejob-5.2.6/lib/active_job/queue_adapters/resque_adapter.rb:47:in `perform'
【解決】原因がわかりました。
様々なTSVファイルを作ってインポートの試行錯誤を行った結果、以下の列（データ項目）が含まれるTSVファイルをインポートすると、バリデーションエラーが起こることがわかりました。（バリデーションエラーの原因となったデータ項目を出力してくれるとありがたいです。）
isbn
jpno
ncid
現状では、Enjuのエクスポート機能で出力したTSVファイルをそのままインポートすると、エラーすることになります。注意が必要です。
また、以下の列（データ項目）がある場合インポートは失敗しませんが、無視されることがわかりました。これには価格、ユーザー定義の識別子、シリーズ情報、メモが含まれており、バッチ処理で変更したいこともあるでしょうから、無視しないで読み込んでいただけるとありがたいです。
year_of_publication
manifestation_created_at
manifestation_updated_at
price
manifestation_required_role
identifier:isbn
identifier:issn
identifier:jpno
identifier:doi
identifier:iss_itemno
identifier:lccn
identifier:loc_identifier
identifier:ncid
series_statement_id
series_statement_original_title
series_statement_title_subseries
series_statement_title_subseries_transcription
series_statement_title_transcription
series_statement_creator
series_statement_volume_number
series_statement_series_master
series_statement_root_manifestation_id
series_statement_manifestation_id
series_statement_position
series_statement_note
series_statement_created_at
series_statement_updated_at
memo
以下の列（データ項目）が含まれるTSVファイルをインポートすると、バリデーションエラーが起こることがわかりました。
@miyatatk まずこちらは、9月前半までに修正版をリリース予定です。