sequencescape
sequencescape copied to clipboard
Published
20 hours ago •
sanger
sanger
DPL-397 [BUG] As a lab user I would like improved error handing on QC file upload to make parsing errror more visible
Describe the bug If a QC file is uploaded with invalid data, this information is incorrectly propagated back to the user, resulting in a 500 from both Limber and Sequencescape. Ideally we should still store the uploaded QC file (in case it was never actually intended to be parsed) but return a clear error message to the user.
RT Ticket Number If applicable
To Reproduce Steps to reproduce the behaviour:
- Upload an invalid CSV file (Example on RT ticket) via the file upload feature in Limber
The plate in the RT ticket was a 'LCA PBMC' plate. I don't think it hugely matters, but it may be useful information.
Expected behaviour
- The user should receive a clear message explaining why their file failed to parse, ideally with a line number.
- We should probably still persist the uploaded file, as we support arbitrary file upload.
- Partial processing of invalid uploads is risky, as it could result in a scenario where the error was ignored, and the plate was processed with incomplete data.
Screenshots If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- Windows or Mac?
- Browser Chrome, Firefox, Safari or other?
- Browser version (use 'About' to look up)?
Additional context
Sequencescape exception
A Core::Registry::UnregisteredError occurred in #:
Unable to locate for "QcResult" ({"CustomMetadatumCollection"=>"Io::CustomMetadatumCollection", "BarcodePrinter"=>"Io::BarcodePrinter", "ActiveRecord::Relation"=>"Io::ActiveRecord::Relation", "Plate"=>"Io::Plate", "Well"=>"Io::Well", "ActiveRecord::AssociationRelation"=>"Io::ActiveRecord::Relation", "Aliquot"=>"Io::Aliquot", "Sample"=>"Io::Sample", "Search"=>"Io::Search", "Search::FindAssetByBarcode"=>"Io::Search", "Search::FindModelByName"=>"Io::Search", "SampleTube"=>"Io::SampleTube", "BaitLibrary"=>"Io::BaitLibrary", "PlateCreation"=>"Io::PlateCreation", "User"=>"Io::User", "PlatePurpose"=>"Io::PlatePurpose", "TransferRequestCollection"=>"Io::TransferRequestCollection", "TransferRequest"=>"Io::TransferRequest", "AssetAudit"=>"Io::AssetAudit", "Search::FindQcableByBarcode"=>"Io::Search", "TransferTemplate"=>"Io::TransferTemplate", "Transfer::BetweenPlates"=>"Io::Transfer::BetweenPlates", "Array"=>"Io::Array", "DilutionPlate"=>"Io::Plate", "WorkingDilutionPlate"=>"Io::Plate", "SubmissionPool"=>"Io::SubmissionPool", "StateChange"=>"Io::StateChange", "TagLayoutTemplate"=>"Io::TagLayoutTemplate", "TagGroup"=>"Io::TagGroup", "PooledPlateCreation"=>"Io::PooledPlateCreation", "SpecificTubeCreation"=>"Io::SpecificTubeCreation", "LibraryTube"=>"Io::LibraryTube", "SpikedBuffer"=>"Io::LibraryTube", "StockMultiplexedLibraryTube"=>"Io::StockMultiplexedLibraryTube", "PlateConversion"=>"Io::PlateConversion", "Qcable"=>"Io::Qcable", "LotType"=>"Io::LotType", "WorkCompletion"=>"Io::WorkCompletion", "PlatePurpose::Input"=>"Io::PlatePurpose", "PlatePurpose::InitialPurpose"=>"Io::PlatePurpose", "QcablePlatePurpose"=>"Io::PlatePurpose", "QcableLibraryPlatePurpose"=>"Io::PlatePurpose", "DilutionPlatePurpose"=>"Io::DilutionPlatePurpose", "QcFile"=>"Io::QcFile", "Submission"=>"Io::Submission", "Order"=>"Io::Order", "LinearSubmission"=>"Io::Order", "Receptacle"=>"Io::Receptacle", "RequestType"=>"Io::RequestType", "Transfer::BetweenTubesBySubmission"=>"Io::Transfer::BetweenTubesBySubmission", "MultiplexedLibraryTube"=>"Io::MultiplexedLibraryTube", "BaitLibraryLayout"=>"Io::BaitLibraryLayout", "Lot"=>"Io::Lot", "TubeFromTubeCreation"=>"Io::TubeFromTubeCreation", "Tube::Purpose"=>"Io::Tube::Purpose", "IlluminaHtp::StockTubePurpose"=>"Io::Tube::Purpose", "IlluminaHtp::InitialStockTubePurpose"=>"Io::Tube::Purpose", "Transfer::BetweenSpecificTubes"=>"Io::Transfer::BetweenSpecificTubes", "Tube"=>"Io::Tube", "SubmissionTemplate"=>"Io::SubmissionTemplate", "ExtractionAttribute"=>"Io::ExtractionAttribute", "TagLayout"=>"Io::TagLayout", "BulkTransfer"=>"Io::BulkTransfer"})
app/api/core/registry.rb:22:in `lookup_target_class_through_model_hierarchy!'
-------------------------------
Request:
-------------------------------
* URL : https://.../api/1/plate-uuid/qc_files
* HTTP Method: POST
* IP address : ...
* Parameters : {}
* Timestamp : 2022-05-26 11:36:56 +0100
* Server : sss1-prod
* Rails root : /var/www/sequencescape/releases/20220516171117
* Process: 26368
-------------------------------
Session:
-------------------------------
* session id: [FILTERED]
* data: {}
-------------------------------
Environment:
-------------------------------
* CONTENT_LENGTH : 2013
* CONTENT_TYPE : sequencescape/qc_file
* GATEWAY_INTERFACE : CGI/1.2
* HTTP_ACCEPT : application/json
* HTTP_ACCEPT_ENCODING : gzip;q=1.0,deflate;q=0.6,identity;q=0.3
* HTTP_CONNECTION : close
* HTTP_CONTENT_DISPOSITION : form-data; filename="Export.csv"
* HTTP_COOKIE : api_key=
* HTTP_HOST : ...
* HTTP_USER_AGENT : Ruby
* HTTP_VERSION : HTTP/1.0
* HTTP_X_FORWARDED_FOR : ,...
* HTTP_X_FORWARDED_PROTO : https
* HTTP_X_SEQUENCESCAPE_CLIENT_ID : ...=
* ORIGINAL_FULLPATH : /api/1/plate-uuid/qc_files
* ORIGINAL_SCRIPT_NAME :
* PATH_INFO : /plate-uuid/qc_files
* QUERY_STRING :
* REMOTE_ADDR : ...
* REQUEST_METHOD : POST
* REQUEST_PATH : /api/1/plate-uuid/qc_files
* REQUEST_URI : /api/1/plate-uuid/qc_files
* ROUTES_63360_SCRIPT_NAME :
* SCRIPT_NAME : /api/1
* SERVER_NAME : ...
* SERVER_PORT : 443
* SERVER_PROTOCOL : HTTP/1.1
* SERVER_SOFTWARE : puma 5.6.4 Birdie's Version
* action_dispatch.authenticated_encrypted_cookie_salt : authenticated encrypted cookie
* action_dispatch.backtrace_cleaner : #<Rails::BacktraceCleaner:0x0000559ff199d9f0>
* action_dispatch.content_security_policy : #<ActionDispatch::ContentSecurityPolicy:0x0000559ff2576c40>
* action_dispatch.content_security_policy_nonce_directives: ["script-src"]
* action_dispatch.content_security_policy_nonce_generator : #<Proc:0x0000559ff2576970 /var/www/sequencescape/releases/20220516171117/config/initializers/content_security_policy.rb:43 (lambda)>
* action_dispatch.content_security_policy_report_only : true
* action_dispatch.cookies : #<ActionDispatch::Cookies::CookieJar:0x00007fd6c6d5a5d0>
* action_dispatch.cookies_digest :
* action_dispatch.cookies_rotations : #<ActiveSupport::Messages::RotationConfiguration:0x0000559fee641480>
* action_dispatch.cookies_serializer : json
* action_dispatch.encrypted_cookie_cipher :
* action_dispatch.encrypted_cookie_salt : encrypted cookie
* action_dispatch.encrypted_signed_cookie_salt : signed encrypted cookie
* action_dispatch.http_auth_salt : http authentication
* action_dispatch.key_generator : #<ActiveSupport::CachingKeyGenerator:0x0000559ff6373d18>
* action_dispatch.logger : #<Syslog::Logger:0x0000559ff10943e0>
* action_dispatch.parameter_filter : [:password, :password, :credential_1, :uploaded_data, :password]
* action_dispatch.redirect_filter : []
* action_dispatch.remote_ip : ...
* action_dispatch.request.content_type : sequencescape/qc_file
* action_dispatch.request.parameters : {}
* action_dispatch.request.path_parameters : {}
* action_dispatch.request.query_parameters : {}
* action_dispatch.request.request_parameters : {}
* action_dispatch.request.unsigned_session_cookie : {}
* action_dispatch.request_id : 98803128-39f9-4ff4-9247-62130afe4975
* action_dispatch.routes : #<ActionDispatch::Routing::RouteSet:0x0000559ff0fe62b8>
* action_dispatch.secret_key_base : 1a109ca0f7555e2562326cd196bad5f4152ea855243ac97de24d392b913ae1bc10a04e6d3b7c86c240e926004062ecccc7bb64e77af7ef6d576e52e8f430d69a
* action_dispatch.show_detailed_exceptions : false
* action_dispatch.show_exceptions : true
* action_dispatch.signed_cookie_digest :
* action_dispatch.signed_cookie_salt : signed cookie
* action_dispatch.use_authenticated_cookie_encryption : false
* action_dispatch.use_cookies_with_metadata : false
* puma.config : #<Puma::Configuration:0x0000559fee04cbb0>
* puma.request_body_wait : 0
* puma.socket : #<TCPSocket:0x00007fd6a4ac3258>
* rack.after_reply : []
* rack.cors : #<Rack::Cors::Result:0x00007fd6a4ac08c8>
* rack.errors : #<IO:0x0000559fed9537f0>
* rack.hijack : #<Puma::Client:0x00007fd6a4ac31b8>
* rack.hijack? : true
* rack.input : #<StringIO:0x00007fd6a4ac1a48>
* rack.logger : #<Rack::NullLogger:0x0000559ff60c2f10>
* rack.multiprocess : false
* rack.multithread : true
* rack.request.cookie_hash : {"api_key"=>""}
* rack.request.cookie_string : api_key=
* rack.request.query_hash : {}
* rack.request.query_string :
* rack.run_once : false
* rack.session : #<ActionDispatch::Request::Session:0x00007fd6a48a59f8>
* rack.session.options : #<ActionDispatch::Request::Session::Options:0x00007fd6a48a58b8>
* rack.tempfiles : []
* rack.url_scheme : https
* rack.version : [1, 6]
* sinatra.error : Validation failed: Units can't be blank
* sinatra.error.params : {"captures"=>["plate-uuid", "qc_files"]}
* sinatra.route : POST \/([\da-f]{8}(?:-[\da-f]{4}){3}-[\da-f]{12})(?:\/([^\/]+(?:\/[^\/]+)*))?
-------------------------------
Backtrace:
-------------------------------
app/api/core/registry.rb:22:in `lookup_target_class_through_model_hierarchy!'
app/api/core/registry.rb:28:in `lookup_target_class_through_model_hierarchy!'
app/api/core/registry.rb:28:in `lookup_target_class_through_model_hierarchy!'
app/api/core/registry.rb:40:in `lookup_for_object'
app/api/core/service/error_handling.rb:68:in `api_error'
app/api/core/service/error_handling.rb:16:in `block (2 levels) in registered'
This looks like we're trying to render the API error, and failing as we can't translate the object. We probably want to handle things a bit more elegantly than going down this route, but it couldn't harm to improve lookup_target_class_through_model_hierarchy! in the context of api_error to provide a default IO handler. Not sure if this should apply outside of API error or not.
Limber error
This is essentially the call to the SS API,
A StandardError occurred in qc_files#create:
There is an issue with the API connection to Sequencescape ({"status":500,"error":"Internal Server Error"})
-------------------------------
Request:
-------------------------------
* URL : https://.../limber_plates/plate-uuid/qc_files
* HTTP Method: POST
* IP address : ...
* Parameters : {"authenticity_token"=>"[FILTERED]", "qc_file"=>#<ActionDispatch::Http::UploadedFile:0x00007f6e08b1f490 @tempfile=#<Tempfile:/tmp/RackMultipart20220526-28322-mcsurz.csv>, @original_filename="Export.csv", @content_type="text/csv", @headers="Content-Disposition: form-data; name=\"qc_file\"; filename=\"Export.csv\"\r\nContent-Type: text/csv\r\n">, "controller"=>"qc_files", "action"=>"create", "limber_plate_id"=>"plate-uuid"}
* Timestamp : 2022-05-26 11:36:56 +0100
* Server : limb-prod
* Rails root : /var/www/limber/releases/20220511094609
* Process: 28322
-------------------------------
Session:
-------------------------------
* session id: [FILTERED]
* data: {...}
-------------------------------
Environment:
-------------------------------
* CONTENT_LENGTH : 2391
* CONTENT_TYPE : multipart/form-data; boundary=----WebKitFormBoundaryGiCM8hNJEZSt9wEF
* GATEWAY_INTERFACE : CGI/1.2
* HTTP_ACCEPT : text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
* HTTP_ACCEPT_ENCODING : gzip, deflate, br
* HTTP_ACCEPT_LANGUAGE : en-GB,en-US;q=0.9,en;q=0.8
* HTTP_CACHE_CONTROL : max-age=0
* HTTP_CONNECTION : close
* HTTP_COOKIE : user_name=...+...; user_id=...; _limber_session=HqwU3z2K2pDcoFN0iGcu8t85XDXtSjBH9KZNUdBruofEPl4R3ywJ%2Bnlg01yL%2BhI80OA000%2FV9EV4n%2BqBrBqz8jtI1KBqbzlem8rTtwD0jYz6dZybfUZ0g%2BUcp%2F6JcW5qXjyHANpRUciLQZDizSNDCit5qRhCJO3U%2BEMyI2X8QGDrtE2FKKVjusqyLGluMU0M%2FFP0jxYyfE1nyNEm4yolUXlSRuZB4pauegCqvkEgs67ZlTw3kQySo5Rb%2B9jhFStP35oR7OTBArYG7IZHVxiPJb41MQbmQgrwwBKMyjrM4%2BomHlZcs9%2Bdep%2BGMi9lrbpYAJr1Vh5Swt4tZUxyx7iSQJ2GpRto4SrSXYlXwY%2Bm%2B4aB7MuOTA%2FBf%2FKYTzKA8dRyo6oP%2B4OYXlR3aBfn95ziyicnD0%2B1dWOvlGeNNLdpPLl2QgE%3D--mB2geeH93mupnB9k--IkEgOi49a9gFpcIgfISRjw%3D%3D
* HTTP_HOST : ...
* HTTP_ORIGIN : https://...
* HTTP_REFERER : https://.../limber_plates/plate-uuid
* HTTP_SEC_CH_UA : " Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"
* HTTP_SEC_CH_UA_MOBILE : ?0
* HTTP_SEC_CH_UA_PLATFORM : "Windows"
* HTTP_SEC_FETCH_DEST : document
* HTTP_SEC_FETCH_MODE : navigate
* HTTP_SEC_FETCH_SITE : same-origin
* HTTP_SEC_FETCH_USER : ?1
* HTTP_UPGRADE_INSECURE_REQUESTS : 1
* HTTP_USER_AGENT : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36
* HTTP_VERSION : HTTP/1.0
* HTTP_X_FORWARDED_FOR : ...
* HTTP_X_FORWARDED_HOST : ...:443
* HTTP_X_FORWARDED_PROTO : https
* HTTP_X_REAL_IP : ...
* ORIGINAL_FULLPATH : /limber_plates/plate-uuid/qc_files/
* ORIGINAL_SCRIPT_NAME :
* PATH_INFO : /limber_plates/plate-uuid/qc_files
* QUERY_STRING :
* REMOTE_ADDR : 127.0.0.1
* REQUEST_METHOD : POST
* REQUEST_PATH : /limber_plates/plate-uuid/qc_files/
* REQUEST_URI : /limber_plates/plate-uuid/qc_files/
* ROUTES_11560_SCRIPT_NAME :
* SCRIPT_NAME :
* SERVER_NAME : ...
* SERVER_PORT : 443
* SERVER_PROTOCOL : HTTP/1.1
* SERVER_SOFTWARE : puma 5.6.4 Birdie's Version
* action_controller.instance : #<QcFilesController:0x00007f6e08b2c230>
* action_dispatch.authenticated_encrypted_cookie_salt : [FILTERED]
* action_dispatch.backtrace_cleaner : #<Rails::BacktraceCleaner:0x00007f6e14009090>
* action_dispatch.content_security_policy :
* action_dispatch.content_security_policy_nonce_directives:
* action_dispatch.content_security_policy_nonce_generator :
* action_dispatch.content_security_policy_report_only : false
* action_dispatch.cookies : #<ActionDispatch::Cookies::CookieJar:0x00007f6e08c98808>
* action_dispatch.cookies_digest :
* action_dispatch.cookies_rotations : #<ActiveSupport::Messages::RotationConfiguration:0x0000557c90942598>
* action_dispatch.cookies_same_site_protection : #<Proc:0x00007f6e140085f0 /var/www/limber/shared/vendor/bundle/ruby/2.7.0/gems/railties-7.0.2.4/lib/rails/application.rb:613>
* action_dispatch.cookies_serializer : json
* action_dispatch.encrypted_cookie_cipher : [FILTERED]
* action_dispatch.encrypted_cookie_salt : [FILTERED]
* action_dispatch.encrypted_signed_cookie_salt : [FILTERED]
* action_dispatch.http_auth_salt : [FILTERED]
* action_dispatch.key_generator : #<ActiveSupport::CachingKeyGenerator:0x0000557c931e3088>
* action_dispatch.log_rescued_responses : true
* action_dispatch.logger : #<Syslog::Logger:0x0000557c91f86778>
* action_dispatch.parameter_filter : [:passw, :secret, :token, :_key, :crypt, :salt, :certificate, :otp, :ssn]
* action_dispatch.permissions_policy :
* action_dispatch.redirect_filter : []
* action_dispatch.remote_ip : ...
* action_dispatch.request.content_type : multipart/form-data
* action_dispatch.request.formats : [#<Mime::Type:0x0000557c92583478 @synonyms=["application/xhtml+xml"], @symbol=:html, @string="text/html", @hash=1165804490146581750>]
* action_dispatch.request.parameters : {"authenticity_token"=>"[FILTERED]", "qc_file"=>#<ActionDispatch::Http::UploadedFile:0x00007f6e08b1f490 @tempfile=#<Tempfile:/tmp/RackMultipart20220526-28322-mcsurz.csv>, @original_filename="Export.csv", @content_type="text/csv", @headers="Content-Disposition: form-data; name=\"qc_file\"; filename=\...
* action_dispatch.request.path_parameters : {:controller=>"qc_files", :action=>"create", :limber_plate_id=>"plate-uuid"}
* action_dispatch.request.query_parameters : {}
* action_dispatch.request.request_parameters : {"authenticity_token"=>"[FILTERED]", "qc_file"=>#<ActionDispatch::Http::UploadedFile:0x00007f6e08b1f490 @tempfile=#<Tempfile:/tmp/RackMultipart20220526-28322-mcsurz.csv>, @original_filename="Export.csv", @content_type="text/csv", @headers="Content-Disposition: form-data; name=\"qc_file\"; filename=\...
* action_dispatch.request.unsigned_session_cookie : {"session_id"=>"d10d43431680adc56915c4938dfdc2ad", "_csrf_token"=>"[FILTERED]", "user_uuid"=>"...", "user_name"=>"..."}
* action_dispatch.request_id : 180ca378-dbd1-484d-9407-a00038f88b50
* action_dispatch.routes : #<ActionDispatch::Routing::RouteSet:0x0000557c92272c70>
* action_dispatch.secret_key_base : [FILTERED]
* action_dispatch.show_detailed_exceptions : false
* action_dispatch.show_exceptions : true
* action_dispatch.signed_cookie_digest :
* action_dispatch.signed_cookie_salt : [FILTERED]
* action_dispatch.use_authenticated_cookie_encryption : [FILTERED]
* action_dispatch.use_cookies_with_metadata : true
* puma.config : #<Puma::Configuration:0x0000557c8f8792c0>
* puma.request_body_wait : 0
* puma.socket : #<TCPSocket:0x00007f6e08b3aa60>
* rack.after_reply : []
* rack.errors : #<IO:0x0000557c8f17f7d0>
* rack.hijack : #<Puma::Client:0x00007f6e08b3aa10>
* rack.hijack? : true
* rack.input : #<StringIO:0x00007f6e08b39c00>
* rack.multiprocess : false
* rack.multithread : true
* rack.request.cookie_hash : {"user_name"=>"...", "user_id"=>"...", "_limber_session"=>"...
* rack.request.cookie_string : user_name=...+...; user_id=...; _limber_session=...
* rack.request.form_hash : {"authenticity_token"=>"[FILTERED]", "qc_file"=>{:filename=>"Export.csv", :type=>"text/csv", :name=>"qc_file", :tempfile=>#<Tempfile:/tmp/RackMultipart20220526-28322-mcsurz.csv>, :head=>"Content-Disposition: form-data; name=\"qc_file\"; filename=\"Export.csv\"\r\nContent-Type: text/csv\r\n"}}
* rack.request.form_input : #<StringIO:0x00007f6e08b39c00>
* rack.request.query_hash : {}
* rack.request.query_string :
* rack.run_once : false
* rack.session : #<ActionDispatch::Request::Session:0x00007f6e08b2db58>
* rack.session.options : #<ActionDispatch::Request::Session::Options:0x00007f6e08b2dae0>
* rack.tempfiles : [#<Tempfile:/tmp/RackMultipart20220526-28322-mcsurz.csv>]
* rack.url_scheme : https
* rack.version : [1, 6]
