dbptk-ui icon indicating copy to clipboard operation
dbptk-ui copied to clipboard

Improve BLOB persistence and presentation

Open luis100 opened this issue 7 years ago • 2 comments

Some binaries columns might contain very small amount of bits, but a large quantity of rows, which will can bring problems to the file system. Also, when a column has just a couple of bits, allowing to download them instead presenting directly on the interface and allow to search by the "binary" code.

So, instead of keeping information always in files they could be kept directly in Solr in some different formats:

  • Up to 8 bits, defined on the column size: Keep in Solr as text encoded a '0's and '1's, indexed=true, stored=true, example: 010101
  • Up to 1 KB, measured on each cell: Keep in Solr as base64 encoded binary ready to be presented as download link using the data URI scheme in the web interface, indexed=false, stored=true, example: data:application/octetstream;base64,iVBORw0KGgoAAA ANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4 //8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU 5ErkJggg==
  • Above 1KB, all other cases: Keep in disk and serve via a servlet, keep the relative path to the file on the index, might also keep the mime type and file name if present, indexed=false, stored=true, example: file:application/octetstream:lobs/table1/row1/col1.bin

luis100 avatar May 04 '17 10:05 luis100

Note that CLOBS should follow different rules:

  • Up to 256 KB keep in SOLR, indexed=true, stored=true
  • Above 256 KB keep in file and serve as a text file via a servlet.

luis100 avatar May 04 '17 10:05 luis100

The MIME type detection is on another issue #112, in this issue I assume the MIME type, and file name, are always unknown.

luis100 avatar May 04 '17 10:05 luis100