convertSTL icon indicating copy to clipboard operation
convertSTL copied to clipboard

Fails to detect binary format STL file

Open AKA-Axanar opened this issue 10 years ago • 8 comments

Go to http://www.eng.nus.edu.sg/LCEL/RP/u21/wwwroot/stl_library.htm#Download and download the porsche STL file. The STL file is in binary format but the converter thinks it's ascii and so fails to convert it.

AKA-Axanar avatar Jun 07 '15 14:06 AKA-Axanar

I have used that porsche file for other purposes, and believe there is some bad data in it.

toreyheinz avatar Jun 07 '15 20:06 toreyheinz

Opening and saving the file as UTF-8, allows convetSTL to recoginze it as binary. I had another version of the file that I had "resized" through SOLIDWORKS, and that one converted just fine.

I believe the "blame" lies with the porsche file. Let me know if you actually need the porsche in ascii format, and I can get it to you.

toreyheinz avatar Jun 07 '15 20:06 toreyheinz

Zeroing out the 80 byte header caused convertSTL to recognize it as a binary file. But that shouldn't be necessary. If the header doesn't start with ascii "solid" (optionally prefixed by spaces) then the file should be treated as a binary format file.

The format of the binary file seems fine. It has the correct number of bytes based on the number of facets. There is nothing in the file that should make convertSTL think it is an ascii formatted STL file.

AKA-Axanar avatar Jun 07 '15 22:06 AKA-Axanar

Currently the script checks if the header has "solid" in it, and if it doesn't, it treats the file as a binary file. I was curious how/why it would think the porsche file was ascii, so I examined the file directly, and the first line DOES have "solid" in it!

File.new('porsche.stl', 'r').gets
#=> "104-9.stl   tol 0.01   ac 0.02   solid  layer 209                          3DV*\x904O\u0002\u0000\xD7\xFD\u0018\xBD\u0000\u0000\u0000\...

Same thing for the brain-gear.stl

File.new('brain-gear.stl', 'r').gets
#=> "solid brain_gear\u0000STL Output                                                     \x8A#\u0001\u0000\u0000\u0000\u0000\u0000\u0000\u0000

So I'm thinking simply checking for "solid" is not a reliable way to detect binary vs ascii.

toreyheinz avatar Jun 08 '15 12:06 toreyheinz

Ah, that explains what's happening. You could skip any leading spaces and then see if the next word is "solid".

AKA-Axanar avatar Jun 08 '15 13:06 AKA-Axanar

OK, so this is interesting. A number of other tools/technics fail to detect these as binary files because the first "line" has non-binary data.

http://stackoverflow.com/questions/2355866/ruby-how-to-determine-if-file-being-read-is-binary-or-text https://github.com/djberg96/ptools

require 'ptools'
File.binary?('porsche.stl') #=> false
File.binary?('brain-gear.stl') #=> false

https://github.com/djberg96/ptools/blob/master/lib/ptools.rb#L90

def self.binary?(file)
  return false if image?(file)
  bytes = File.stat(file).blksize
  bytes = 4096 if bytes > 4096
  s = (File.read(file, bytes) || "")
  s = s.encode('US-ASCII', :undef => :replace).split(//)
  ((s.size - s.grep(" ".."~").size) / s.size.to_f) > 0.30
end

https://github.com/brianmario/charlock_holmes

require 'charlock_holmes'
CharlockHolmes::EncodingDetector.detect('porsche.stl')
#=> {:type=>:text, :encoding=>"ISO-8859-1", :ruby_encoding=>"ISO-8859-1", :confidence=>75, :language=>"en"}

CharlockHolmes::EncodingDetector.detect('brain-gear.stl')
#=> {:type=>:text, :encoding=>"ISO-8859-1", :ruby_encoding=>"ISO-8859-1", :confidence=>60, :language=>"da"}

Using the unix file command does detect the file correctly.

file porsche.stl
#=> porsche.stl: data

file brain-gear.stl
#=> brain-gear.stl: data

file cylinder.stl
#=> cylinder.stl: ASCII text

And this works as well: https://www.ruby-forum.com/topic/122170#544577

class File
  def self.binary?(name)
    ascii = control = binary = 0

    File.open(name, "rb") {|io| io.read(1024)}.each_byte do |bt|
      case bt
        when 0...32
          control += 1
        when 32...128
          ascii += 1
        else
          binary += 1
      end
    end

    control.to_f / ascii > 0.1 || binary.to_f / ascii > 0.05
  end
end

File.binary?('porsche.stl') #=> true

Given what we've found I think we can come up with a reliable solution, but I need to set this aside for now.

toreyheinz avatar Jun 08 '15 15:06 toreyheinz

Hi, I had same problem, it does not recognize binary stl file.

A binary stl file is defined as follow : BINARY STL file format is accessed by byte. The format is as follows: the first 80 bytes are used for description, and the next 4 bytes represents the total number of the facets(Long Int), followed by the facet information (normal and 3 vertices), the normal and vertices are stored in floating point format, each occupying 4 bytes. At the end of each facet information section, there are two bytes spaces, then the next facet is repeated till the end of the file. When BINARY format is used to describe STL file, the data size is much smaller than ASCII format, so most STL files available now use BINARY format.

<BINARY STL file format>::=<STL file entity name><facet number N> <STL file entity name >::=<80 bytes entity name, spaces are used to fill the blank> <facet number N>::=<4 bytes long integer> ::=<2 bytes spaces><2 bytes spaces> ... ... ::=<lx,ly,lz, float, 12 bytes> ::=<x1,y1,z1,x2,y2,z2,x3,y3,z3, float, 36 bytes>

sancelot avatar Apr 10 '18 09:04 sancelot

@cmpolis I've got an AutoCAD-generated binary STL which is recognized as ASCII STL by the script,because it started by AutoCAD solid (containing solid).

I think https://github.com/cmpolis/convertSTL/blob/572a5e1ee1d7f4d370b2b4e5d32ba6517c11fa41/convertSTL.rb#L38 should be changed to the following:

if tempLine.start_with? "solid"

based on the documentations on wikipedia

A binary STL file has an 80-character header (which is generally ignored, but should never begin with "solid" because that may lead some software to assume that this is an ASCII STL file). and the aforementioned AutoCAD-generated file.

hadisfr avatar Dec 24 '18 19:12 hadisfr