sending UTF-8 data over SSL can result in lost data

[bporterfield]

Not quite sure yet if this belongs here. Requests seem to hang over ssl when sending back data read from a file. The file contains a mix of single-byte UTF-8 characters and double-byte UTF-8 characters (all at the front in my example, but not in the real case). Over http it works fine, and the same app works in https and http in MRI.

Rack app:

require 'rack/utils'
require 'openssl'
require 'webrick'
require 'webrick/https'

class SslEncodingIssue
  def call(env)

    # file can be downloaded from https://gist.github.com/1947380
    out = File.read("gistfile1.txt")    

    headers = {}
    headers["Content-Length"] = [out].inject(0) { |l, p| l + Rack::Utils.bytesize(p) }.to_s
    headers["Content-Type"] = "text/html"

    [200, headers, [out]]
  end
end

pkey = OpenSSL::PKey::RSA.new(File.read("keypair.pem"))
cert = OpenSSL::X509::Certificate.new(File.read("cert.pem"))

Rack::Handler::WEBrick.run(
  SslEncodingIssue.new,
  :Port => 3000,
  :SSLEnable => true,
  :SSLVerifyClient => OpenSSL::SSL::VERIFY_NONE,
  :SSLCertificate => cert,
  :SSLPrivateKey => pkey,
  :SSLCertName => [ [ "CN",WEBrick::Utils::getservername ] ] 
)

Run rackup and ping http://localhost:3000/ - for me, the request hangs for a bit, and curl responds with: * transfer closed with 2 bytes remaining to read.

Change :SSLEnable = true to = :SSLEnable = false and hit the http url, and the problem goes away. I've put this test case into Webrick, but was having the same issue with other server.

Problem is very dependent on length of output - remove a few lines of dots and the issue does not occur.

Happy to provide more info!