External Signing

[heruwaspodov]

Hi mas @gettalong

I will implement hash signing with another service. The codes like below (the gist is attached) https://gist.github.com/heruwaspodov/ecdb35c19e57abb97894389a1bde4707#file-annotation_signature_hash-rb

def signing_mechanism
      @signing_mechanism ||= lambda do |digest, hash|
        payload = {
          digest: digest,
          hash: AesCipher.new.encrypt(hash)
        }.to_json

        # Golang Services
        # response = RestClient.post('http://127.0.0.1:7777/hash_signing', payload,
        #                            content_type: :json)

        # Ruby Services
        response = RestClient.post('http://127.0.0.1:3000/api/v1/hash_signing', payload,
                                   content_type: :json)

        signed_hash = JSON.parse(response.body)['data']['signed_hash']
        signature = AesCipher.new.decrypt(signed_hash)

        binary_signature = signature.force_encoding('ASCII-8BIT')

        puts "Final signature class: #{binary_signature.class}"
        puts "Final signature encoding: #{binary_signature.encoding}"
        puts "Final signature length: #{binary_signature.bytesize}"

        if verify_signature(hash, binary_signature)
          puts 'Signature verified successfully!'
        else
          puts 'WARNING: Signature verification failed!'
        end

        binary_signature
      end
    end

I already created the Golang service to sign the hash using private_key (I use the Hexapdf private key). The gist is -> https://gist.github.com/heruwaspodov/ecdb35c19e57abb97894389a1bde4707#file-go_hash_signing_service-go response = RestClient.post('http://127.0.0.1:3000/api/v1/hash_signing', payload, content_type: :json)

but the result is always an invalid signature, like the picture below. Meanwhile, the verified signature is true (show the -> puts 'Signature verified successfully!'). I already checked the encryption between Ruby and Golang. It works to encrypt with Golang and decrypt with Ruby and vice versa. https://onecompiler.com/go/437uxvhrm https://onecompiler.com/ruby/437uxu6q3 the binary is not changed and always same identic.

The strange thing, is if I hit another service that was created with ruby (https://gist.github.com/heruwaspodov/ecdb35c19e57abb97894389a1bde4707#file-ruby_hash_signing_service-rb) response = RestClient.post('http://127.0.0.1:3000/api/v1/hash_signing', payload, content_type: :json)

Showing a message : WARNING: Signature verification failed!, but the certificate is safe.

could you give me advice mas. @gettalong? :') I am already stuck in this thank you

[gettalong]

I can't really help you with the Go service. However, I would reduce the complexity to locate the problem. For example, remove the encryption/decryption part of the hash. Does it work if you don't call the Go service at all but you inline the implementation of the Ruby service?

It would also help if you would provide the resulting PDF files so that I can inspect them.

[heruwaspodov]

Both services Ruby and Golang use encrypt decrypt because if it directly sends a hash as binary is not supported. but I will try to find a way to send it directly. Signing with the Ruby service will be successful, but my team proposes using the Golang service. Ruby service is my example of writing with Golang.

This is a PDF file signed by Golang Services.

hash_signing1738546351.pdf

[gettalong]

Could you provide the same file signed by the Ruby service? I would like to compare the information.

Generally, if it works with the Ruby service but not the Go service, then it hints that the problem must be with the Go service.

[heruwaspodov]

this is a file signed by Ruby services

hash_signing1738577681.pdf

this is a file signed by Golang service

hash_signing1738577929.pdf

[gettalong]

Okay, the two files look very similar. And the signed data object is also very similar.

So judging from this I would say there is some subtle error with the Go service. What happens if you provide the same data to the Ruby and the Go service?

[heruwaspodov]

using same HexaPDF key => HexaPDF.demo_cert.key. I never try because the hashing from the same file is always different.

I tried using a real certificate, using Ruby

https://gist.github.com/heruwaspodov/ab8e46bdf57b78c96937bedae63020ff

But the result like below

hash_signing1738744015.pdf

[gettalong]

I'm not sure we are speaking of the same thing.

What I meant was: If you call both the Ruby service and the Go service in the same run, with the same hash, and then compare the returned data, is the returned signature the same? You would need to make sure that both the Ruby and Go signing service do exactly the same thing with the same data. Are you sure the Go signing service doesn't hash the given data again?

[heruwaspodov]

yes, mas. I got your point. I never tried to call the Ruby service and the Go service in the same run, and then compare the returned data. But I ignore the Golang service first. I focused on the Ruby services first. The next level from the Ruby service uses a real certificate, but the result is invalid. When using the HexaPdf private key, the certificate is valid. I have two ways to execute: encryption and sending as binary. But it is the same; the certificate is invalid because it is altered or corrupted.

https://gist.github.com/heruwaspodov/ab8e46bdf57b78c96937bedae63020ff

hash_signing1738819324.pdf

[gettalong]

Okay, I'm sorry but I'm not quite sure what is and isn't working here anymore.

Let's reduce the amount of code to look at and go step by step:

1. Please use the example code from https://hexapdf.gettalong.org/documentation/digital-signatures/signing-pdfs-howto.html#visual.
2. Use the HexaPDF certificate to create a signed PDF and check if it's valid (this should be the case as the example is unmodified).
3. Then use your real certificate and check the result again.
4. Next adjust the example code using your code from https://gist.github.com/heruwaspodov/ecdb35c19e57abb97894389a1bde4707#file-annotation_signature_hash-rb and e.g. update the visual appearance generation to the one you are using.
5. Use the HexaPDF certificate again to create a signed PDF and check if it's valid.
6. Then use your real certificate again and check the result.

This way we should find out where the problem is, starting at a known good version (step 2).

[heruwaspodov]

Hi mas @gettalong I have changed the method for implement the external signing method. And its work. Based on https://hexapdf.gettalong.org/documentation/digital-signatures/signing-pdfs-howto.html#visual

and this is my code https://gist.github.com/heruwaspodov/519ce4167a9eba420a69a2b74eaa9a48

and this is the pdf result

hash_signing1740158935.pdf

this HSM is using AWS.

But if using Aliyun (because i will move using Aliyun), with same code, the error is appear like below:

Input message length exceeded the max supported message length [16000 bytes]

what should i do, mas @gettalong ?

[gettalong]

I'm sorry but I don't know what "Aliyun" is and you only provide images and not a full backtrace or information when the error happens.

What I see in you code is that you use @widget before it is defined (https://gist.github.com/heruwaspodov/519ce4167a9eba420a69a2b74eaa9a48#file-document_service-rb-L16) as only the call to #field_signature at https://gist.github.com/heruwaspodov/519ce4167a9eba420a69a2b74eaa9a48#file-document_service-rb-L20 creates the @widget variable. That is most probably the reason that there is an invalid entry in the /Annots array.

[heruwaspodov]

I'm sorry for not getting back to you sooner. Aliyun is an Alicloud provider. I already asked the Aliyun teams about this error that occurred. This is because openSSL in alicloud has a limitation size even though the file is chunked.

so, I tried to move to another method using the method like below

this is my code in client

require "hexapdf"
require "#{HexaPDF.data_dir}/cert/demo_cert.rb"

class Signing < Service
  def initialize(path)
    @path = path
    @doc = HexaPDF::Document.open(@path)
  end

  def call
    signing03
  end
  # try with hash
  def signing03
    signing_mechanism = lambda do |_digest_method, data|
      response = RestClient.post(
        "https://staging-backend.wspdv.io/core/api/v1/signing_hash",
        data,
        { content_type: "application/octet-stream" }
      )
      binary_signature = response.body
      binary_signature
    end

    @doc.sign("signed.pdf", external_signing: signing_mechanism,
              reason: "Signing 03",
              certificate: public_key,
              certificate_chain: [ intermediate1_cert, intermediate2_cert ])
  end


  def public_key
    # HexaPDF.demo_cert.msign
    OpenSSL::X509::Certificate.new(File.read("certs/cert.crt"))
  end

  def intermediate1_cert
    # HexaPDF.demo_cert.sub_ca
    OpenSSL::X509::Certificate.new(File.read("certs/intermediate1.crt"))
  end

  def intermediate2_cert
    # HexaPDF.demo_cert.root_ca
    OpenSSL::X509::Certificate.new(File.read("certs/intermediate2.crt"))
  end
end

and this is a service to sign

in controller:

 def signing_hash
        binary_data = request.body.read
        signed_hash = SigningHashService.call(binary_data)

        response.headers['Content-Type'] = 'application/octet-stream'
        response.headers['Content-Disposition'] = 'attachment; filename=hash_binary.bin'
        response.stream.write signed_hash
      ensure
        response.stream.close
      end

and this service

# frozen_string_literal: true

class SigningHashService < ApplicationService
  require 'hexapdf'
  require "#{HexaPDF.data_dir}/cert/demo_cert.rb"

  def initialize(hash)
    @hash = hash
  end

  def call
    setup_engine!
    sign!
    cleanup_engine!
    @sign
  end

  def setup_engine!
    Certifications::CloudHsm.new.setup!
  end

  def cleanup_engine!
    Certifications::CloudHsm.new.cleanup!
  end

  def private_key
    if Rails.env.test? || Rails.env.development?
      @private_key ||= HexaPDF.demo_cert.key
      return @private_key
    end

    @private_key ||= OpenSSL::PKey::RSA.new(File.read('/opt/cloudhsm/etc/private-key.pem'))
  end

  def sign!
    @sign = private_key.sign_raw(OpenSSL::Digest.new('SHA256'), @hash, { rsa_padding_mode: 'pss' })
  end
end

when executing the service from the client is success, but the result is invalid

[gettalong]

@heruwaspodov Could you please provide a fully working script (e.g. something I can run like ruby script.rb) so that I can reproduce the problem? And/or provide a result PDF that is invalid for inspection?

[heruwaspodov]

For the sample code i wrote in the same class, actually I want to separate the signing process (private_key signing the document) to another service. this is the code.

https://gist.github.com/heruwaspodov/b4ebb224d33df58c2949ec2ea7555839

You can exec with ruby script like below

ruby signing.rb /path/to/your/document.pdf

This is a pdf result

signed.pdf

[gettalong]

@heruwaspodov I'm sorry but running the script doesn't work as it complains about OpenSSL::Engine doesn't exist (anymore).

[heruwaspodov]

OpenSSL::Engine.by_id('cloudhsm') This code activates the engine cloudhsm, mas @gettalong. If the cloudhsm is not activated, it can not sign.

[gettalong]

This doesn't work:

ruby 3.4.1 $ irb
irb(main):001> require 'openssl'
=> true
irb(main):002> OpenSSL::Engine.by_id('cloudhsm')
(irb):2:in '<main>': uninitialized constant OpenSSL::Engine (NameError)

[gettalong]

@heruwaspodov I looked at the code from https://github.com/gettalong/hexapdf/issues/346#issuecomment-2763631247 again and saw that you are using the PSS padding mode for RSA. That is currently not supported with the built-in signed data creator. You would need to switch to the standard PKCS#1 padding scheme.