curb
curb copied to clipboard
taf2
error running the curb test suite with curl 8.7.1
this is from the Debian package build, but it doesn't look specific to Debian:
RUBYLIB=. GEM_PATH=/home/vagrant/ruby-curb-1.0.5/debian/ruby-curb/usr/lib/x86_64-linux-gnu/rubygems-integration/3.1.0:/home/vagrant/ruby-curb-1.0.5/debian/.debhelper/generated/_source/home/.local/share/gem/ruby/3.1.0:/var/lib/gems/3.1.0:/usr/local/lib/ruby/gems/3.1.0:/usr/lib/ruby/gems/3.1.0:/usr/lib/x86_64-linux-gnu/ruby/gems/3.1.0:/usr/share/rubygems-integration/3.1.0:/usr/share/rubygems-integration/all:/usr/lib/x86_64-linux-gnu/rubygems-integration/3.1.0 ruby3.1 -S rake --rakelibdir /gem2deb-nonexistent -f debian/ruby-tests.rake
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_multi.rb:189: warning: assigned but unused variable - in_file_stack
Loaded suite /usr/share/rubygems-integration/all/gems/rake-13.2.1/lib/rake/rake_test_loader
Started
.............................E
========================================================================================================================================
Error: test_easy_http_verbs(TestCurbCurlEasy): Curl::Err::ReadError: Failed to open/read local data from file/application: client read function EOF fail, only only 4/5 of needed bytes read
/home/vagrant/ruby-curb-1.0.5/lib/curl/easy.rb:80:in `perform'
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_easy.rb:1064:in `http_put'
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_easy.rb:1064:in `test_easy_http_verbs'
1061: assert_equal "POST\n", curl.body_str
1062: curl.http('PURGE')
1063: assert_equal 'PURGE', curl.body_str
=> 1064: curl.http_put('hello')
1065: assert_equal "PUT\nhello", curl.body_str
1066: curl.http('COPY')
1067: assert_equal 'COPY', curl.body_str
========================================================================================================================================
.................................................................E
========================================================================================================================================
Error: test_put_data(TestCurbCurlEasy): Curl::Err::ReadError: Failed to open/read local data from file/application: client read function EOF fail, only only 6/7 of needed bytes read
/home/vagrant/ruby-curb-1.0.5/lib/curl/easy.rb:80:in `perform'
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_easy.rb:885:in `test_put_data'
882: curl = Curl::Easy.new(TestServlet.url)
883: curl.put_data = 'message'
884:
=> 885: curl.perform
886:
887: assert_match(/^PUT/, curl.body_str)
888: assert_match(/message$/, curl.body_str)
========================================================================================================================================
E
========================================================================================================================================
Error: test_put_data_null_bytes(TestCurbCurlEasy): Curl::Err::ReadError: Failed to open/read local data from file/application: client read function EOF fail, only only 2/3 of needed bytes read
/home/vagrant/ruby-curb-1.0.5/lib/curl/easy.rb:80:in `perform'
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_easy.rb:896:in `test_put_data_null_bytes'
893: curl = Curl::Easy.new(TestServlet.url)
894: curl.put_data = "a\0b"
895:
=> 896: curl.perform
897:
898: assert_match(/^PUT/, curl.body_str)
899: assert_match("a\0b", curl.body_str)
========================================================================================================================================
.E
========================================================================================================================================
Error: test_put_remote(TestCurbCurlEasy): Curl::Err::ReadError: Failed to open/read local data from file/application: client read function EOF fail, only only 6/7 of needed bytes read
/home/vagrant/ruby-curb-1.0.5/lib/curl/easy.rb:80:in `perform'
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_easy.rb:873:in `http_put'
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_easy.rb:873:in `test_put_remote'
870: def test_put_remote
871: curl = Curl::Easy.new(TestServlet.url)
872: curl.headers['Content-Type'] = 'application/json'
=> 873: assert curl.http_put("message")
874: assert_match(/^PUT/, curl.body_str)
875: assert_match(/message$/, curl.body_str)
876: assert_match(/message$/, curl.body)
========================================================================================================================================
...............................O
========================================================================================================================================
Omission: Skip, libcurl too old (< 7.22.0) [test_connection_keepalive(TestCurbCurlMulti)]
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_multi.rb:16:in `test_connection_keepalive'
========================================================================================================================================
......E
========================================================================================================================================
Error: test_multi_easy_http_01(TestCurbCurlMulti):
Curl::Err::AbortedByCallbackError: <200> expected but was
<0>.
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_multi.rb:523:in `block in test_multi_easy_http_01'
520: { :url => TestServlet.url, :method => :get }
521: ]
522: Curl::Multi.http(urls, {:pipeline => true}) do|easy, code, method|
=> 523: assert_equal 200, code
524: case method
525: when :post
526: assert_match(/POST/, easy.body_str)
/home/vagrant/ruby-curb-1.0.5/lib/curl/multi.rb:143:in `block (2 levels) in http'
/home/vagrant/ruby-curb-1.0.5/lib/curl/multi.rb:164:in `perform'
/home/vagrant/ruby-curb-1.0.5/lib/curl/multi.rb:164:in `http'
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_multi.rb:522:in `test_multi_easy_http_01'
========================================================================================================================================
..E
========================================================================================================================================
Error: test_multi_easy_put_01(TestCurbCurlMulti):
Curl::Err::AbortedByCallbackError: </PUT/> was expected to be =~
<"">.
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_multi.rb:508:in `block in test_multi_easy_put_01'
505: { :url => TestServlet.url, :method => :put, :put_data => "message",
506: :headers => {'Content-Type' => 'application/json' } }]
507: Curl::Multi.put(urls, {}, {:pipeline => true}) do|easy|
=> 508: assert_match(/PUT/, easy.body_str)
509: assert_match(/message/, easy.body_str)
510: end
511: end
/home/vagrant/ruby-curb-1.0.5/lib/curl/multi.rb:67:in `block in put'
/home/vagrant/ruby-curb-1.0.5/lib/curl/multi.rb:143:in `block (2 levels) in http'
/home/vagrant/ruby-curb-1.0.5/lib/curl/multi.rb:164:in `perform'
/home/vagrant/ruby-curb-1.0.5/lib/curl/multi.rb:164:in `http'
/home/vagrant/ruby-curb-1.0.5/lib/curl/multi.rb:67:in `put'
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl_multi.rb:507:in `test_multi_easy_put_01'
========================================================================================================================================
..................................E
========================================================================================================================================
Error: test_put(TestCurl): Curl::Err::ReadError: Failed to open/read local data from file/application: client read function EOF fail, only only 6/7 of needed bytes read
/home/vagrant/ruby-curb-1.0.5/lib/curl/easy.rb:80:in `perform'
/home/vagrant/ruby-curb-1.0.5/lib/curl.rb:26:in `http'
/home/vagrant/ruby-curb-1.0.5/lib/curl.rb:26:in `http'
/home/vagrant/ruby-curb-1.0.5/lib/curl.rb:39:in `put'
/home/vagrant/ruby-curb-1.0.5/tests/tc_curl.rb:20:in `test_put'
17: end
18:
19: def test_put
=> 20: curl = Curl.put(TestServlet.url, {:foo => "bar"})
21: assert_equal "PUT\nfoo=bar", curl.body_str
22: end
23:
========================================================================================================================================
....
Finished in 13.048177976 seconds.
----------------------------------------------------------------------------------------------------------------------------------------
180 tests, 614 assertions, 0 failures, 7 errors, 0 pendings, 1 omissions, 0 notifications
96.0894% passed
----------------------------------------------------------------------------------------------------------------------------------------
13.80 tests/s, 47.06 assertions/s
The corresponding Debian bug is https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1069258
In the Debian bug, someone points to https://curl.se/mail/lib-2024-03/0054.html , which might indicate that this is a bug in curb, not in curl.
It works fine with curl 8.6.0.
In previous versions of curl, the read callback was sent a buffer of 65KB. Curl 8.7.1 now appears to use a buffer size equal to the object being PUT.
This in turn exposes a possible bug in https://github.com/taf2/curb/blob/13144ec5d50ffea0460298cc5de8a0b33db78d22/ext/curb_easy.c#L51
The remaining < read_bytes section works with the previously large buffer whereas the section marked // they're equal for some reason decrements the read_bytes value, causing the byte mismatch error in curl.
It's unclear why this behavior exists; changing remaining < read_bytes to remaining <= read_bytes is likely a sufficient fix for this issue.
git bisect for curl.git shows the following change brings about this error:
https://github.com/curl/curl/commit/9369c30cd87c041cf983bcdfabd1570980abbaf6
(as the above debian bug says)
I have not read APIs of curl, and have not read the above change in detail, however reading the comment of the above commit:
test547, test555, test1620: fix the length check in the lib code to
only fail for reads *smaller* than expected. This was a bug in the
test code that never triggered in the old implementation.
and the actual code change, such as https://github.com/curl/curl/commit/9369c30cd87c041cf983bcdfabd1570980abbaf6#diff-bfeebefe5ca654a47ef7409092bf938d0002810757743093dcabc5fb301ad9feR50
seems to be suggesting that curb side code should be changed as:
diff --git a/ext/curb_easy.c b/ext/curb_easy.c
index 9054be6..74fa0e6 100644
--- a/ext/curb_easy.c
+++ b/ext/curb_easy.c
@@ -79,7 +79,7 @@ static size_t read_data_handler(void *ptr,
remaining = len - rbcu->offset;
str_ptr = RSTRING_PTR(str);
- if( remaining < read_bytes ) {
+ if( remaining <= read_bytes ) {
if( remaining > 0 ) {
memcpy(ptr, str_ptr+rbcu->offset, remaining);
read_bytes = remaining;
, as already mentioned above https://github.com/taf2/curb/blob/13144ec5d50ffea0460298cc5de8a0b33db78d22/ext/curb_easy.c#L82 And actually the above curb side change works with curl 8.6.0, 8.7.0, 8.7.1 and 8.8.0 . Would you look at this?
