failed submissions

[cindymorgan]

Describe the issue

To: [email protected] From: "Michael Richardson" [email protected] Date: Sun, 04 Aug 2024 17:00:08 -0400 Subject: failed submissions

While attempting to upload draft-ietf-opsawg-pcap-04, draft-ietf-opsawg-pcaplinktype-04 and a third pcapng document (not yet attempted), I'm getting:

• curl -sS -D versioned/.draft-ietf-opsawg-pcaplinktype-04.upload -F [email protected] -F xml=@versioned/draft-ietf-opsawg-pcaplinktype-04.xml https://datatracker.ietf.org/api/submission curl: (55) Failed sending data to the peer make: *** [lib/upload.mk:27: versioned/.draft-ietf-opsawg-pcaplinktype-04.upload] Error 55 make: *** Deleting file 'versioned/.draft-ietf-opsawg-pcaplinktype-04.upload'

When I repeated for pcap-04, it said:

{"error": "Validation Error", "messages": ["A submission with same name and revision is currently being processed. <a href="/submit/status/144825/">Check the status here."]}+ echo

and I got the confirmation email, and it worked.

I can't seem to do the same for pcaplinktype-04, but I'll try posting the XML through the DT now.

--

From: "Michael Richardson" [email protected] To: [email protected] Subject: Re: [rt5.ietf.org #34978] AutoReply: failed submissions Date: Sun, 04 Aug 2024 17:01:27 -0400

IETF Support via RT [email protected] wrote: Hide quoted text

and I got the confirmation email, and it worked.

Hide quoted text

I can't seem to do the same for pcaplinktype-04, but I'll try posting the XML through the DT now.

When I tried through DT, it said it was already being processed.

--

Subject: Re: [rt5.ietf.org #34978] AutoReply: failed submissions Date: Sun, 04 Aug 2024 18:40:11 -0400 From: "Michael Richardson" [email protected] To: [email protected]

I don't know if this is helpful.

obiwan-work/opsawg/pcapng mcr 10058 %curl -v -sS -D versioned/.draft-ietf-opsawg-pcapng-02.upload -F [email protected] -F xml=@versioned/draft-ietf-opsawg-pcapng-02.xml https://datatracker.ietf.org/api/submission

• Trying [2606:4700::6810:2d63]:443...
• Connected to datatracker.ietf.org (2606:4700::6810:2d63) port 443 (#0)
• ALPN: offers h2,http/1.1
• TLSv1.3 (OUT), TLS handshake, Client hello (1):
• CAfile: /etc/ssl/certs/ca-certificates.crt
• CApath: /etc/ssl/certs
• TLSv1.3 (IN), TLS handshake, Server hello (2):
• TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
• TLSv1.3 (IN), TLS handshake, Certificate (11):
• TLSv1.3 (IN), TLS handshake, CERT verify (15):
• TLSv1.3 (IN), TLS handshake, Finished (20):
• TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
• TLSv1.3 (OUT), TLS handshake, Finished (20):
• SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
• ALPN: server accepted h2
• Server certificate:
• subject: CN=ietf.org
• start date: Jul 30 02:42:38 2024 GMT
• expire date: Oct 28 02:42:37 2024 GMT
• subjectAltName: host "datatracker.ietf.org" matched cert's "*.ietf.org"
• issuer: C=US; O=Let's Encrypt; CN=E5
• SSL certificate verify ok.
• using HTTP/2
• h2h3 [:method: POST]
• h2h3 [:path: /api/submission]
• h2h3 [:scheme: https]
• h2h3 [:authority: datatracker.ietf.org]
• h2h3 [user-agent: curl/7.88.1]
• h2h3 [accept: /]
• h2h3 [content-length: 193329]
• h2h3 [content-type: multipart/form-data; boundary=------------------------3d08092c21291cbc]
• Using Stream ID: 1 (easy handle 0x55695a3690a0) Show quoted text
• TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
• TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
• old SSL session ID is stale, removing
• Connection #0 to host datatracker.ietf.org left intact curl: (55) Failed sending data to the peer

@mcr

Code of Conduct

• [X] I agree to follow the IETF's Code of Conduct

[jennifer-richards]

First: I believe all the drafts mentioned in these messages were successfully posted on Aug 4, some via manual posts and some via the api. I..e, I don't think anything is currently blocked due to this issue.

Going through the logs, I see the API calls that were posting these. On 2024-08-04 (all times UTC) I see:

• at 20:49:55: draft-ietf-opsawg-pcap-04 was posted with HTTP response status 200. Validation succeeded at 20:49:58.
• at 20:50:01: draft-ietf-opsawg-pcaplinktype-04 was posted with HTTP response status 200. Validation succeeded at 20:50:45.
• at 20:50:24: something was posted with HTTP response status 400 (logs do not indicate what draft)
• at 20:56:40: something was posted with HTTP response status 400 (logs do not indicate what draft)

I'm not sure why you'd see failures for the initial uploads - the first attempts for both submissions were 200s from the datatracker and both were validated successfully. After that, a 400 response is expected because the submission already exists.

[mcr]

Jennifer Richards @.***> wrote: > Going through the logs, I see the API calls that were posting these. On 2024-08-04 (all times UTC) I see: > * at 20:49:55: draft-ietf-opsawg-pcap-04 was posted with HTTP response status 200. Validation succeeded at 20:49:58. > * at 20:50:01: draft-ietf-opsawg-pcaplinktype-04 was posted with HTTP > response status 200. Validation succeeded at 20:50:45.

Yes, I would agree that they were accepted by the system, but the 200 didn't get to me. Perhaps the 200 code didn't get through CloudFlare?

I don't know what code 55 in curl means; my impression is that the socket closed before CURL felt that the transfer was finished.

> I'm not sure why you'd see failures for the initial uploads - the first
> attempts for both submissions were 200s from the datatracker and both
> were validated successfully. After that, a 400 response is expected
> because the submission already exists.

Yes. (I wonder if submission already exists shouldn't return 409)

Today, different results, sorta. First time I try the CURL, I got code 55. Alas, it scrolled in a way I can't get back. So I tried again, and got duplicate submission, which I would expect.

-- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works | IoT architect [ ] @.*** http://www.sandelman.ca/ | ruby on rails [

[jennifer-richards]

Yes, I would agree that they were accepted by the system, but the 200 didn't get to me. Perhaps the 200 code didn't get through CloudFlare?

Perhaps possible, but I don't think it's likely. We did recently add a nginx-based proxy to the stack between you and the datatracker so that seems more likely to be at fault, but the only reports I can find of problems with that involve sessions being closed when multiple files are being uploaded. I don't think that applies here.

I don't know what code 55 in curl means; my impression is that the socket closed before CURL felt that the transfer was finished.

I did find this report that matches, at least superficially. I haven't read through the details any deeper than a skim, just putting it here for info: https://github.com/curl/curl/issues/10591

[mcr]

In actions/runner-images#7329 a working workaround was proposed: --http1.1. This allowed us to continue pushing builds affected by this bug until the underlying issue is fixed.

I'll try this.

[jennifer-richards]

Thanks - keep us updated. What version of curl are you using?

Noticed just now that the ticket I linked involves a PUT, not a POST, so it may not be the same issue. Someone did report problems with POST but I haven't been able to find a proper ticket for that.

[mcr]

Jennifer Richards @.***> wrote: > Thanks - keep us updated. What version of curl are you using?

So sticking to 1.1 seems to help. I haven't checked if my curl version is among those broken yet. (Been sick for three days)

%curl --version curl 7.88.1 (x86_64-pc-linux-gnu) libcurl/7.88.1 OpenSSL/3.0.13 zlib/1.2.13 brotli/1.0.9 zstd/1.5.4 libidn2/2.3.3 libpsl/0.21.2 (+libidn2/2.3.3) libssh2/1.10.0 nghttp2/1.52.0 librtmp/2.3 OpenLDAP/2.5.13 Release-Date: 2023-02-20, security patched: 7.88.1-10+deb12u6 Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

obiwan-work/opsawg/pcapng mcr 10029 %curl -v -sS -D versioned/.draft-ietf-opsawg-pcapng-02.upload -F @.*** -F @.***/draft-ietf-opsawg-pcapng-02.xml https://datatracker.ietf.org/api/submission

• Trying [2606:4700::6810:2c63]:443...
• Connected to datatracker.ietf.org (2606:4700::6810:2c63) port 443 (#0)
• ALPN: offers h2,http/1.1
• TLSv1.3 (OUT), TLS handshake, Client hello (1):
• CAfile: /etc/ssl/certs/ca-certificates.crt
• CApath: /etc/ssl/certs
• TLSv1.3 (IN), TLS handshake, Server hello (2):
• TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
• TLSv1.3 (IN), TLS handshake, Certificate (11):
• TLSv1.3 (IN), TLS handshake, CERT verify (15):
• TLSv1.3 (IN), TLS handshake, Finished (20):
• TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
• TLSv1.3 (OUT), TLS handshake, Finished (20):
• SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
• ALPN: server accepted h2
• Server certificate:
• subject: CN=ietf.org
• start date: Jul 30 02:42:38 2024 GMT
• expire date: Oct 28 02:42:37 2024 GMT
• subjectAltName: host "datatracker.ietf.org" matched cert's "*.ietf.org"
• issuer: C=US; O=Let's Encrypt; CN=E5
• SSL certificate verify ok.
• using HTTP/2
• h2h3 [:method: POST]
• h2h3 [:path: /api/submission]
• h2h3 [:scheme: https]
• h2h3 [:authority: datatracker.ietf.org]
• h2h3 [user-agent: curl/7.88.1]
• h2h3 [accept: /]
• h2h3 [content-length: 193329]
• h2h3 [content-type: multipart/form-data; boundary=------------------------f11fb582d10e47a8]
• Using Stream ID: 1 (easy handle 0x55ab73d83580)

POST /api/submission HTTP/2 Host: datatracker.ietf.org user-agent: curl/7.88.1 accept: / content-length: 193329 content-type: multipart/form-data; boundary=------------------------f11fb582d10e47a8

• TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):

• TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):

• old SSL session ID is stale, removing

• Connection #0 to host datatracker.ietf.org left intact curl: (55) Failed sending data to the peer [1] 21605 exit 55 curl -v -sS -D versioned/.draft-ietf-opsawg-pcapng-02.upload -F -F obiwan-work/opsawg/pcapng mcr 10030 %curl --http1.1 -v -sS -D versioned/.draft-ietf-opsawg-pcapng-02.upload -F @.*** -F @.***/draft-ietf-opsawg-pcapng-02.xml https://datatracker.ietf.org/api/submission

• Trying [2606:4700::6810:2d63]:443...

• Connected to datatracker.ietf.org (2606:4700::6810:2d63) port 443 (#0)

• ALPN: offers http/1.1

• TLSv1.3 (OUT), TLS handshake, Client hello (1):

• CAfile: /etc/ssl/certs/ca-certificates.crt

• CApath: /etc/ssl/certs

• TLSv1.3 (IN), TLS handshake, Server hello (2):

• TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):

• TLSv1.3 (IN), TLS handshake, Certificate (11):

• TLSv1.3 (IN), TLS handshake, CERT verify (15):

• TLSv1.3 (IN), TLS handshake, Finished (20):

• TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):

• TLSv1.3 (OUT), TLS handshake, Finished (20):

• SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

• ALPN: server accepted http/1.1

• Server certificate:

• subject: CN=ietf.org

• start date: Jul 30 02:42:38 2024 GMT

• expire date: Oct 28 02:42:37 2024 GMT

• subjectAltName: host "datatracker.ietf.org" matched cert's "*.ietf.org"

• issuer: C=US; O=Let's Encrypt; CN=E5

• SSL certificate verify ok.

• using HTTP/1.1

POST /api/submission HTTP/1.1 Host: datatracker.ietf.org User-Agent: curl/7.88.1 Accept: / Content-Length: 193329 Content-Type: multipart/form-data; boundary=------------------------49a22a8f68b6cd73

• We are completely uploaded and fine
• TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
• TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
• old SSL session ID is stale, removing < HTTP/1.1 400 Bad Request < Date: Sat, 10 Aug 2024 19:29:16 GMT < Content-Type: application/json < Content-Length: 89 < Connection: keep-alive < cross-origin-opener-policy: unsafe-none < referrer-policy: strict-origin-when-cross-origin < strict-transport-security: max-age=3600; includeSubDomains < vary: origin < x-content-type-options: nosniff < x-frame-options: SAMEORIGIN < CF-Cache-Status: DYNAMIC < Server: cloudflare < CF-RAY: 8b1264c84c33a1db-YYZ < alt-svc: h3=":443"; ma=86400 <
• Connection #0 to host datatracker.ietf.org left intact {"error": "Validation Error", "messages": ["Invalid revision (revision 03 is
• expected)"]}%

[rjsparks]

Hi @mcr - just in case, double-check the contents of the file you are uploading (and that you're uploading the file you intend)? It has a filename that indicates -02, which is already in the draft repository.

[jennifer-richards]

curl 7.88.1 (x86_64-pc-linux-gnu) libcurl/7.88.1 OpenSSL/3.0.13 zlib/1.2.13 brotli/1.0.9 zstd/1.5.4 libidn2/2.3.3 libpsl/0.21.2

That is the version that was reported as having an issue with HTTP/2 in the curl ticket. If it's the same issue (unclear, given the PUT vs POST change), then 8.2.1 should be clear of the problem.

@rjsparks, I think those logs are just meant to demonstrate that the --http1.1 flag avoids the curl: (55) Failed sending data to the peer error that's in the first but not the second.

[rjsparks]

Got it - sorry for the noise

[mcr]

I guess that I just got upgraded to the broken version when I last patched desk/laptop, probably for critical SSH bug. I don't see another update in Debian Stable to curl yet. I can certainly leave --http1.1 in somehow. Annoying. Maybe MT can put a patch in for now.

[mcr]

trying to submit draft-ietf-anima-brski-cloud, version -09. This one uses my 10 line Makefile, and I added --http1.1 easily, but despite appearing to succeed, the status says "cancelled", and I got no emails.

It could be a different problem.

curl --http1.1 -S -F @." -F @.;type=application/xml" https://datatracker.ietf.org/api/submission | jq % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 106k 100 144 100 106k 43 32863 0:00:03 0:00:03 --:--:-- 32904 { "id": "144937", "name": "draft-ietf-anima-brski-cloud", "rev": "09", "status_url": "https://datatracker.ietf.org/api/submission/144937/status" } obiwan-work/anima/brski-cloud mcr 10054 %make submit curl --http1.1 -S -F @." -F @.;type=application/xml" https://datatracker.ietf.org/api/submission | jq % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 106k 100 144 100 106k 169 125k --:--:-- --:--:-- --:--:-- 125k { "id": "144938", "name": "draft-ietf-anima-brski-cloud", "rev": "09", "status_url": "https://datatracker.ietf.org/api/submission/144938/status" } obiwan-work/anima/brski-cloud mcr 10054 %date Mon 12 Aug 2024 01:21:28 PM EDT

obiwan-work/anima/brski-cloud mcr 10055 %curl https://datatracker.ietf.org/api/submission/144938/status {"id": "144938", "state": "cancel", "state_desc": "Cancelled"}% obiwan-work/anima/brski-cloud mcr 10056 %

[rjsparks]

Yes, it's a different issue (and we're tracking making the errors returned when something goes wrong more actionable).

In this case, please look in your source for what might have led to this:

lxml.etree.DocumentInvalid: IDREF attribute target references an unknown ID "considerationsofor-http-redirect"

[jennifer-richards]

Closing, conclusion is that this was a bug in curl (link to their ticket is in the discussion)