clj-http icon indicating copy to clipboard operation
clj-http copied to clipboard

Async requests errors

Open borkdude opened this issue 7 years ago • 1 comments

I'm trying to run async requests with clj-http:

(comment

  (require '[clj-http.client :as client])
  (require '[clj-http.conn-mgr :as conn])
  (require '[clj-http.core :as http-core])
  (def base "https://example.com")
  (def links (for [i (range 0 10)]
               (str base "/" i)))
  (def resps (atom []))
  (def errors (atom []))
  (time
   (client/with-async-connection-pool {:timeout 5 :threads 4 :insecure? false :default-per-route 10}
     (do
       (reset! resps [])
       (let [;; acm (conn/make-reuseable-async-conn-manager {})
             ;; ahclient (http-core/build-async-http-client {} acm base nil)
             handle-response (fn [res] (swap! resps conj res))
             handle-failure (fn [res] (swap! errors conj res))
             futures (for [l links]
                       (do
                         (debug "Getting" l)
                         (client/get l
                                     {:async? true
                                      ;; :connection-manager acm
                                      ;; :http-client ahclient
                                      }
                                     handle-response handle-failure)))]
         ;; wait for all futures to finish
         (doseq [f futures]
           (.get f))))))
  (count @resps)
  (count @errors)
  
  )

I keep seeing errors like:

Jul 19, 2018 9:00:55 PM org.apache.http.impl.nio.client.InternalHttpAsyncClient run
SEVERE: I/O reactor terminated abnormally
java.lang.IllegalStateException: Illegal state ACTIVE
	at org.apache.http.util.Asserts.check(Asserts.java:46)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:313)
	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:194)
	at clj_http.conn_mgr.proxy$org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$ReuseableAsyncConnectionManager$fff3515b.execute(Unknown Source)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
	at java.lang.Thread.run(Thread.java:745)

and

org.apache.http.ConnectionClosedException: Connection closed unexpectedly
  java.util.concurrent.ExecutionException: org.apache.http.ConnectionClosedException: Connection closed unexpectedly

Any pointers?

borkdude avatar Jul 19 '18 19:07 borkdude