unexpected results from http proxy in new_externals branch

[jtippett]

Hi,

Having set up a proxy handler as described in http://davispj.com/2010/09/26/new-couchdb-externals-api.html i am seeing unexpected results. When I try to access another web server running at the URL defined I get this: {"error":"case_clause","reason":"undefined"}

Any clues? It is definitely connecting; I can see the access in the proxied-to server's logs, and if that is not running I get a "connection failed" type error. But it's just not passing through the results as I would have expected.

[davisp]

Is there any sort of traceback in your error logs that you can paste here? If there's no log can you increase the verbosity to debug and try again?

Thanks for testing this out.

[jtippett]

Hi, the error in logs looks like this:

[debug] [<0.762.0>] 'GET' /_sys {1,1} from "127.0.0.1"
Headers: [{'Accept',"application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5"},
          {'Accept-Encoding',"gzip, deflate"},
          {'Accept-Language',"en-us"},
          {'Cache-Control',"max-age=0"},
          {'Connection',"keep-alive"},
          {'Host',"0.0.0.0:5984"},
          {'User-Agent',"Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5"}]
[debug] [<0.762.0>] OAuth Params: []
[error] [<0.762.0>] Uncaught error in HTTP request: {error,{case_clause,undefined}}
[info] [<0.762.0>] Stacktrace: [{couch_httpd_proxy,stream_response,3},
             {couch_httpd,handle_request_int,5},
             {mochiweb_http,headers,5},
             {proc_lib,init_p_do_apply,3}]
[info] [<0.762.0>] 127.0.0.1 - - 'GET' /_sys 500
[debug] [<0.762.0>] httpd 500 error response:
 {"error":"case_clause","reason":"undefined"}

It says it's a 500 error response but the log from the (rails) app being hit by the request reports a 200, and responds normally to a conventional browser access.

Is there something in complex HTML that could trip it?

In further testing I created a special method which return nothing except the text "ok", producing a different error:

[debug] [<0.206.0>] 'GET' /_sys/media/ok {1,1} from "127.0.0.1"
Headers: [{'Accept',"application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5"},
          {'Accept-Charset',"ISO-8859-1,utf-8;q=0.7,*;q=0.3"},
          {'Accept-Encoding',"gzip,deflate,sdch"},
          {'Accept-Language',"en-US,en;q=0.8"},
          {'Cache-Control',"max-age=0"},
          {'Connection',"keep-alive"},
          {'Host',"localhost:5984"},
          {'User-Agent',"Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3"}]
[debug] [<0.206.0>] OAuth Params: []
[debug] [<0.206.0>] Minor error in HTTP request: {error,retry_later}
[debug] [<0.206.0>] Stacktrace: [{couch_httpd_proxy,handle_proxy_req,2},
             {couch_httpd,handle_request_int,5},
             {mochiweb_http,headers,5},
             {proc_lib,init_p_do_apply,3}]
[info] [<0.206.0>] 127.0.0.1 - - 'GET' /_sys/media/ok 500
[debug] [<0.206.0>] httpd 500 error response:
 {"error":"error","reason":"retry_later"}

Hope this helps... thanks a lot for the externals work, very exciting : )

---

update 1: i have realised that the "try again later" is due to too many failures, not because of any difference in the HTML being served.

Further interesting data: In the example local.ini in your branch, there is an example for proxying google. For me, this works perfectly at http://127.0.0.1:5984/_google/ but fails with a redirect loop when attempted at http://localhost:5984/_google/. There's no difference between 127.0.0.1 and localhost when it comes to my local proxying attempts though.

[davisp]

Ok, I'm pretty sure I know what the first error is. Can you describe the config for the second a bit more? I can't see the difference you're pointing out.

[davisp]

I pushed a new version of the proxy patch that should behave for the first error you had. Let me know how it works out for you.

[jtippett]

Awesome. The first error looks to be fixed now, I will continue testing but all looks good.

The second - well I'm not even sure it is an error. The config is simply the default example you have provided in local.ini:

[httpd_global_handlers]
_google = {couch_httpd_proxy, handle_proxy_req, <<"http://www.google.com">>}

If I attempt to access that in this manner:

http://localhost:5984/_google/

I get a redirect loop. Accessing it in this manner:

http://127.0.0.1:5984/_google/
http://0.0.0.0:5984/_google/

works as expected.

As I said I am not even sure that is a bug; I certainly do not care about it, but just thought you might want to know : )

Cheers and thanks again

James

[davisp]

Just started playing with trying to get the proxy to be able to run the entire Futon test suite. Looks like that second error is due to me stripping trailing slashes on the URL path. I'll let you know when I get it fixed.

Thanks for the reports.

[davisp]

Pushed a preliminary fix for that redirect loop. Still doesn't run the test suite. I'll try and get the entire test suite passing tomorrow.