webroot option

[nbonfire]

I'd like to put this behind a reverse proxy, so a webroot option in the config would be nice.

For example, https://myserver.com/comicstreamer would redirect through nginx to http://myserver.local:32500/comicstreamer

Right now that doesnt work.

[bonebox]

+1. This project appears abandoned so I doubt it will happen, but here's hoping.

[davide-romanini]

Project is not really dead, but proceeds very slowly. :-) I wonder if a webroot option is really necessary, maybe you can use rewrite rules for the same purpose??

[bonebox]

Glad to know someone is still working on it!

I'm really not too familiar with rewrite rules--do you have an example of how it might work for this? A Google search didn't return anything very useful.

[davide-romanini]

ProxyPass and rewrites are a lot hackish, but very useful. I'm not sure I fully understood the wanted beahviour, but at https://httpd.apache.org/docs/2.2/rewrite/proxy.html there are some informations, at least for apache. Nginx has similar support, you should refer to official documentation for details, for example: http://nginx.org/en/docs/http/converting_rewrite_rules.html, http://nginx.org/en/docs/http/ngx_http_rewrite_module.html.

[bonebox]

Thanks, yeah I've looked at all that. The ProxyPass directives work in conjunction with setting a webroot--I have them working fine with other web services that support webroot, such as Ubooquity, for example. I don't think it's possible to get it working correctly with just rewrite rules. I tried some stuff but none of it works--I can get the root page to show up (without styling, at least) but any links in ComicStreamer are based off a hard coded webroot of / so they break. That's where being able to specify the webroot in ComicStreamer comes into play.

[nbonfire]

Like bonebox said, rewrite rules alone aren't enough. They can convert an incoming url to whatever, but they can't rewrite the html going back to the client. For example, if you set up comicstreamer behind a proxy at myserver.com/comics, when you first load you will be immediately redirected to myserver.com/login (which is already used in my case so I can't just add another rewrite rule to handle that). Even if I could get past that, none of the images would work without a rewrite rule for any arbitrary uri getting redirected to /comics/whatever. So just using rewrite rules could work if comicstreamer was the only site being hosted behind that apache/nginx serverr, but if it were the only site being hosted you wouldn't need rewrite rules. Basically the minimum work that needs to be done to support this is

• a spot in the config for the webroot to be specified
• adjust templates to use the webroot in all links if specified.

The rest can be handled by rewrite rules.

[blindpet]

I would like to see webroot as well when you get around to it. SickRage uses tornado and may provide some code that you can use.

[davide-romanini]

From referenced commit 489cd1e I've added a webroot option, for example:

$ comicstreamer --webroot /comics

will expose all urls at http://localhost:32500/comics/. It's a quick and dirty fix, let me know if you find something broken eventually.

[bonebox]

Awesome, thanks!!! I'll check it out when I get a chance.

[bonebox]

I gave it a try, but haven't been able to get it to work with the same rewrite rules I'm using for my other sites. @davide-romanini did you test this and if so what did you use for rewrite rules? FYI, below is what I'm using. This works for my other sites that I have configured for reverse-proxy (with the webroot folder being different for each, of course).

ProxyPass /comics/ http://localhost:32500/comics/
ProxyPassReverse /comics/ http://localhost:32500/comics/

[nbonfire]

I won't have a chance to try this until the weekend. Did you get an error? Are the images not showing up? If the page loads, can you check the html source to see if the urls are rewritten to include the webroot?

[bonebox]

I can sort of get it to load by pointing to the local ip of my NAS like this http://x.x.x.x/comics/--but the theming is messed up and most links are broken. Adding back in the port fixes it ``http://x.x.x.x:32500/comics/`. It doesn't work at all for me when accessing from my domain which is the reason I need the reverse-proxy in the first place. I just get "page not found" error.

[davide-romanini]

I made a quick test with a python reverse proxy (https://github.com/metajack/tape)

  python tape -P /comics=http://localhost:32500/comics -P /static=http://localhost:32500/static

and it works. Pay attention to slashes (/), I tried three different wrong configurations before this one ;-)

[bonebox]

Yeah, I tried all different types of slashes, including that. The first thing that got me was the config; I mistakenly put just comics since the webroot setup on my other services don't require a slash I assumed it was the same. Looking again at your instructions I realized it needed to be /comics. I don't know what the problem could be, since that same setup works for my other services. I'll be interested to know if anyone else gets it working successfully.

[bonebox]

@davide-romanini out of curiosity when you say "it works", what do you mean exactly? What url are you testing to see that it works?

[bonebox]

Also, I just noticed that this partially breaks integration with Chunky comic reader (@mike-ferenduros). Even with the webroot field empty, Chunky gives a 404 when loading the ComicStreamer browser in-app. The folder navigation still works, though.

[davide-romanini]

I made only a quick test with tape as noted above. All pages and api calls worked for me. Note that also the /static path need to be reversed for correct browser display. Could you post here the relevant CS logs? You can find them in your home under .ComicStreamer/logs. Also your apache access logs could be useful.

[bonebox]

I added the reverse-proxy for /static and I can access it locally now without including the port, so that's progress, at least. I still cannot access it externally from my domain. I get a 400 Bad Request Illegal URL error. Did you test accessing it externally or just internally? BTW, what's the purpose of /static? That seems odd to also have to configure a reverse-proxy for that--is that part of the "quick and dirty fix" you mentioned?

Also, Chunky reader still does not work properly. It's almost the opposite as before--now the web browser will load, but browsing directly by folders (using the API, I'm assuming) doesn't work. Actually, the web browser doesn't really work correctly within Chunky either--once you browse a few links deep or attempt to load a comic, it stops working.

I'll post the logs here in a bit.

[bonebox]

I'm not seeing much in the ComicStreamer.log. It does appear to be seeing my attempt to access externally, because every time I try, I see this in the log:

2015-08-28 10:06:09,602 - DEBUG - 302 GET /comics/ (127.0.0.1) 2.96ms

I also noticed that (some, not all of the time) the url (https://my.domain.com/comics/) gets redirected to https://my.domain.com/login?next=%2Fcomics%2F. I do have authentication enabled, if that is related.

When accessing via Chunky, I see stuff like the below. But it doesn't appear to be logging any errors for the API. For instance when a folder or comic won't load, nothing is getting logged.

15-08-28 10:12:23,157 - DEBUG - 200 GET /comics/scanstatus?api_key=*key_removed* (127.0.0.1) 3.07ms
2015-08-28 10:12:24,267 - ERROR - Could not open static file u'/volume1/@appstore/ComicStreamer/comicstreamerlib/static/blindpet_logo_crop_small.png'
2015-08-28 10:12:24,270 - DEBUG - 304 GET /comics/folders/browse (127.0.0.1) 67.34ms
2015-08-28 10:12:24,313 - DEBUG - 304 GET /comics/folders/?&api_key=*key_removed* (127.0.0.1) 3.05ms
2015-08-28 10:12:24,327 - WARNING - 404 GET /static/blindpet_logo_crop_small.png (127.0.0.1) 10.73ms
2015-08-28 10:12:25,673 - DEBUG - 200 GET /comics/scanstatus?api_key=*key_removed* (127.0.0.1) 3.47ms
2015-08-28 10:12:31,702 - ERROR - Could not open static file u'/volume1/@appstore/ComicStreamer/comicstreamerlib/static/blindpet_logo_crop_small.png'
2015-08-28 10:12:31,705 - DEBUG - 304 GET /comics/folders/browse/0 (127.0.0.1) 57.30ms
2015-08-28 10:12:32,066 - DEBUG - 304 GET /comics/folders/0?&api_key=*key_removed* (127.0.0.1) 325.43ms
2015-08-28 10:12:32,081 - WARNING - 404 GET /static/blindpet_logo_crop_small.png (127.0.0.1) 11.83ms

[bonebox]

I noticed that when I access the web browser from its local ip (which works, as I mentioned) the log is a little different. It is:

015-08-28 10:37:35,848 - DEBUG - 200 GET /comics/ (127.0.0.1) 2047.91ms

Followed by several similar lines like this:

2015-08-28 10:37:36,147 - DEBUG - 304 GET /comics/comic/7525/thumbnail?api_key=key_removed (127.0.0.1) 8.38ms

But when accessed externally it again looks like this (mentioned in previous post):

2015-08-28 10:06:09,602 - DEBUG - 302 GET /comics/ (127.0.0.1) 2.96ms

And it's not followed by anything else in the log.

So the successful load has a 200 GET, and the failed has a 302 GET.

[davide-romanini]

Ok, it definitely requires more testing :-) I confirm that authentication wasn't tested at all so it could be a problem. Could you provide more details of your setup and apache configuration so I can try to reproduce the exact use case?

[bonebox]

I'm running it on a Synology NAS (DS414). What details would you like to know? There are several apache config files that it's using--is there something specific you're looking for?

As I previously mentioned I'm running other services with reverse-proxy just fine. Those are CouchPotato, SickRage, Mylar, Ubooquity and Deluge web client. Here is what my reverse-proxy config looks like for those:

LoadModule ssl_module modules/mod_ssl.so

NameVirtualHost *:443
<VirtualHost *:443>
  ServerName www.mydomain.com
#  SSLCipherSuite HIGH:MEDIUM
  SSLProtocol all -SSLv2
  SSLCertificateFile /usr/local/keys/ssl.cer
  SSLCertificateKeyFile /usr/local/keys/ssl.key
  SSLEngine on
  SSLProxyEngine on
  ProxyRequests Off
  ProxyVia Off
  <Proxy *>
     Order deny,allow
     Allow from all
  </Proxy>
  ProxyPass / https://localhost:5002/
  ProxyPassReverse / https://localhost:5002/

</VirtualHost>

ProxyPass /couchpotato/ http://localhost:5053/couchpotato/
ProxyPassReverse /couchpotato/ http://localhost:5053/couchpotato/

ProxyPass /mylar/ http://localhost:8090/mylar/
ProxyPassReverse /mylar/ http://localhost:8090/mylar/

ProxyPass /sickbeard/ http://localhost:8083/sickbeard/
ProxyPassReverse /sickbeard/ http://localhost:8083/sickbeard/

ProxyPass /ub/ http://localhost:2202/ub/
ProxyPassReverse /ub/ http://localhost:2202/ub/

##Deluge##
<Location /deluge>
RequestHeader append X-Deluge-Base "/deluge/"
ProxyPass http://localhost:8112
ProxyPassReverse http://localhost:8112
Order deny,allow
Deny from all
Allow from all

</Location>

[davide-romanini]

I got a chance to do some more testing:

• started up a VM with ubuntu mate
• installed stock apache2 and enabled mod_proxy
• added the following config to /etc/apache2/sites-enabled/000-default.conf:

ProxyPass /comics/ http://localhost:32500/comics/
ProxyPassReverse /comics/ http://localhost:32500/comics/

• started up comicstreamer ./comicstreamer --webroot "/comics"
• started up a firefox at http://localhost/comics/

All functionalities work as expected, at least at browser level (didn't tried with chunky). At first I also had to proxypass the `/static' path. I just commited a little change to avoid even this step and it seems to work fine.

[bonebox]

Yeah, I can access it internally--from a previous comment:

I added the reverse-proxy for /static and I can access it locally now without including the port, so that's progress, at least. I still cannot access it externally from my domain. I get a 400 Bad Request Illegal URL error. Did you test accessing it externally or just internally?

The problem is I cannot access it externally via my domain like I can with my other services, which is really the main reason for using a reverse-proxy.

[bonebox]

I updated to the latest and was able to remove the proxypass for /static as you mentioned to access locally. But I still have the same problem externally. Just to reiterate, whenever I access externally (https://domain/comics/) the link gets changed to https://domain/login?next=%2Fcomics%2F along with a page not found.

[davide-romanini]

The problem is related to authentication, I've activated it but since the cookie was already set it just didn't redirected me to the login page. I've added the missing webroot piece even to that part, hoping it finally solves your problems.

[bonebox]

That did the trick--thanks!!!

[bonebox]

Unfortunately, Chunky still doesn't work. I'm assuming something probably needs to be updated in the app to support this change. @mike-ferenduros could you please look into this when you get a chance?

[bonebox]

This is working now which is great, but one thing I noticed is I'm using https on my server and it's working ok with my other services, but when I connect to ComicStreamer using https, it doesn't use the encrypted connection. Is there something with ComicStreamer that might be preventing/overriding this?

[nbonfire]

you should be able to use a redirect rule to force https. for example, in nginx I use this in my server block:

rewrite ^ https://$server_name$request_uri? permanent;