Help installing ShaarliGo

[frenchvandal]

Hello,

I really love your work with ShaarliGo, how the interface is clean and minimalist (the dark theme is awesome in your prod link).

Until ShaarliGo is production-ready with all the features you planned are completed, I was willing to give it a try on a domain name I have.

I am not very familiar with Go apps and Apache/lighttp (I am a former nginx user, I use Caddy now which is simpler for TLS).

So my idea was to expose ShaarliGo on a port with Apache, and then reverse proxy it with Caddy. The reverse proxy works fine but I have issues serving ShaarliGo content with Apache.

I guess I missed some step along the way. I have decided to open this issue as it may be useful for anyone else wanting to install ShaarliGo.

So here is what I did:

1. my webspace is /usr/local/apache2/htdocs I uploaded the /app /assets /tpl folders (chmod 755 to my Apache user) and the shaarligo.cgi file (chmod to 555) I downloaded from your link.
2. in the _httpd.conf_, I activated _LoadModule rewrite_module modules/mod_rewrite.so_. In the DocumentRoot Directory directive I changed the options to Options FollowSymLinks ExecCGI and the AllowOverride to All, and I uncommented AddHandler cgi-script .cgi

When I start Apache and hit my url frenchvandal.com the url is rewritten to https://frenchvandal.com/o/p/ but I get a 404 not found error.

In the logs, I see:

shaarligo    | [Sat Apr 25 08:34:43.822852 2020] [autoindex:error] [pid 7:tid 139841987639040] [client 172.18.0.2:40582] AH01276: Cannot serve directory /usr/local/apache2/htdocs/: No matching DirectoryIndex (index.html) found, and server-generated directory index forbidden by Options directive
shaarligo    | 172.18.0.2 - - [25/Apr/2020:08:34:43 +0000] "GET / HTTP/1.1" 403 199
shaarligo    | 172.18.0.2 - - [25/Apr/2020:08:34:44 +0000] "GET /favicon.ico HTTP/1.1" 200 -

I tried different tweaks the conf with different methods. I noticed the favicon is always served with a code 200, the best I managed to get was a 404 error with /assets/404.html in the body.

Am I missing something? Any help is welcome and I would be very grateful.

Also I was wondering a few questions about the ShaarliGo configuration itself but for now my priority is to have the app served correctly.

Thank you very much!

[mro]

@frenchvandal thanks a lot for the report, may that be helpful? https://jung-kurt.github.io/cgi/?

[mro]

I started the publicly editable https://github.com/mro/ShaarliGo/wiki/Caddy-Deployment - maybe we can collect the findings there.

[mro]

@frenchvandal as to the installation procedure – you will need some modules activated (which I don't know precisely yet), but you will only need to copy 1 single file – the shaarli go.cgi. On first request https://frenchvandal.com/shaarligo.cgi it will unpack all required assets (and .htaccess) out of it's belly.

That's what I tried saying at https://github.com/mro/ShaarliGo#install--update

I tested it deploying to a shared webspace Webhoster, that's why I don't know the exact apache config (yet).

[frenchvandal]

I think it is a matter of finding the correct settings in Apache conf, I am almost there!!!

I do see the cgi script at https://frenchvandal.com/shaarligo.cgi but it treats it as a download and does not execute the script:

Regarding Caddy, yes I do know that plugin for CGI. I will do some research but as far as I remember it was written for Caddy v1 and there were some breaking changes in v2, so it is not compatible.

[frenchvandal]

Almost good 👍

[mro]

great progress! The drastic warning means the ./app directory is currently publicly visible – which should be prevented by

1. it's app/.htaccess and
2. the file access permissions of the app/ directory.

[mro]

…and the wording here is German :-( as I was caught by surprise by it's international success :-)

[mro]

btw. there's nothing of real privacy in app unless you enter 3rd party auth-tokens in config.yaml.

[frenchvandal]

It was a bit complicated to conclude the installation.

I had to change the Apache conf before and after executing the cgi script. As of next week, I will write down the steps I did to have the installation completed, and it can be shared in the git wiki for future reference.

It seems now ShaarliGo works plainly at my URL 😄

I do not understand German so it is not easy to navigate in the app. If you need help for i18n, I have a few friends that are fluent in German and can help me for an English and French translations. I am fluent in Portuguese, if needed.

[mro]

@frenchvandal 🚀 awesome! I may do a proper English localization during next week. #20

[mro]

related https://github.com/caddyserver/caddy/issues/29, https://github.com/dbohdan/caddy-cgi

[frenchvandal]

@mro my lighttpd is up. I have some troubles having the configuration file being accepted.

I have copied lighttpd.conf (I changed my var.shaarli_go_path_0) from the repo in static/app/ into /etc/lighttpd/ and I run the web server with the command lighttpd -f /etc/lighttpd/lighttpd.conf

I got the following error:

shaarligo    | 2020-04-28 07:05:29: (configfile.c.1641) a default document-root has to be set
shaarligo    | 2020-04-28 07:05:29: (server.c.1188) setting default values failed

Is that your method too with lighttpd?

Adding a server.document-root = "/var/www/html/" directive does not seem to work.

[mro]

tl;dr: I will look into it and improve the hints in the lighttpd conf template.

You're an unexpected usecase. Before, I assumed users either already had an up-and-running web server, self-hosted, or just bought cheap shared hosting (Apache, without config access that is) – web server also running.

You are pushing into new grounds, both in terms of server administration (which I never intended writing howtos for, I'm just not skilled enough), as well as helping me to straighten the edges in a rough, drafty software.

You mentioned you're familiar with nginx – would it be an idea to prepare a nginx config (which in turn I know nothing about)?

Is that your method too with lighttpd?

my (personal) method with https://l.mro.name or https://demo.0x4c.de/shaarligo is via the lighty module simple_vhost and then renaming app/lighttpd.conf to …/my.example.com/server.conf and editing it.

Without simple_vhost, I recommend putting the conf into /etc/lighttpd/conf-available/, e.g. named 99-shaarligo.conf and enabling it via $ sudo /usr/sbin/lighty-enable-mod shaarligo and again editing it to match the current installation.

Does that make sense?

[frenchvandal]

Yes that makes sense! I will try your suggested method later today.

You were faster than me to comment, I was about to write that I managed to serve a valid lighttpd.conf but I cannot access my webspace to execute the install with shaarligo.cgi. I get a 403 error and the file is downloaded by my browser 😅 Maybe some settings to tune a little bit.

Unlike Caddy, nginx normally comes with a CGI support. I should be able to adapt the conf!

[mro]

hm, that's what https://github.com/mro/ShaarliGo/blob/cad0a124bf71117aec43a71923aa3987e0ba240a/static/app/99-lighttpd-shaarligo.conf#L26 is for – so I guess the whole configuration block isn't active. Maybe the url regex doesn't match?

[frenchvandal]

I think I have to use a vhost too in lighttpd, but I am not sure it will work fine behind a reverse proxy. If I cannot make it work, I will go back to Apache. I may try this solution too.

BTW did you deactivate gzip compression on all ShaarliGo? (in that case I will unload the compression module from the web server)

[mro]

I think I have to use a vhost too in lighttpd,

why do you?

BTW did you deactivate gzip compression on all ShaarliGo? (in that case I will unload the compression module from the web server)

Keep it on. Over the wire there's still gzip compression, That's the web server's business (and decision). Just the index.xml.gz files on disc turned to index.xml – and so there's no more obligation to explicitly declare the http responses Content-Encoding: gzip.

[frenchvandal]

Eventually I have decided to stick with Apache. It may not be as fast as Lighttpd but I have more configuration options. There is a ready-made Docker official image (so less work for me) and, well, at least I managed to make ShaarliGo work!

I have a few questions, maybe you can share your opinion with me.

I will keep the gzip compression on the web server, and I will test brotli compression support.

The thing is brotli compression on-the-fly is slow with higher compression (basically level 4 out of 11 is the same speed as gzip with a better compression rate).

It is a better practice to pre-compress the static assets (extension .br) and serve them with Apache rewrite conditions.

So my idea is to pre-compress with brotli on my server all the text static files (like css, svg, html, xlst) at the max level of 11 (so an insane compression!)

Then if the client browser accepts brotli encoding, the static assets served will be much smaller (and I can do the same method with gzip).

Do you think it is worth it speed/performance wise? It may seem a lot of overthinking, as ShaarliGo is already is light app. I am trying to tweak things to downgrade my VPS to a cheap $5 plan 😅

Also my main fear is to break shaarligo.cgi if all the assets are compressed.

Anyway I will make some tests today so I should have my answer soon!

[mro]

Eventually I have decided to stick with Apache. It may not be as fast as Lighttpd but I have more configuration options. There is a ready-made Docker official image (so less work for me) and, well, at least I managed to make ShaarliGo work!

A working service is what it's all about, any webserver is fine and should be exchangeable later, too.

I have a few questions, maybe you can share your opinion with me.

I will keep the gzip compression on the web server, and I will test brotli compression support.

The thing is brotli compression on-the-fly is slow with higher compression (basically level 4 out of 11 is the same speed as gzip with a better compression rate).

It is a better practice to pre-compress the static assets (extension .br) and serve them with Apache rewrite conditions.

So my idea is to pre-compress with brotli on my server all the text static files (like css, svg, html, xlst) at the max level of 11 (so an insane compression!)

Then if the client browser accepts brotli encoding, the static assets served will be much smaller (and I can do the same method with gzip).

Do you think it is worth it speed/performance wise? It may seem a lot of overthinking, as ShaarliGo is already is light app. I am trying to tweak things to downgrade my VPS to a cheap $5 plan 😅

I test it even on a 2€ shared space https://www.variomedia.de/hosting/easy/ and it should run on the smallest VPS you can get.

Pre-compression is IMO hobby – if it gives you fun, do it. I expect the difference in time/electricity be small. Also impact on usability under bad network conditions may not be visible – but I have no first-hand experience with that. Very much depends on your usage habits. QUIC (caddy…) may be interesting, however.

More impact on UX may have the cache expiry times of the /themes/* – not being transferred means 100% compression :-). I'd raise them to 1 year and forget about asset compression altogether.

Remain the single posts, the feeds and search results. The latter cannot be pre-compressed. The others was, what I did before and turned away from. I started concerned in terms of saving on-disc storage and pre-compressed them all. Meanwhile I find it more important to have layperson inspectable plain-text files, not even gzip compressed. https://github.com/mro/ShaarliGo/issues/22 The web server does (some) transport compression, but still the sizes are ridiculously small from a 'modern' perspective.

The posts & feeds could be compressed, but that adds complexity to the webserver config which I just got rid of. 16e9df0d

Also my main fear is to break shaarligo.cgi if all the assets are compressed.

try recklessly. The cgi should be independent from the assets.

Anyway I will make some tests today so I should have my answer soon!

Nice! Am curious.

[mro]

@frenchvandal the 500 from ~13:38 is resolved?

[frenchvandal]

So far my tests with Brotli have failed. I cannot use Brotli on-the-fly because of my set-up.

In Apache, Brotli needs to be served with http/2, and http/2 needs SSL. Apache listens on port 80 with no SSL, the TLS on port 443 is handled by Caddy. The browser gets http/2 with Caddy, but the requests from Apache to Caddy are in http/1.1. So no real solution here, unless I reuse the certificates from Caddy in Apache. That would add a lot of complexity though!

Gzip on-the-fly is OK, this is pretty standard. I can do the encoding either in Caddy or Apache.

Thanks a lot for your feedback!

Now I do not think it is relevant to pre-encode all xml assets from /o/… This may be a little bit faster for the visitor but it means to have a triple storage of each file (.xml, xml.gz and .xml.br)

For the fun of the challenge, I can try the method just with the theme assets.

@frenchvandal the 500 from ~13:38 is resolved?

Yes all good!

It was an issue with the Alpine image. Alpine-based images are smaller in size, but sometimes they have weird outputs :(

I switched back to the Debian-based Apache image like my first install of last week.

Now that I understand its logic, ShaarliGo is easy to set up, pretty much the default Apache config works fine. I just needed to activate the missing modules like rewrite or cgi. I found all the info in the .htaccess file :)

[mro]

all the info in the .htaccess

maybe the install prose should make clear, that the cgi automatically unpacks everything, including a .htaccess, and thus requires the server's AllowOverride to be at least FileInfo Indexes Limit for 0-config install.

And that this is not a hardened installation, naturally. 🔐

[frenchvandal]

I am almost production-ready!!! I have closed my previous VPS and installed Shaarligo (again and again) on my new cheap VPS. All went fine, success!

There are a few things to tweak/improve but they are not significant. I will raise issues later with a few questions or feature requests.

Here is something that bothers me and, for now, I am stuck 😢

When I login, my session does not persist!

I enter my credentials in the login page, I am redirected to /o/p but I am still considered as a visitor, not logged in. This problem happens in the latest version of Chrome on Windows.

The weirdest thing is that I can login correctly with Chrome, when I am in private navigation mode (New Incognito Window).

I tried on other contexts with my devices:

• Same problem on Chrome/Android
• Safari/iOS logs fine, no issue 😲
• Firefox/Windows logs fine too!
• Edge/Windows is the worst: Shaarligo does not even display correctly (only the theme background, no text, no images)

Do you have any idea? No clue where to start, maybe this is a browser issue, specific to Chrome.

At least I have some workarounds to be able to post, I just need to use a different browser 🤣

P.S. it may be an issue an issue related to my settings in my personal laptop/smartphone, because I can log in Shaarligo with no issues in my work laptop (where I do not sync my personal browser history in Chrome), so no need to dig more I guess.

[mro]

Hi @frenchvandal still stuck or made progress?