Custom URL for your Drummer blog

[scripting]

There are a number of people who want to use Drummer to host a blog using their own domain.

The instructions on the blogging page only worked for the home page, the subordinate page links would still point to oldschool.scripting.com.

I thought we'd have to use custom installs of drummerCms to solve the problem, but along the way I found a simpler approach which I have implemented.

Basically the instructions on the Blogging page now work, with an additional step.

There's a new head-level attribute called urlBlogWebsite, which tells drummerCms where the links should point.

It works. I've set it up for clueless.lucky.wtf. Click around. The links all stay within the domain.

Now, you still need a server, and need to set up PagePark. But this is a lot easier than setting up drummerCms.

Please let me know if it works or you have problems.

[dcoletta]

Getting closer! After reconfiguring PagePark and setting up AWS credentials, I now get this error from PagePark when I try to fetch blog.colettas.org:

4:55:14 PM GET blog.colettas.org:80 /  pool-96-237-62-173.bstnma.fios.verizon.net
s3ListObjects: error == Access Denied

Seems like I should first need to create a bucket and set permissions on it before this can work. I hunted around in https://github.com/scripting/oldSchoolBlog for a while to see if I could figure this out, but gave up.

Can you tell me how to configure this bucket? And/or whatever else I'm still missing?

[scripting]

As far as I know you should not need to set up a bucket or anything like that.

WHat this is probably saying is that the S3 software isn't finding your credentials file, or it doesn't like what's in it.

Did you see the howto I posted in the Change Notes outline?

[dcoletta]

Yes, I followed that howto -- thank you for that.

I'm sure it is finding my credentials file, because if I move the file out of the way I go back to the error: s3ListObjects: error == Missing credentials in config and if I restore the credentials file, the error is: s3ListObjects: error == Access Denied

It's possible I generated the wrong kind of credentials. I'll dig into that a little and see.

[jackbaty]

I'll try this as soon as I can, so thanks.

But before I do that, I want to confirm that this couldn't be as "simple" as running a reverse proxy (via Caddy in my case). I've tried proxying one of my domains to oldschool.scripting.com, and it sort of works, but I get a 404. It may be a host header problem. Assuming I can figure out how Caddy does this, is this approach even feasible? Caddy is a terrific, simple web server that can be a reverse proxy in like 3 lines, so that would be cool. Please delete this comment if it confuses the thread too much.

[tmanternach]

I got this to work using nginx on my domain's server. It's a really simple code block in nginx:

server {
  listen 80;
  server_name trevormanternach.com www.trevormanternach.com;
  location / {
    proxy_pass http://oldschool.scripting.com/trvr/;
  }
}

Drummer doesn't seem to support https at all though, is that correct?

[tmanternach]

@jackbaty I've never used Caddy before, but I just hacked around enough to come up with this Caddyfile:

http://localhost:2017 {

    rewrite / /trvr
    reverse_proxy oldschool.scripting.com:80 {
      header_up Host {upstream_hostport}
      header_up X-Forwarded-Host {host}
    }
}

I can then browse to http://localhost:2017 and it loads my blog. You'd just need to put your own domain (and probably port 80) on the first line (leave http:// in there) and then replace my rewrite line with your blog path.

[scripting]

You csn also access oldschool.scripting.com content through this URL.

https://s3.amazonaws.com/oldschool.scripting.com

[scripting]

Also if you reach any conclusions on using other software, such as nginx or caddy, please include enough of a howto here so I can include it in the docs. For now I'm moving on to other stuff, I think my programming work here is done.

[jackbaty]

@tmanternach I was so close! Your forwarded host bit got me going locally. Still no luck on a real server, but getting there. Maybe an SSL thing. I don't have a server with nginx installed but may spin one up just to see if it makes a difference. Thanks!

Were you able to use the new urlBlogWebsite attribute and get all internal URLs (rather than oldschool.scripting.com)?

[tmanternach]

@jackbaty It's working for me at http://trevormanternach.com/ right now. Links stay in that same domain. I spent way too long trying to get SSL to work, and I gave up for now. Cheers!

[oevl]

With the new head-level attribute urlBlogWebsite I am able to use Nginx to serve my Drummer outlines from my own domain, http://oevl.info/outlines acting as a proxy to the S3 bucket. No need for rewrite rules.

location /outlines {
         proxy_http_version     1.1;
         proxy_redirect off;
         proxy_set_header       Host "oldschool.scripting.com";
         proxy_set_header       X-Real-IP $remote_addr;
         proxy_set_header       X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_intercept_errors on;
         proxy_pass             http://oldschool.scripting.com/oevl/;
         }

[mistersugar]

@jackbaty @oevl @tmanternach

I've lost track of what you're doing as you try these various strategies. Are you doing this still with PagePark? Or are you using Nginx or Caddy instead of PagePark?

My sites are hosted by Opalstack, and I've been successful in using PagePark for other uses. Eventually I'll want to learn from you all so I can use my own domain for my Drummer blog.

[oevl]

@mistersugar I am not using PagePark, just Nginx with a configuration that serves content from the S3 bucket that Dave shares with me as a Drummer user (oldschool.scripting.com/oevl) like it was hosted at my own domain.

[am1t]

I am so close, but might have missed an aspect. I managed to get the domain configured properly. However, I believe I missed the trailing slash / when I first configured the urlBlogWebsite. So, the baseUrl configuration is incorrect in the generated index.html. It isn't replaced even after fixing the configuration and rebuilding the blog. I have tried the below things without any success.

1. Changed the value in OPML header.
2. Deleted and added the header parameter again
3. Modified every page before rebuilding.
4. Tried hard reload for the pages

Nothing seems to regenerate index.html with correct baseUrl. So, as things stand, I wait till the whole index.html is rebuilt? The custom domain is available here.

[scripting]

@am1t — the index.html file is always rebuilt.

[am1t]

Thank you, Dave, for the confirmation. Here's the snapshot of config variable from index.html -- specifically the baseUrl and link params are without training /. I believe that is messing up the Permalink for individual posts?

[jackbaty]

@oevl Your nginx config works well also. Thanks for sharing. It proxies fine using my domain.

However, like @am1t, baseUrl does not reflect my urlBlogWebsite setting so the links still go to oldschool.scripting.com. I am seeing the setting in opmlHead: `"urlBlogWebsite": "http://rudimentarylathe.org/",

I thought it was due to my using a custom template so I switched back to the default. No change. (I didn't see any commits to the drummerCMS repo and the templates never mention baseUrl, but thought it worth a shot anyway.)

If you find a way to kick it so that it uses "urlBlogWebsite" for links, let me know.

[am1t]

@jackbaty In the same boat. Noting here a behaviour that I observed for reference. I first saw this issue last night -- the changes made to urlBlogWebsite weren't affecting the baseUrl in config variable in index.html. But then they did change when I built the blog again in the morning. So, I thought possibly, the index.html wasn't rebuilt when I made the change the first time.

But Dave has confirmed above that the file is rebuilt every time we build the blog. So, I am not sure now what I am missing.

[scripting]

OK -- I'm up and and at work. It's really early for me.

@am1t -- I'm looking at your rendered blog at: http://oldschool.scripting.com/_am1t/

I clicked on every link on the home page and they all went to http://ol.amitgawande.com2021

So I see the problem.

I'm going to try rebuilding your site myself, so don't be spooked if you see things change. :-)

Update 1

This is what I saw in the log for drummerCms

mustRebuildPage: returning true because it's the index file.
published: /oldschool.scripting.com/_am1t/fb/rss.xml
ping: urlServer == http://rpc.rsscloud.io:5337/ping, urlFeed == http://ol.amitgawande.comfb/rss.xml
published: /oldschool.scripting.com/_am1t/rss.json
ping: urlServer == http://rpc.rsscloud.io:5337/ping, urlFeed == http://ol.amitgawande.comrss.json
published: /oldschool.scripting.com/_am1t/rss.xml
ping: urlServer == http://rpc.rsscloud.io:5337/ping, urlFeed == http://ol.amitgawande.comrss.xml
published: /oldschool.scripting.com/_am1t/index.json
published: http://ol.amitgawande.comindex.html
published: /oldschool.scripting.com/_am1t/homepage.html

Just reporting the data, not sure what to make of it yet.

Update 2

My next approach was to try building @am1t's site on a local copy of drummerCms in the JavaScript debugger so I could step through what it was doing.

But in this case, it correctly built the site. Now when I go to his home page, and click the links, they work.

So @am1t could you please check your site and verify that the links are now correct, and let me know.

Next up I'm going to experiment with @jackbaty's site.

Update 3

I think Jack's site is fixed now too.

I also added a bit of code that checks for the trailing / and if it's not there, adds it.

Now I'm going get back to work on the timeZone stuff, I think I have it figured out.

[am1t]

Brilliant! Thank you, Dave, for looking at and resolving the issue. The links are all correct now. Appreciate the efforts you put in so early in the morning! 👍🙏

[scripting]

@am1t -- you guys are worth it. :-)

[jackbaty]

It works! This is terrific. Thanks Dave! One question, if I may. Does changing to a custom template affect the baseUrl setting at all? I've made a few tweaks to my template and will switch back to that if you confirm that it won't affect the links.

Thanks again, and good morning! :)

[scripting]

@jackbaty -- with software you never know, but with that caveat no -- the custom template and the baseUrl are very far from each other in the scheme of things. Hard to imagine one affecting each other.

[jackbaty]

Just following up that using a custom template didn't affect the baseUrl so that's nice :).

I wrote a few quick notes about this on the wiki

[dcoletta]

I'm still getting this error:

12:16:52 PM GET blog.colettas.org:80 /  pool-96-237-62-173.bstnma.fios.verizon.net
s3ListObjects: error == Access Denied

This is even after I tried switching from an AWS root key/secret to an IAM user with full Administrator access. I can confirm that PagePark is correctly loading the key/secret pair and making a call to AWS, since I can see on the AWS console that the key is being used.

If I knew what resource PagePark was trying and failing to load, I could keep debugging, but I'm not sure how to proceed otherwise.

[scripting]

@dcoletta -- I am now able to focus my attention on this. I'm only one person and I'm being pulled from fire to fire. Hopefully this will settle down soon, and other people will be able to pitch in and help get people up to speed.

Anyway --

The resource that PagePark is trying to access is a bucket in my account called oldschool.scripting.com.

My next step is to go see what permissions are set for that. Back in a bit.

[dcoletta]

Thanks - I understand the demands of just-released beta software based on past experience and I completely appreciate your fractional attention!

Here's a bit more info:

root@ubuntu-pagepark:~/pagePark-master# aws s3 ls oldschool.scripting.com/david_coletta

An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

[scripting]

On the flipside -- you're paying a heavy price for being the first person coming through this territory. ;-)

I was able to access a file in your part of that bucket through the web..

https://s3.amazonaws.com/oldschool.scripting.com/david_coletta/index.json

But then it makes sense that you're not allowed to list the contents of that bucket. I don't people to be able to do that. It would allow a competitor to spam all my users, for example. ;-)

We're getting closer to the answer here.

[scripting]

It didn't fail when I tested it, clearly -- because I am the owner of that resource.

So we should assume your credentials file is correct. Sorry I missed that.

[scripting]

@dcoletta - try using urlSiteContents instead, just for fun, to see what happens...

https://github.com/scripting/pagePark/blob/master/docs/config.md#urlsitecontents-in-configjson