🚧 Monorepo - disruption to install from source / development workflows

[ErisDS]

Hey everyone 👋

A big heads up, we're actively working on changing Ghost into a monorepo - it's done but the cleanup and fallout is not handled yet.

As such, the documentation for install from source is out of date, and many of our tools don't work quite how they did a week ago.

If you urgently need a hand to get a source install working, let us know. Otherwise the work should be complete in another 2-3 weeks.

[ErisDS]

Some updates:

The install from source documentation has now been updated.

We've now merged the previously-separate Ghost-Admin repo into this monorepo - it lives at ghost/admin instead of being a submodule.

This makes it significantly easier to get a source install off the ground and to contribute to ghost.

One gotcha - if you need to run a knex-migrator command, it needs to be run from inside ghost/core or form the top level, you need to prefix the command with yarn.

We're working on ironing out all the issues, but if you spot anything that isn't working please let us know!

[rbtnn]

Excuse me. How do we make a zip file from this Monorepo? I used following commands and installed the custom build.

yarn build release
ghost install --zip path/to/file.zip

[daniellockyer]

Hey @rbtnn 🙂 We're moving away from zip files in favor of .tgz file produced by npm pack. There's still a bit of friction, but the easiest solution is:

1. From the latest Ghost main, run npm pack inside ghost/core => this produces a ghost-X.X.X.tgz file in ghost/core
2. Ensure you have the latest Ghost-CLI, which supports importing .tgz files
3. ghost install local --archive path/to/archive.tgz

[rbtnn]

@daniellockyer Thank you so much for your advice! I'll wait until this Monorepo is stable becuase I customaize both TryGhost/Ghost and TryGhost/Admin.

[daniellockyer]

TryGhost/Admin has been merged into this repository under ghost/admin - https://github.com/TryGhost/Ghost/tree/main/ghost/admin 🙂

[rbtnn]

Are .zip file and .tgz file produced by npm pack in the same directory structure?

[daniellockyer]

They are not - the .zip has all of the files in the top-level, but the .tgz has them inside package/ as npm pack and NPM packages expect. That said, .zip is deprecated and all tooling will focus on .tgz from now on

[rbtnn]

.zip is deprecated and all tooling will focus on .tgz from now on

I see.

[JuanDelgadillo]

Hey folks!

I faced the same issue generating the custom zip file for my site after the Ghost migration into a monorepo.

I created a Github release action that helps me convert the build file from tgz to zip.

I will be using it till Ghost team finishes ironing out remaining issues from the monorepo, and zip to tgz transitions.

This is the Github action in case you are wondering how it works:

name: Release
on:
  push:
    tags:
      - '*'
jobs:
  automate:
    runs-on: ubuntu-18.04
    env:
      RELEASE_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      GITHUB_REF_NAME: ${{ secrets.GITHUB_REF_NAME }}
      FORCE_COLOR: 1
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
          token: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
          submodules: true
      - uses: actions/setup-node@v3
        env:
          FORCE_COLOR: 0
        with:
          node-version: '16.14.2'
          cache: yarn

      - run: yarn

      - run: npx --yes daniellockyer/monobundle
        working-directory: ghost/core
      
      - run: npm pack
        working-directory: ghost/core

      - run: mkdir -p .dist/release/
        working-directory: ghost/core

      - run: tar -xvzf "./ghost-$(echo $GITHUB_REF_NAME | tr -d "v").tgz" -C .dist/ 
        working-directory: ghost/core

      - run: cd .dist/package && zip -r "../release/Ghost-$(echo $GITHUB_REF_NAME | tr -d "v").zip" ./*
        working-directory: ghost/core

      - name: Check out action
        uses: actions/checkout@v2
        with:
          repository: juandelgadillo/action-ghost-release
          ref: main
          token: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
          path: ./.github/actions/action-ghost-release
      - name: Run my action
        uses: ./.github/actions/action-ghost-release

You might need to adapt it to your needs but the more important commands are:

• yarn

  • This install all npm dependencies for Ghost core and Admin

• npx --yes daniellockyer/monobundle

  • This packages required dependencies for the project to work properly and places them under components folder

• npm run pack

  • This generates the tgz file

• tar -xvzf "./ghost-$(echo $GITHUB_REF_NAME | tr -d "v").tgz" -C .dist/

  • This decompress the tgz file

• And, cd .dist/package && zip -r "../release/Ghost-$(echo $GITHUB_REF_NAME | tr -d "v").zip" ./*

  • This generates a zip file that ghost-cli understands

  The variable $GITHUB_REF_NAME references my own semver version.

[ErisDS]

I will be using it till Ghost team finishes ironing out remaining issues from the monorepo, and zip to tgz transitions.

Can you let us know what details you're waiting for us to fix? As far as we are concerned the transition to .tgz is done & everything is working.

[blueinversion]

After this change, what are the steps to deploy a custom fork to production?

I couldn't find any docs on this, they all refer to local or dev builds.

From the latest Ghost main, run npm pack inside ghost/core => this produces a ghost-X.X.X.tgz file in ghost/core

Should I run yarn before this? How do I install the dependencies?

I'm also confused whether grunt release to generate archives is still recommended/required.

ghost install local --archive path/to/archive.tgz

What is the alternate command meant for production installations?

The ghost-cli documentation has no mention of the --archive option or .tgz. In fact, commands using .zip are still there.

I'm a little lost, any hints to nudge me towards the right direction would be appreciated, thanks!

[JuanDelgadillo]

Can you let us know what details you're waiting for us to fix? As far as we are concerned the transition to .tgz is done & everything is working.

Hi Hannah,

Actually, by the time of writing my previous message, I wasn't aware of this commit from Daniel: https://github.com/TryGhost/Ghost-CLI/commit/a079cdc051bd818e3dd3f53cfc8665f818f6e80e

That commit adds Ghost CLI support I was waiting for so that I could update my instances using the new .tgz package and Ghost CLI.

I was able to install my custom Ghost v5.9.4 build after I updated my Ghost CLI to latest version v1.22.0, which includes the --archive flag. The update worked without issues.

This commit https://github.com/TryGhost/Ghost/commit/89106bce1c4d8dc52ea693a7b846076cfc7038e9 also simplified how my Github action works.

So Daniel has fixed all remaining issues I was waiting for.

Thanks to Daniel, to you, and to the rest of the Ghost team for fixing those remaining issues, and also for adding such great features to the product, lately. It's very interesting and insightful to watch commit by commit. Great job.

[yaizudamashii]

With removal of grunt release is there any other comparable command added to yarn? We need to build custom fork for our production and the dev build has problems with stripe webhooks.

[blueinversion]

With removal of grunt release is there any other comparable command added to yarn? We need to build custom fork for our production and the dev build has problems with stripe webhooks.

I had the same concerns. I found this Github action useful, haven't got around to testing it yet. But I think it should have all the steps necessary.

Still waiting for an official documentation on this.

[MohdMusharraf]

I followed install from Source doc but my Casper theme is not activating.

[daniellockyer]

Hello all 🙂 Thank you for your patience whilst we've been converting the repo to a monorepo. Hopefully you're finding the experience a lot easier now all the code is in one place.

To make some of the processes smoother, I've added a yarn build command, which outputs a .tgz file. This file is in the same format as the one we publish to npm, so it can be installed with ghost install --archive ghost-x.x.x.tgz. This is documented on the source install page and the Ghost-CLI docs.

If anyone runs into any further issues with the new development workflow - please let me know 🙂