Installer not compatible with DDEV

[BernhardBaumrock]

Short description of the issue

DDEV is a great docker based development environment. I wanted to add it to the list of supported CMSs here: https://ddev.readthedocs.io/en/stable/users/quickstart/

While trying to find the best commands I found out that when using the config option --upload-dirs='site/assets/files' the processwire installer won't run properly.

Expected behavior

It should be able to use --upload-dirs='site/assets/files' in DDEV and still be able to install PW.

Actual behavior

Please see the screenshot of the problem here: https://github.com/ddev/ddev/issues/6381#issuecomment-2296974294

Explanation

The "upload-dirs" config setting tells DDEV to bind-mount that directory to the container. If that is not set, ddev will always show a warning about it being not defined. Also DDEV has a feature called "mutagen" which will create full copies of all files inside the container rather than mounting the whole folder. This is great for performance (very fast load times), but it is not so great when large files have to be synced.

That's why I recommend adding /site/assets/files to the upload_dirs setting, but everybody can add any other folders as well (like site/assets/cache or such).

Optional: Suggestion for a possible fix

In the linked issue above I asked if DDEV can do anything about the issue, but I got the answer that the folder needs to be created on startup so that docker can mount it properly, which makes sense. So I think PW should be a little more specific about the check if PW is already installed and not only check for the "site" folder being present.

For example it could check for /site and /site/config.php and only in that case jump to the next step of the installation.

Steps to reproduce the issue

1. Download PW
2. Create an empty folder /site/assets/files
3. Run the installer

[rfay]

It's not actually that it's not compatible with DDEV, it's that it's using too simplistic a technique to see if the site has been installed. It could look for what's inside the site directory instead of just its existence.

(This is also only when performance_mode: mutagen, which is the default for macOS.) It won't happen on Linux/WSL2. The actual mechanics are described in https://github.com/ddev/ddev/issues/6381#issuecomment-2297225086

[BernhardBaumrock]

It's not actually that it's not compatible with DDEV, it's that it's using too simplistic a technique to see if the site has been installed. It could look for what's inside the site directory instead of just its existence.

Yeah that's what I suggested as possible fix :)

(This is also only when performance_mode: mutagen, which is the default for macOS.) It won't happen on Linux/WSL2. The actual mechanics are described in https://github.com/ddev/ddev/issues/6381#issuecomment-2297225086

Thx for the clarification!

[ryancramerdesign]

@BernhardBaumrock

So I think PW should be a little more specific about the check if PW is already installed and not only check for the "site" folder being present.

I'm not sure I understand, where is this check you are referring to? I'm sure it can be made more specific, I'm just not clear about what/where this is? The installer checks for a /site/install/install.sql and for any site profiles available to install, but I'm not sure if that's what you are talking about or not. It also checks for a /site/assets/installed.php file, and if present, then the installer won't run.

[matjazpotocnik]

I think Bernhard is talking about line 285 in install.php - just the existence of the site folder is not a sufficient check that the site profile is already installed.

[ryancramerdesign]

@matjazpotocnik Thanks. That check from that line is actually:

1. The file /site/install/install.sql does NOT exist; and
2. The directory /site/ does exist.

But that check doesn't lead to anything other than it not asking you to select a profile. A more specific check might be to verify the existence of the /site/assets/installed.php file, like the installer does later. I don't have experience with ddev so not clear if this the sort of thing that's being asked for?

[BernhardBaumrock]

Hey @ryancramerdesign thx for looking into this.

DDEV is a docker-based development environment. You don't need to know anything about docker though, you just need to add a config like this:

name: yourproject
type: php
php_version: "8.2"
webserver_type: apache-fpm
...

Then all you need is to fire the command "ddev start"

So far so good. For better performance there is the "mutagen sync" feature, which syncs files from your local file system to the container. I don't understand the full details, but for better performance one can set this setting when using mutagen:

upload_dirs:
  - site/assets/files

The problem here is that when I run "ddev start" in an empty project it will create that folder, because when the container starts it will mount that directory in the container, which will create that directory if it does not exist.

And that's a problem, because when I then add the files to install PW the folder /site/assets/files already exists and the installer will not run properly (and I can't select a profile, which might exist in /site-rockfrontend/...

So the only thing to make sure is that we can run the installer even when /site/assets/files (and maybe others like /site/assets/logs) already exist.

Checking for /site/assets/installed.php sounds like a good solution!

[ryancramerdesign]

@BernhardBaumrock The installer should be okay with that site/assets/files directory existing, as there's a check for it here: https://github.com/processwire/processwire/blob/dev/install.php#L1036 ... so I think I still don't understand what's needed? Also not clear why checking for site/assets/installed.php is a good solution if the issue is related to site/assets/files?

[ryancramerdesign]

Actually, the more relevant line is probably this one: https://github.com/processwire/processwire/blob/dev/install.php#L1918

The built-in mkdir function counts it as a success if the directory already exists. So whether it exists already or not shouldn't matter from the perspective of the installer. But I don't know if I'm looking at the right thing, as I don't have ddev or understand the issue, so might need you to help me track it down.

[BernhardBaumrock]

Hey Ryan, thank you for coming back to this. Sorry that I still was not clear enough. Here is a short video that shows the problem clearly: https://youtu.be/r7LhhCzCHvg

You don't need DDEV (though I'd highly recommend you try it out - you can switch php versions simply by changing the config file which is great to check for PHP8.4 issues etc.). You just need to create the folder /site/assets/files before installing PW, that's it.

[ryancramerdesign]

@BernhardBaumrock Thanks for the video. I'm not sure I understand everything that's going on in it, but on that screen that said "/site/assets/ is writable" and then said to make /site/modules/ writable, is that the place where you want it to create the /site/assets/files/ directory?

[BernhardBaumrock]

Hey Ryan,

I'm really confused. It's so simple and this thread makes it look so complicated. I already posted 3 simple steps to reproduce it in the very first post.

1. Create an empty folder /site/assets/files in the project root
2. Copy PW files in the project root
3. Run the installer

This is all you need to do to reproduce the error shown in the video. You don't need DDEV to reproduce this.

The only thing why this is an issue with DDEV is that DDEV creates the folder /site/assets/files during "ddev start". This is shown at the very beginning of the video.

DDEV needs this folder to exist to properly mount it. And the problem is that when this folder exists before installation I can not install ProcessWire.

Do you understand now?

[ryancramerdesign]

@BernhardBaumrock While I can see the issue with those steps, this is not compatible with how the installer works. The installer has to be what installs the /site/assets/files because it's a directory that has to be writable (and its contents) by Apache. The permissions used are specified by you as one of the steps during installation. The installer has to place the files into it while setting their permissions consistent with your settings.

If you wanted the installer to install your files, you'd instead want to place them in /site-profilename/install/files/ or if you've already renamed /site-profilename/ to /site/ on your own, then it would be /site/install/files/. Furthermore, if you've already renamed the profile site directory to /site/, then it's also fine to have a /site/assets/files/ in there as well.

But it seems that you are trying to just have a /site/assets/files/ on its own with nothing else from the profile, while having a separate /site-profilename/, and that's not going to work with the installer. If it did, it would mean that the installer would have to create/copy all the other directories into your existing /site/ directory, and that would mean that Apache can write to files it shouldn't be allowed to (depending on the server).

[BernhardBaumrock]

Hey @ryancramerdesign thx for coming back to this. It seems we are now speaking the same language.

Sounds like the easiest solution is to first boot up DDEV, then install PW, then change the DDEV config to add the /site/assets/files "upload_dirs" setting, and then restart DDEV to register those changes.

I'd hoped there was a solution that worked without some manual steps involved. Thank you nevertheless.