Cannot use "Customizations to source" option, fatal error

[Chadnic]

Summary

When you use the yml file option

### If you want to perform customizations to the source and have access to it, then uncomment this line - This includes modules

and properly comment out the other two lines the following error pops up and FreeScout is wholly inaccessible:

Warning: Trying to access array offset on value of type null in /www/html/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php on line 159

Fatal error: Uncaught ReflectionException: Class "view" does not exist in /www/html/overrides/laravel/framework/src/Illuminate/Container/Container.php:752 Stack trace: #0 /www/html/overrides/laravel/framework/src/Illuminate/Container/Container.php(752): ReflectionClass->__construct('view') #1 /www/html/overrides/laravel/framework/src/Illuminate/Container/Container.php(631): Illuminate\Container\Container->build('view') #2 /www/html/overrides/laravel/framework/src/Illuminate/Container/Container.php(586): Illuminate\Container\Container->resolve('view', Array) #3 /www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(732): Illuminate\Container\Container->make('view', Array) #4 /www/html/vendor/laravel/framework/src/Illuminate/Foundation/helpers.php(110): Illuminate\Foundation\Application->make('view', Array) #5 /www/html/vendor/laravel/framework/src/Illuminate/Foundation/helpers.php(971): app('Illuminate\\Cont...') #6 /www/html/vendor/laravel/framework/src/Illuminate/Foundation/Exceptions/Handler.php(398): view() #7 /www/html/vendor/laravel/framework/src/Illuminate/Foundation/Exceptions/Handler.php(295): Illuminate\Foundation\Exceptions\Handler->renderHttpException(Object(Symfony\Component\HttpKernel\Exception\HttpException)) #8 /www/html/vendor/laravel/framework/src/Illuminate/Foundation/Exceptions/Handler.php(187): Illuminate\Foundation\Exceptions\Handler->prepareResponse(Object(Illuminate\Http\Request), Object(Symfony\Component\HttpKernel\Exception\HttpException)) #9 /www/html/app/Exceptions/Handler.php(53): Illuminate\Foundation\Exceptions\Handler->render(Object(Illuminate\Http\Request), Object(InvalidArgumentException)) #10 /www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(326): App\Exceptions\Handler->render(Object(Illuminate\Http\Request), Object(InvalidArgumentException)) #11 /www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(120): Illuminate\Foundation\Http\Kernel->renderException(Object(Illuminate\Http\Request), Object(InvalidArgumentException)) #12 /www/html/public/index.php(83): Illuminate\Foundation\Http\Kernel->handle(Object(Illuminate\Http\Request)) #13 {main} thrown in /www/html/overrides/laravel/framework/src/Illuminate/Container/Container.php on line 752

Steps to reproduce

Simply use the option to customize source rather than the option to use "stock" Freescout as shown in the example YAML file.

You should be able to customize source and have it stick rather than being deleted every time upon yml changes etc.

This error is present in all modern browser types: Mozilla and Chromium flavors.

My current YML file looks like this for this section:

 volumes:
    ### If you want to perform customizations to the source and have access to it, then uncomment this line - This includes modules
    - ./data:/www/html
    ### Or, if you just want to use Stock Freescout and hold onto persistent files like cache and session use this, one or the other.
    # - ./data:/data
    ### If you want to just keep the original source and add additional modules uncomment this line
    #- ./modules:/www/html/Modules
    - ./logs/:/www/logs

Changing the YML file to this:

 volumes:
    ### If you want to perform customizations to the source and have access to it, then uncomment this line - This includes modules
    # - ./data:/www/html
    ### Or, if you just want to use Stock Freescout and hold onto persistent files like cache and session use this, one or the other.
    - ./data:/data
    ### If you want to just keep the original source and add additional modules uncomment this line
    - ./modules:/www/html/Modules
    - ./logs/:/www/logs

Works fine, no errors and everything functions as expected but there is no way to customize the source and have it stick as it's wiped out on updates or any sort of docker-compose up -d

Have loaded many times and tried on multiple machine types and O/S and this problem is consistent across all of them. Have done updates via docker-compose pull and removed orphans and pruned, and the results are the same.

Any help would be appreciated!

[tiredofit]

I don't experience it on my end and I run a couple installations in this fashion.

Alternatively you can drop your customizations into /assets/custom and they will be copied into the webroot on startup overwriting the stock files.

Even more you can use /assets/custom-scripts which will fire a series of scripts to do whatever you'd like inside the container (After the above /assets/custom overwrites occur).

[Chadnic]

Thank you for the response. I didn't realize there was a way to do customizations in the folder like that.

To use the /assets/custom what does the directory structure need to look like if I need to overwrite a file deep in /www/html?

And the scripts I will certainly be using because the other problem I have is sometimes it won't fetch mail even with the root crontab functioning fine. It does it once at start and then won't do it every minute, BUT if I add the line to the crontab myself and run it as root instead of nginx user (and fixing some of the unwritable file errors that go along with it) then it works every time like butter, but get wiped upon container restart.

P.S. I DID read the note: (Optional) Copy source code over existing source code in /www/html upon container start. Use exact file/folder structure

But when you say exact file/folder structure, do you mean from root? or from /www/html on down. So in other words, do I need a /www/html directory in there as well or do I just start from there and go down.

[tiredofit]

I do it here: https://github.com/tiredofit/docker-freescout/blob/d72acde87d550b73a0ee275cb656e5c062bf17e0/install/etc/cont-init.d/30-freescout#L78 and it means that you would want to put your files in /assets/custom in reference to how they would land in /www/html. Afterwords it also sets the correct permissiosn for you..

Scripts, just make sure that you set the the bit executible.

Most of my images contain this feature so you can stay as close as possible to image upstream yet still have your own configuration, I think it helps :)

[tiredofit]

You also might be really interested in the https://github.com/tiredofit/docker-alpine repository. It is a ridiculously powerful base image that suits the needs of many of my clients and is the base for anything I do. There is lots of flex, specifically with Scheduled tasks, customization of permissions, users, and so on.

[Chadnic]

Thanks! I read in docker-apline that you can put a file directly into /assets/cron and it will pick it up on container restart and it works perfect. Does /assets/cron get fully wiped when you do a docker-compose up -d or do an upgrade? Because right now it is adding the cronjob I want perfectly and fetching mail is working every time no matter how many times I stop and or restart the container. This is the first time I've had that consistency.

For the scripts, I put my file: crontabup.sh and made it executable. I made a directory in /assets called custom-scripts and dropped the crontabeup.sh in there. When I run crontabup.sh manually it runs fine and adds the job. But it never runs upon container restart. Do I need to define something in the docker-compose.yml file under volumes? Such as:

./custom-scripts/:/assets/custom-scripts and put this under my volumes: section?

Lastly, I can see that this Alpine image is hugely powerful. I just downloaded it and may try to run this. I'm surprised I haven't seen more like this. I used to love Pico BSD on a diskette. This Alpine image and its portability are awesome.

P.S. I noticed there is no /assets/custom folder by default, which makes me think it needs to be mapped in the yml file like logs and modules are but there is no syntax example in the examples folder. I think the same is true for custom-scripts, I am just unsure of the syntax to use.

[tiredofit]

Yes - you would need to export volumes as you eluded to to be able to them let them persist between restarts. Docker does some weird things when your actual host restarts, I have a couple watchguards in there to not do certain things after the container "hot starts", I always recommend to start the containers up again after a cold host start.

You'd also map the /assets/custom folder as well with the docker -v /yourdir:/assets/custom or however you are doing things command to make sure it was available inside the container.

Also watch out for the CRON_YOURCOMMAND="* * * * * date > /tmp/randomfile" - You can get a lot done within an environment variable without having to hack around inside the container. Theres obviously limitations, but now you have a few ways of getting things done.

My images are (obviously) opinionated in how I do things, but I think you and I may be cut from a similar piece of fabric. It's flexible to get stuff done for most people and then customizable for when you need it to be a bit different. I'm serving a couple thousand people daily with great success.

[Chadnic]

Thanks for the direction. I am working on getting Alpine up and running now. I love the idea of moving a set of directories around (Custom and custom-scripts) and keeping the rest of my environment pristine. I've already begun thinking of ideas of how to keep several dozen containers different, but in sync at the OS level using this. It's awesome.

UPDATE:

I modified my docker-compose.yml file to add:

    - ./custom/:/assets/custom
    - ./custom-scripts/:/assets/custom-scripts

The /custom folder works beautifully. I edit files there, put them in the right directory, and bingo, it works. I will keep all my custom source here.

The /custom-scripts however does not work. I can put a script there and it never fires off. Here is my script called crontabup.sh

#!/bin/bash
(crontab -l; echo "* * * * * php /www/html/artisan schedule:run >> /dev/null 2>&1")|awk '!x[$0]++'|crontab -

Super simple, and works perfectly when I do a simple ./crontabup.sh from the bash prompt. But sitting in the custom-scripts folder it does nothing.

Any ideas?

P.S. Of course THIS time, the cron is working and it's fetching every minute, but that only happens about 25% of the time. Hot start, cold start, reboot the actual machine, nothing seems to make a pattern. If I add the above crontab it works every time, without fail, which I why I wanted to script it in to be 100% sure it's fetching for when I can't be around if there is a crash or some strange restart that I didn't expect.

[tiredofit]

Theoretically that should work - but maybe not being called from the script executor (which is root).

There's another way I use which might work: Take your file with the contents of:

* * * * * php /www/html/artisan schedule:run >> /dev/null 2>&1 and throw it in /assets/cron/whateverfilenameyouwant - You could export a volume (which may give you some weirdness) or in your custom script echo the contents to that file, and upon cron start it will get read. If that makes sense...

[Chadnic]

Makes sense. The /assets/cron directory gets wiped out every time there is an update I believe but not on restart.

I can do what you suggested and create the script in /assets/custom-scripts to write a file to /assets/cron as you suggest, echoing it. Does timing matter or is the cron directory read once in a while to pick up changes?

I'm wary of exporting a volume to a place that is supposed to be overwritten on update but I can try that if all else fails.

[Chadnic]

Follow up:

I got everything working, but it took a bit of guessing and good luck to see what was going on.

So if you write the job to the crontab (it's really redundant because there is already a job there doing the same thing under the nginx user instead) it suffers from the same problem that the default job suffers from. It's supposed to fetch mail every minute, but sometimes it works fine and other times it doesn't. It's hit or miss.

If you add the job manually to crontab AFTER the machine is all booted etc it works pretty much flawlessly and never gets stuck.

This can be accomplished by creating a crontab job that fires off a script. The script should sleep for a minute or two, then write * * * * * php /www/html/artisan schedule:run >> /dev/null 2>&1 as a job back into the crontab. Totally kludgy but works.

The problem with adding in this job afterward running as root (instead of nginx user) is you get the file lock error complaint as some files are not writable by the nginx users. To fix this, I added the following job to run every 30 minutes after an initial delay of adding the job into crontab of about 90 seconds.

*/5 * * * * sudo chown -R nginx:www-data /www/html/storage/framework/cache/data/

This then forces the grant of read/write access to all the files Freescout needs in the cache.

For further development I plan to have the chown job just run once, as I haven't seen a situation yet where after you clear up the first lock errors it ever creates another one.

Unfortunately I am not sure if the custom-scripts ever fire, so I could do the entire thing in a script that is constantly running in the background instead of crontab by using loop and sleep so I will experiment with that next.

[tiredofit]

OK good to know. I know in one of my environments we are using the custom-scripts successfully, so let me know if not working on your end.

I'll also draw my team over to this issue to see if this behaviour is affecting some our own large installation. I believe there are 100+ users and 60 odd mailboxes so I'm sure they will be interested.

You also might be able to get away with prefixing sudo -u ${NGINX_USER} before calling php and it may remove the need for the additional ownership reset job..

[Chadnic]

I was thinking the exact same thing, that it might be the user of the job, but that the job is kicking off at some strange time during container startup and it's just getting stuck. Maybe deleting the job that's there and re-adding it after some delay will get it to work? Not sure yet.

I'll experiment with the custom-scripts directory when I get a chance, but the custom directory is working exactly as promised so that was a huge win for custom pages.

[Chadnic]

UPDATE:

It doesn't matter the script. Nothing you put in /assets/custom-scripts/ fires off no matter how simple of a script. That explains why the writes to the crontab don't work.

[tiredofit]

Weird. Its been a while for me when I built this -

• Can you make sure the scripts are set to executible?
• Whats the name of the script? Can you make it script.sh ?
• If all else fails crawling through the logs with DEBUG_MODE=TRUE is a painful yet enlightening experience. It's going to dump all the script output (of everything so watch out), but if you know where to look (right after mkdir -p /www/logs/freescout) it might help see whats going wrong.

[Chadnic]

I will do this and report back.

The scripts are executable for sure. They work fine when called directly.

I did not name them script.sh however. I didn't realize that was required. I might have misunderstood the requirements. I'll get back on the machine and try renaming it.

[tiredofit]

I just checked for you - https://github.com/tiredofit/docker-alpine/blob/87283b75aedc92709024e2e3079f10d46b7d4f54/install/assets/functions/00-container#L205 yup. It looks for .sh files.

Sorry, I should have mentioned that.

[Chadnic]

Renamed it to script.sh (all the tests have been .sh) and it doesn't work.

I'll mess with the DEBUG later, I need to figure out a work around for this at the moment. I have some ideas.

[Chadnic]

UPDATE:

I can't get the script to fire off so I had another half-baked idea that I would write all the changes I want done into one script, and call the script from the command: option in the docker-compose.yml file. This works sorta kinda, but not really.

What happens is the script runs, but when the script exits, the entire container restarts, every time. I can't figure out why either.

The script itself works perfect. Does a bunch of things correctly and exits. But if I add it to command, I get the reboot loop.

Still working on it.

[tiredofit]

OK let me share one more little gem:

CONTAINER_POST_INIT_SCRIPT If you wish to execute a script in the container after all services have initialized enter the path here. Seperate multiple by commas

Maybe that will help?

[Chadnic]

WOW! That little gem is certainly a diamond. That added a whole new dimension to the complexity I can make for myself. Love it.

I think at this point I actually got all my custom stuff working with that and there seem to be no bugs so far. I will give a final report back and close out if it all works. Here is what it took so far:

using CONTAINER_POST_INIT_SCRIPT in my docker-compose.yml I call script.sh sitting in the /assets/custom-scripts/ directory so it won't get wiped out on update and or container restart.

Interesting behavior now observed: Freescout will not come up if the script.sh is not finished, so if you put in a command that hangs the script (without using & for example) the system will never come up. Placing * * * * * php /www/html/artisan schedule:run >> /dev/null 2>&1 in script.sh causes it to hang. I suspect that Freescout isn't all the way up and the command has nothing to run against so it hangs.

Script.sh moves some files, copies some others and does some magic. It also takes the cronup file, which has crontab jobs in it and moves it to /assets/cron. This directory gets wiped out on update but not on container restart. Rather than mapping the volume to be persistent, I just have script.sh copy over the commands I want to add and they are picked up automagically. The cron job includes * * * * * php /www/html/artisan schedule:run >> /dev/null 2>&1 because of the aforementioned issue with the default cron job not fetching mails correctly all the time, every time. Adding this fixes that, but causes the file lock issue.

So now we have the cron added and safe from updates but we will still have the file lock issue. This can't be fixed by having script.sh running sudo chown -R nginx:www-data /www/html/storage/framework/cache/ because the Freescout isn't even really up yet until this script exits and the file lock hasn't happened yet. No problem, enter script2.sh.

You can't call script2.sh from CONTAINER_POST_INIT_SCRIPT because it will also fire off before Freescout is actually fully up and won't allow Freescout to come up until it exits. So the best bet was to call script2.sh from inside script1.sh and add the & at the end of command and force exit out of script.sh. Inside script2.sh just start with a sleep 90 to give Freescout time to come up and make sure there is a full cycle or two of * * * * * php /www/html/artisan schedule:run >> /dev/null 2>&1 from the cronjob we add right at the beginning of container start and we unlock the files just one time (no need to add a cron to do it over and over) and bingo, it works. Done and done.

Freescout is now fetching mail every minute without error, every time, all the time. I am also able to do a ton of stuff pre-Freescout up and running and post-Freescout up and running adding huge customization. My entire site at this point is extremely customized but safe from updates or container restarts.

Fantastic options in this O/S. I love it.

[Chadnic]

More UPDATES:

Firing off a second script dose still in fact cause Freescout to wait to come up. I put script.sh calling script2.sh and Freescout does wait to come up, so my idea doesn't work as I had hoped. I can't figure out why it's waiting to start but it is. If I put a 20 minute sleep command in the second script Freescout will come up in just a hair over 20 minutes (6-8 seconds later). So basically I'm stuck doing a few work arounds.

I just add a job to crontab by copying over the appropriate file into the /assets/cron/ directory from/assets/custom-scripts/and crontab picks it up. Cronup is the file name in my case without any extension. Take note: not an SH file. Just named cronup. The file is not executable or anything, it's just lines that need to be added to crontab in crontab format, don't use commands that would insert it into crontab, just write it exactly as you would want crontab to run it. Works perfect.

One of the lines I add is a line to fire off a script every 5 minutes. That script however is set to run the command within the script only once. Essentially, when the script is run the first time, it writes out a file called scripthasrun. When the script sees there is a file in the same directory called scripthasrun it exits without running the command again. This keeps it from wasting CPU resources running a command that doesn't need to run again. The CPU resources used to run the command every 5 minutes is essentially 0 because all it does is look for the file, see it, then exit, and never continues.

This has me up and running solidly for 12 hours now and I am spamming mail through it at a high rate and load testing. The custom sorting I built and deliveries to mailboxes inside Freescout seem fine. Custom replies, and everything seems fine. This O/S is amazingly efficient and uses no resources by itself and is scaling quite nicely. I haven't tipped it over yet, even delivering thousands of messages a minute. We will see how many in can handle a second.