Provide a recipe for Google App Engine

[gusev-genotek]

I have deployed yii2 based app to GAE. GAE can autoscale the instances horizontally. This presents some challenges:

1. Using a centralized file storage for all instances. I am using gcsfuse to mount a single google cloud storage bucket for all user generated content etc, however I am yet to find a way to automatically mount a google bucket. As GAE can start and stop the instances (even in manual scaling mode) at will, the mount of the google storage bucket is not guaranteed.
2. Caching and Session handling cannot be done in files, Redis and MemCache are combersome with GAE. Pretty often I get "Exception (Integrity constraint violation) 'yii\db\IntegrityException' with message 'SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '1503914243' for key 'cache_expire_unique'" exception while using DBCache.
3. GAE flexible environment routes requests from a single front end service to the instances via a ngnix server to port 8080. The worker instances with yii2 application are configured with apache2 servers serving 8080. How can this configuration be tuned?

It would be great to have comments on the above and best practice recommendations.

[samdark]

Indeed, it would be great to get info about it. I haven't deployed Yii 2 to GAE so if you did, please share your experience.

[gusev-genotek]

@samdark The above is basically my experience. To that I could add that the deployment is done via a Dockerfile and app.yaml files. What other details shall I provide?

[samdark]

Well, examples of such files would be cool.

[gusev-genotek]

I am attaching 2 Dockerfiles:

1. Base image with our dependencies which are not changing often. Important for this discussion is the gcsfuse part. Currently this image can be found in docker hub as vladgen/yii2-apache-php7-bcm-gd-r-gcs Dockerfile-yii2-gae-base.txt

2. Actual application image using 1. with the application php code that changes often. Dockerfile-yii2-gae-app.txt

3. app.yaml file app.yaml.txt

to deploy this to GAE, place second Dockerfile and app.yaml into your yii2 project, change the settings to suit your app and run, for example

gcloud app deploy app.yaml --quiet --verbosity debug

Once it's deployed, ssh to your GAE instance (via a google web console), run

container_exec gaeapp /bin/bash

to get into your worker instance and then execute one of the gcsfuse mount commands. Since I am using fstab with gcsfuse I do

mount -a

to mount all gs buckets as folders, as described in the /etc/fstab.

The latter is done, by, for example

MY_BUCKET_NAME MY_LOCAL_PATH gcsfuse rw,noauto,allow_other,implicit_dirs,dir_mode=777,file_mode=777

[tunecino]

I don't know much about either GAE or dockers but I've seen a talk about a custom docker images they made for running PHP on their App Engine Flexible Runtime: https://github.com/GoogleCloudPlatform/php-docker

They where also talking about PHP security patches they added to their images. see it here: https://youtu.be/9PedC_6ZC3Q?t=7m19s

[tunecino]

I just deployed a similar template to this to a flex engine which is same as the advanced one except that it has api and auth folders as restful entries so I use no assets or session. But I use Redis within RedisLab (free up to 30mb and they are in the same datacenter) to store my short-living access tokens + I need it for later use with yii2-queue extension.

My steps where:

1. GAE account + create new project + enable billing + download their SDK (better described here: https://cloud.google.com/php/getting-started/hello-world) NOTE: I use their Flexible environment instead of the Standard one as the latter uses php 5.5. read more about differences and tradeoffs of each here: https://cloud.google.com/php/quickstarts

2. I used their SQL Cloud for DB which auto scales. You have to follow any of those steps and note your connectionName somewhere:

• Using Cloud SQL for MySQL with PHP
• Using Cloud SQL for PostgreSQL with PHP

3. For my 2 folders api and auth I created 2 files. Respectively api.yaml and auth.yaml as I wanted to deploy them as 2 separate services. They look pretty much the same. This is one of them:

runtime: php
env: flex
api_version: 1
service: auth

runtime_config:
  document_root: auth/web

beta_settings:
  cloud_sql_instances: "[connectionName]"

env_variables:
  # framework
  YII_DEBUG: false
  YII_ENV: prod
  #db
  POSTGRES_DSN: pgsql:dbname=alpha;host=/cloudsql/[connectionName]
  POSTGRES_USER: [user]
  POSTGRES_PASSWORD: [your-pass]
  POSTGRES_PORT: 5432

# This sample incurs costs to run on the App Engine flexible environment.
# The settings below are to reduce costs during testing and are not appropriate
# for production use. For more information, see:
# https://cloud.google.com/appengine/docs/flexible/python/configuring-your-app-with-app-yaml
manual_scaling:
  instances: 1
resources:
  cpu: 1
  memory_gb: 0.5
  disk_size_gb: 10

handlers:
- url: /.*
  script: index.php

NOTE: I don't think that is the proper way to create microservices. You may better have an app.yaml file with common scaling parameters and a dispatch.yaml file linking the other services. Read more about it here:

• Microservices Architecture on Google App Engine
• Configuration Files

4. Use their variables where needed inside your code. In the advanced template it would be inside environments/prod/common/main-local.php to be used after executing the init bat:

'db' => [
    'class' => 'yii\db\Connection',
    'dsn' => getenv('POSTGRES_DSN'),
    'username' => getenv('POSTGRES_USER'),
    'password' => getenv('POSTGRES_PASSWORD'),
    'charset' => 'utf8',
    'enableSchemaCache' => true,
    'schemaCacheDuration' => 3600,
    'schemaCache' => 'cache',
],

You can also change the entry script to this in case you think you may need to enable debug mode in their server:

defined('YII_DEBUG') or define('YII_DEBUG', getenv('YII_DEBUG') === 'true');
defined('YII_ENV') or define('YII_ENV', getenv('YII_ENV') ?: 'prod');

5. I created an nginx-app.conf file (same level with yaml files: app root) with the following content:

location / {
  try_files $uri $uri/ /index.php$is_args$args;
}

Which I guess should be auto merged with their nginx configs that you can see here: https://github.com/GoogleCloudPlatform/php-docker/blob/4454562b9be8134d630feded98a5bdb206e686ab/php-base/nginx.conf#L106

6. See a list of enabled/disabled php extensions here: https://cloud.google.com/appengine/docs/flexible/php/runtime. The ones you need to activate add them to the require section of your package.json file:

  "require": {
    "php": "7.2.*",
    "ext-gd": "*",
    "ext-redis": "*",
    "ext-intl": "*",
    ...
  },

NOTE: That documentation says you can enable them by adding a php.inifile. Don't do that. It doesn't work with Flex environment. (see https://github.com/GoogleCloudPlatform/php-docs-samples/issues/446) use the composer file instead.

7. Use scripts in package.json similar to how it was done in Laravel and Symfony tutorials here and here to add Yii related scripts under any event you need like flashing db or like in my first deploy which was:

"post-install-cmd": [
  "php init --env=Production --overwrite=All",
  "php yii migrate/up --interactive=0"
],

8. Respectively (in my case):

gcloud app deploy api.yaml
gcloud app deploy auth.yaml

There is many options you can also add like manually set version --version=alpha0or --verbosity=info... It will take long the first time and the script will give you the final url. You can also SSH to your instance within GAE interface and do stuff like docker exec -it gaeapp /bin/bash to see your code. I had to go there the first time to manually run the init script as I didn't know about composer post-install-cmd.

[tunecino]

There is also a list of disabled functions here:

exec
passthru
proc_open
proc_close
shell_exec
show_source
symlink
system

These needed could be whitelisted in the Yaml file within a comma separated string. I had to add this to make the yii2-queue extension work by runnig ./yii queue/listen:

runtime_config:
  document_root: api/web
  whitelist_functions: proc_open <--