airflow 1.10+ broken with cloudsql(mysql) explicit_defaults_for_timestamp=off

[barney-s]

Since 1.10.0, Airflow requires MySQL to have explicit_defaults_for_timestamp flag set to ON. CloudSQL can only support MySQL with explicit_defaults_for_timestamp=OFF for now.

[barney-s]

Seeing this error:

2019-01-28 03:50:50,394] {{__init__.py:51}} INFO - Using executor CeleryExecutor
DB: mysql://v9dbimx6cr3mbgzw:***@cc-base-sql:3306/e9qcv7p27gi36b5m
[2019-01-28 03:50:50,676] {{db.py:338}} INFO - Creating tables
INFO  [alembic.runtime.migration] Context impl MySQLImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 0e2a74e0fc9f, Add time zone awareness
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 32, in <module>
    args.func(args)
  File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 1011, in initdb
    db_utils.initdb(settings.RBAC)
  File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 92, in initdb
    upgradedb()
  File "/usr/local/lib/python3.6/site-packages/airflow/utils/db.py", line 346, in upgradedb
    command.upgrade(config, 'heads')
  File "/usr/local/lib/python3.6/site-packages/alembic/command.py", line 174, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.6/site-packages/alembic/script/base.py", line 416, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/local/lib/python3.6/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.6/site-packages/alembic/util/compat.py", line 68, in load_module_py
    module_id, path).load_module(module_id)
  File "<frozen importlib._bootstrap_external>", line 399, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 823, in load_module
  File "<frozen importlib._bootstrap_external>", line 682, in load_module
  File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.6/site-packages/airflow/migrations/env.py", line 91, in <module>
    run_migrations_online()
  File "/usr/local/lib/python3.6/site-packages/airflow/migrations/env.py", line 86, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/local/lib/python3.6/site-packages/alembic/runtime/environment.py", line 807, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/local/lib/python3.6/site-packages/alembic/runtime/migration.py", line 321, in run_migrations
    step.migration_fn(**kw)
  File "/usr/local/lib/python3.6/site-packages/airflow/migrations/versions/0e2a74e0fc9f_add_time_zone_awareness.py", line 46, in upgrade
    raise Exception("Global variable explicit_defaults_for_timestamp needs to be on (1) for mysql")
Exception: Global variable explicit_defaults_for_timestamp needs to be on (1) for mysql
[2019-01-28 03:50:51,465] {{settings.py:174}} INFO - setting.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800
[2019-01-28 03:50:51,948] {{__init__.py:51}} INFO - Using executor CeleryExecutor

[barney-s]

Update: Google Composer patched airflow to work around this issue. Will wait for that to be open-sourced in airflow.