airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Bring back Documentation publishing in Canary runs

Open potiuk opened this issue 1 year ago • 1 comments

Currently, documentation publishing in Canary runs is disabled. This caused several problems (that will get worse over time):

  • we do not test publishing worklfow - and it might stop working any time and break release process
  • the "sphinx inventory" is not published for airlfow and providers which means that docs building will take more time and will lead to potential errors

https://github.com/apache/airflow/blob/7cf54a734e1eefa04ab710cb2eb364d529c6b1b1/.github/workflows/static-checks-mypy-docs.yml#L246

The reason is that public runners default "workspace" disk is too small to fit checked out documentation while being published - and "self-hosted ASF runners" were unstable even if they had bigger disk.

There are few ways we can address it:

  • apply the same workaround that @kaxil did for "airflow-site" building : https://github.com/apache/airflow-site/pull/1055
  • switch back to "ASF self-hosted runners" https://github.com/apache/airflow/actions/runners?tab=self-hosted
  • switch to large or medium "ARC self-hosted runners" after https://github.com/apache/airflow/pull/41728 is completed

It's been disabled in https://github.com/apache/airflow/pull/40921

potiuk avatar Sep 01 '24 19:09 potiuk

@kaxil - shall you apply the same fix to the workflow as you did in airlfow-site (trick with mounting another volume) - so that auto-publishing of docs and inventory works ?

potiuk avatar Oct 18 '24 06:10 potiuk