astro-sdk icon indicating copy to clipboard operation
astro-sdk copied to clipboard

Maintain a .astro-registry.yaml file

Open virajmparekh opened this issue 1 year ago • 4 comments

As part ensuring that the SDK page in the Registry is up to date, it would be awesome to have a .astro-registry.yaml file. This is similar to the .provider.yaml file found in OSS providers. This file would make it a ton easier to parse the SDK and ensure all necessary information makes its way into the database (and ultimately, the front end of the Registry).

As of today, this file would look like:

# This file is intended to be a file which specifically calls out modules to be published to the Astronomer
# Registry. The file should not be packaged and solely lives in the repo. The modules listed should pertain
# to latest released version of the repo.

# Python package name (if it's not the name of the repo itself).
package-name: astro-sdk-python
# The name of the provider to be displayed on the Registry.
display-name: Astro SDK
# A description to be displayed on the main Provider page.
description: Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

# The next sections should be organized by module type (e.g. operators, hooks, functions, decorators, etc.).
#
# Within each section, the ``module`` value should be the path.to.the.module.specifically.
# For example, if the import path for a function is `from astro.sql.operators.append import append` then the
# value should be "astro.sql.operators.append.append".
#
# There is an option to include an override for the import path that is displayed on the Registry wrt to the
# Module page itself: ``display_path``. This would be the import path Access Instructions
# (see https://registry.astronomer.io/providers/astro-sdk/modules/append).

# These would be the Functions to be displayed on the Registry.
# (akin to https://registry.astronomer.io/modules?providers=Astro+Sdk&page=1&types=Functions)
functions:
  - module: astro.sql.operators.append.append
    display-path: astro.sql.append
  - module: astro.sql.operators.cleanup.cleanup
    display-path: astro.sql.cleanup
  - module: astro.sql.operators.drop.drop_table
    display-path: astro.sql.drop_table
  - module: astro.sql.operators.export_file.export_file
    display-path: astro.sql.export_file
  - module: astro.sql.operators.load_file.load_file
    display-path: astro.sql.load_file
  - module: astro.sql.operators.merge.merge
    display-path: astro.sql.merge
  - module: astro.sql.operators.transform.transform_file
    display-path: astro.sql.transform_file

# These are the Decorators to be displayed on the Registry.
# (akin to https://registry.astronomer.io/modules?providers=Astro+Sdk&page=1&types=Decorators)
decorators:
  - module: astro.sql.operators.dataframe.dataframe
    display-path: astro.sql.dataframe
  - module: astro.sql.operators.raw_sql.run_raw_sql
    display-path: astro.sql.run_raw_sql
  - module: astro.sql.operators.transform.transform
    display-path: astro.sql.transform

virajmparekh avatar Sep 06 '22 22:09 virajmparekh

Awesome will take care of it @virajmparekh

kaxil avatar Sep 08 '22 12:09 kaxil

@sunank200 to connect with @kaxil to understand more details about this ticket

phanikumv avatar Oct 10 '22 11:10 phanikumv

@kaxil How about moving this to 1.3.0 ?

phanikumv avatar Oct 10 '22 11:10 phanikumv

@kaxil How about moving this to 1.3.0 ?

Yup, fine for me

kaxil avatar Oct 10 '22 19:10 kaxil