We’ve build out pre-configured Docker containers w/ Celery and metrics/monitoring. The Astronomer Airflow module consists of seven components, and you must bring your own Postgres and Redis database.
This kit is fully open-sourced (Apache 2.0) and you can experiment with it at https://open.astronomer.io/.
We also make it easy to use the Airflow CLI remotely (i.e. run commands from your local terminal that execute in the cloud Airflow).
Astronomer Enterprise makes it easy to deploy these containers to Kubernetes - but more importantly, to give Airflow developers a CLI to deploy DAGs through a private Docker registry that interacts with the Kubernetes API.
Remember to run
astro airflow init after creating a new project directory.
Any Python packages can be added to
requirements.txt and all OS level packages can be added to
packages.txt in the project directory.
Houston GraphQL API
- Clusters - Fetches one or more deployments based on input
- ClusterConfig - Fetches config needed to create a module deployment
- createToken - Verify a user’s credentials and issues a token if valid
- createUser - Creates a new user
- updateUser - Update an existing user
- createCluster - Creates a new Airflow Cluster
- updateCluster - Updates an Airflow cluster
- deleteCluster - Deletes an Airflow Cluster