Usage of Astronomer Enterprise Edition requires an Astronomer Enterprise Subscription: Start a free 14-day trial.

Architecture

We’ve build out pre-configured Docker containers w/ Celery and metrics/monitoring. The Astronomer Airflow module consists of seven components, and you must bring your own Postgres and Redis database.

Airflow Module

This kit is fully open-sourced (Apache 2.0) and you can experiment with it at https://open.astronomer.io/.

Astronomer CLI

The Astronomer CLI is under very active development to support Airflow-related commands.

Airflow CLI

We also make it easy to use the Airflow CLI remotely (i.e. run commands from your local terminal that execute in the cloud Airflow).

DAG Deployment

Astronomer Enterprise makes it easy to deploy these containers to Kubernetes - but more importantly, to give Airflow developers a CLI to deploy DAGs through a private Docker registry that interacts with the Kubernetes API.

Components that glue this all together include Phoenix and Commander.

Airflow Deployment

Remember to run astro airflow init after creating a new project directory.

Any Python packages can be added to requirements.txt and all OS level packages can be added to packages.txt in the project directory.

Additional RUN commands can be added to the Dockerfile. Environment variables can also be added to ENV.

Houston GraphQL API

The Houston GraphQL API is the source of truth for the Astronomer Platform. Playground.

Queries:

  • Clusters - Fetches one or more deployments based on input
  • ClusterConfig - Fetches config needed to create a module deployment

Mutations:

  • createToken - Verify a user’s credentials and issues a token if valid
  • createUser - Creates a new user
  • updateUser - Update an existing user
  • createCluster - Creates a new Airflow Cluster
  • updateCluster - Updates an Airflow cluster
  • deleteCluster - Deletes an Airflow Cluster