Gangmax Blog

pyenv-virtualenv and Airflow

This post records how to use “pyenv-virtualenv” and “Airflow” together.

Install “pyenv-virtualenv”

From here.

Install the “pyenv-virtualenv” plugin

1
git clone https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv

Update the “~/.zshenv” file

Update the file by adding the following content(this step is optional):

1
2
# https://github.com/pyenv/pyenv-virtualenv
echo 'eval "$(pyenv virtualenv-init -)"'

Update the “~/.zshrc” file

Update the file by adding the following content:

1
2
132 eval "$(pyenv init -)"
133 export PYENV_VIRTUALENV_DISABLE_PROMPT=1

Now we can use “pyenv-virtualenv” to install “airflow”.

Airflow

From here.

Create an virtual environment for “airflow”

1
2
3
# Note that install airflow with Python 3.8.0 will get error, that's why
# I use "Python 3.7.4" instead.
pyenv virtualenv 3.7.4 airflow

Update the “~/.zshrc” file

Update the file by adding content like below:

1
2
# Add Apache Airflow.
export AIRFLOW_HOME=~/.airflow

Install “Airflow” and start the local Airflow instance

1
2
3
4
5
6
7
8
9
pyenv activate airflow
pip install apache-airflow
# Note that running the following command under Python 3.8.0 will
# get error. So in my case I use "Python 3.7.4".
airflow initdb
# Start web server.
airflow webserver -p 8080
# Start scheduler.
airflow scheduler

Comments