alembic.util.exc.CommandError: Can't locate revision identified by '405de8318b3a'

[manel00]

Hello

I am trying to deploy a docker-compose with an airflow worker, it points to the airflow host docker-compose, but I am retrieving the error from the title... here is the traceback:

airflow-worker_x_1  | Traceback (most recent call last):
airflow-worker_x_1  |   File "/home/airflow/.local/bin/airflow", line 8, in <module>
airflow-worker_x_1  |     sys.exit(main())
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/airflow/__main__.py", line 38, in main
airflow-worker_x_1  |     args.func(args)
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/cli_parser.py", line 51, in command
airflow-worker_x_1  |     return func(*args, **kwargs)
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/cli.py", line 99, in wrapper
airflow-worker_x_1  |     return f(*args, **kwargs)
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/commands/db_command.py", line 82, in upgradedb
airflow-worker_x_1  |     db.upgradedb(to_revision=to_revision, from_revision=from_revision, show_sql_only=args.show_sql_only)
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/session.py", line 71, in wrapper
airflow-worker_x_1  |     return func(*args, session=session, **kwargs)
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/db.py", line 1449, in upgradedb
airflow-worker_x_1  |     command.upgrade(config, revision=to_revision or 'heads')
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/alembic/command.py", line 322, in upgrade
airflow-worker_x_1  |     script.run_env()
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/alembic/script/base.py", line 569, in run_env
airflow-worker_x_1  |     util.load_python_file(self.dir, "env.py")
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file
airflow-worker_x_1  |     module = load_module_py(module_id, path)
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py
airflow-worker_x_1  |     spec.loader.exec_module(module)  # type: ignore
airflow-worker_x_1  |   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
airflow-worker_x_1  |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/airflow/migrations/env.py", line 107, in <module>
airflow-worker_x_1  |     run_migrations_online()
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/airflow/migrations/env.py", line 101, in run_migrations_online
airflow-worker_x_1  |     context.run_migrations()
airflow-worker_x_1  |   File "<string>", line 8, in run_migrations
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/alembic/runtime/environment.py", line 853, in run_migrations
airflow-worker_x_1  |     self.get_context().run_migrations(**kw)
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/alembic/runtime/migration.py", line 611, in run_migrations
airflow-worker_x_1  |     for step in self._migrations_fn(heads, self):
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/alembic/command.py", line 311, in upgrade
airflow-worker_x_1  |     return script._upgrade_revs(revision, rev)
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/alembic/script/base.py", line 431, in _upgrade_revs
airflow-worker_x_1  |     with self._catch_revision_errors(
airflow-worker_x_1  |   File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
airflow-worker_x_1  |     self.gen.throw(typ, value, traceback)
airflow-worker_x_1  |   File "/home/airflow/.local/lib/python3.10/site-packages/alembic/script/base.py", line 265, in _catch_revision_errors
airflow-worker_x_1  |     raise util.CommandError(resolution) from re
airflow-worker_x_1  | alembic.util.exc.CommandError: Can't locate revision identified by '405de8318b3a'

docker-compose.yml:

version: '3'
services:
airflow-worker:
image: apache/airflow:latest
environment:
- AIRFLOW__CORE__EXECUTOR=CeleryExecutor
- AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql+psycopg2://airflow:yourpassword@<postgres_server_ip>:5432/airflow
- AIRFLOW__CELERY__RESULT_BACKEND=db+postgresql://airflow:yourpassword@<postgres_server_ip>:5432/airflow
- AIRFLOW__CELERY__BROKER_URL=redis://<redis_server_ip>:6379/0
command: airflow celery worker

[potiuk]

Yes. You have a database created (migrated to) with later version of airflow than the one you run. See https://airflow.apache.org/docs/apache-airflow/stable/migrations-ref.html - you should use the newer version to downgrade your database or drop your DB completely and recreate it from scratch. For more details read about airflow db migrate command and alembic that is used to manage those versions. Generally speaking you should understand the steps involved in the database migration steps and possibly modify your docker-compose accordingly or manually run airflow db commands if you are playing around with different versions of airflow.

[jimmcslim]

I have observed this error with what is apparently a completely clean environment, running airflow db init. Any suggestions?

[potiuk]

No. The error is that version to start with is wrong. You must have some database with wrong revision id in it.

[jimmcslim]

Hmmm. I have what I believe is a completely clean environment. i.e. no .airflow directory in the current directory, and starting with a fresh venv. I had even done a clean reinstall of the corresponding version of Python with pyenv. FWIW this is on an Apple Silicon Mac. Interestingly, following these same paths on a GitHub Codespace (i.e. x86) didn't result in the same issue.

[potiuk]

It's not venv it's database. You likely have SQLite database in AIRFLOW_HOME (or seeing you are using compose) - postgres or MySQL

[bouli]

I tried to start the airflow quickstart and I had the same problem.

I tried airflow before, so I had some "left overs" from my past "hello worlds".

That probably happens because you are using the same home folder for your new airflow instance of a past project.

Solution 1: I'm not sure this is the best solution, but since I don't have any important data or dags in my local development computer, I just cleaned (deleted everything) in my $AIRFLOW_HOME (~/airflow), and everything is finally working fine. Just be sure you don't delete anything you shouldn't.

Solution 2: Or, the best way in my opinion, just change the home folder:

export AIRFLOW_HOME=$(pwd)