Skip to content

Util: Add PostgreSQL data migration script#1520

Draft
drikusroor wants to merge 1 commit intodevelopfrom
db/migration-script
Draft

Util: Add PostgreSQL data migration script#1520
drikusroor wants to merge 1 commit intodevelopfrom
db/migration-script

Conversation

@drikusroor
Copy link
Copy Markdown
Contributor

@drikusroor drikusroor commented Feb 12, 2025

Introduce a script to facilitate the migration of PostgreSQL data between versions, automating the process of dumping and restoring databases using Docker containers.

  • Shut Down Containers: The script starts by bringing down all running containers and stopping the main database container.
  • Capture Logs: It runs the database container to capture error logs that contain the PostgreSQL version information.
  • Extract Versions: Using regex on the logs, it extracts the old and new PostgreSQL version numbers.
  • Dump Data: It starts a temporary container running the old PostgreSQL version, waits for it to be ready, and then dumps the database using pg_dumpall.
  • Clean Up: The dump file is verified, and the temporary container is stopped and removed along with the old data volume.
  • Restore Data: A new temporary container is started using the new PostgreSQL version. The dump file is copied into this container, and the data is restored using psql.
  • Restart Project: Finally, the script restarts the project using the updated configuration.

# ----------------------------------------------------------------------------
# Step 4: Wait until PostgreSQL inside the container is ready.
#
echo "Waiting for PostgreSQL to be ready..."
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script hangs here, with this error in the container in docker desktop:

2025-02-12 13:59:18 Error: Database is uninitialized and superuser password is not specified.
2025-02-12 13:59:18        You must specify POSTGRES_PASSWORD to a non-empty value for the
2025-02-12 13:59:18        superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
2025-02-12 13:59:18 
2025-02-12 13:59:18        You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
2025-02-12 13:59:18        connections without a password. This is *not* recommended.
2025-02-12 13:59:18 
2025-02-12 13:59:18        See PostgreSQL documentation about "trust":
2025-02-12 13:59:18        https://www.postgresql.org/docs/current/auth-trust.html

Copy link
Copy Markdown
Collaborator

@BeritJanssen BeritJanssen Feb 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran into something similar when doing this manually. I think what happens is that the env variables aren't loaded in this way. I solved it by running docker compose up db and then running the script afterwards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants