Migrating from an all-in-one to a docker compose installation

[awa5114]

I have been using local instance of OpenProject on my windows machine which I installed using the all-in-one configuration. The Dockerfile for the image I'm using contains the following:

FROM openproject/openproject:15

ENV OPENPROJECT_SECRET_KEY_BASE=resing123
ENV OPENPROJECT_HOST__NAME=localhost:8080
ENV OPENPROJECT_HTTPS=false
ENV OPENPROJECT_DEFAULT__LANGUAGE=en

I then build and run as follows:

PS > docker build .
PS> docker run -it -p 8080:80 -v $TEMP/openproject/pgdata:/var/openproject/pgdata -v $TEMP/openproject/assets:/var/openproject/assets openproject-all-in-one:latest

After using this open project instance for a while, I am now attempting to migrate to the Docker Compose installation. I cloned the repo and copied the .env.example file as indicated by the docs.

PS> git clone https://github.com/opf/openproject-deploy --depth=1 --branch=stable/15 openproject
PS> cp .env.example .env

I then tried tweaking the .env file to mimic the environment I had with the all-in-one installation, including mapping the volumes to the same directories where the all-in-one installation was storing data.

TAG=15-slim
OPENPROJECT_HTTPS=false
OPENPROJECT_HOST__NAME=localhost
PORT=127.0.0.1:8080
IMAP_ENABLED=false
DATABASE_URL="postgres://openproject:openproject@127.0.0.1/openproject"
OPENPROJECT_SECRET_KEY_BASE=resing123
RAILS_MIN_THREADS=4
RAILS_MAX_THREADS=16
PGDATA="C:/Users/admin/Desktop/TEMP/openproject/pgdata"
OPDATA="C:/Users/admin/Desktop/TEMP/openproject/assets"
POSTGRES_PASSWORD=""
# All-in-one connection string:\
# postgres://openproject:openproject@127.0.0.1/openproject
DATABASE_URL="postgres://openproject:openproject@db/openproject?pool=20&encoding=unicode&reconnect=true"

Unfortunately, when I try to docker compose up the above, I observe the following behaviors:

• the seeder service keeps on exiting with exit code 2 and restarting over and over again
• The web-1 service throws errors that look like this:

 | /app/vendor/bundle/ruby/3.3.0/gems/pg-1.5.9/lib/pg/connection.rb:709:in `async_connect_or_reset': connection to server at "172.18.0.4", port 5432 failed: Connection refused (PG::ConnectionBad)

• The worker-1 service throws errors that look like this:

| /app/vendor/bundle/ruby/3.3.0/gems/activerecord-7.1.5.1/lib/active_record/connection_adapters/postgresql_adapter.rb:80:in `rescue in new_client': connection to server at "172.18.0.4", port 5432 failed: Connection refused (ActiveRecord::ConnectionNotEstablished)
|   Is the server running on that host and accepting TCP/IP connections?

Being a bit of a noob when it comes to Docker, I would appreciate some help or pointers on what I may be doing wrong. Thanks in advance.

[awa5114]

The documentation also states the following regarding configuring a custom database:

If you run the Compose based docker stack, you can simply override the DATABASE_URL environment variable, and remove the db service from the docker-compose.yml file

But if I remove the db service, then the pgdata volume goes along with it, and the app wouldn't be connected to the data on my hard drive which I was previously mounting to the all-in-one container.

Furthermore, how do I know which DATABASE_URL to use? If I run docker exec container_name printenv DATABASE_URL in the all-in-one container I get the following output:

postgres://openproject:openproject@127.0.0.1/openproject

So should I just copy and paste that or are there other considerations when setting the DATABASE_URL using docker compose?

[awa5114]

I did some additional testing and here are some findings:

• The database that the all-in-one installation consumes is located in C:/Users/admin/Desktop/TEMP/openproject/pgdata, its default credentials are as follows:
  • Username: openproject
  • Password: openproject
  • Database name openproject
• psql urls have the following pattern: postgres://user:pass@host:<port>/dbname, therefore the valid connection string for the above database ought to be: postgres://openproject:openproject@db/openproject. Because the docker-compose.yml file connects the db service to the backend network, other services that consume it can refer to it by name (and not necessarily by IP address)
• A key error is the one thrown by the worker service:

worker-1    | bundler: failed to load command: good_job (/app/vendor/bundle/ruby/3.3.0/bin/good_job)
worker-1    | /app/vendor/bundle/ruby/3.3.0/gems/activerecord-7.1.5.1/lib/active_record/connection_adapters/postgresql_adapter.rb:80:in `rescue in new_client': connection to server at "172.18.0.3", port 5432 failed: Connection refused (ActiveRecord::ConnectionNotEstablished)
worker-1    |   Is the server running on that host and accepting TCP/IP connections?

[tiroessler]

Hello @amine-aboufirass

It is not intended to switch from the all-in-one container to the docker-compose by just mounting the old volumes.
The all-in-one container has different database settings than the docker-compose db container, so you would need to change all of this.
Also, both container run with different UIDs, so there might also be an issue with different ownerships that would need to be fixed.

The recommended way to switch from all-in-one to docker-compose would be creating a backup of your installation and then restoring the backup in a fresh docker-compose installation.

For that you can do the following steps.
First, you need to create a backup of attachments and your database.
You can do this with the following commands:

docker exec -it $OP_CONTAINER_NAME su - postgres -c 'pg_dump -d openproject -x -O' > openproject.sql
docker exec $OP_CONTAINER_NAME tar cz -O -C /var/openproject/assets . > assets.tar.gz

When you have your backups you can restore them on a fresh docker compose installation.
For that you first stop all container and only start the db container:

docker compose stop
docker compose up -d db

Now you can copy the database backup to your db container:

docker compose cp openproject.sql db:/tmp

Now you can open a database console:

docker compose exec -it -u postgres db psql

Now you need to delete the old database and create a new one:

DROP DATABASE openproject; CREATE DATABASE openproject OWNER postgres;

After that you can restore your database with the following commands:

\c openproject
\i /tmp/openproject.sql

Now you can start all docker container:

docker compose up -d 

Now you can restore the asses by using the following commands:

docker compose cp  assets.tar.gz  web:/tmp
docker compose exec -it -u root web tar -C /var/openproject/assets -xf /tmp/assets.tar.gz
docker compose exec -it -u root web chown 1000:1000 -R /var/openproject/assets

Now you should be able to use docker-compose with your data restored

[felixgonschorek]

Thank you for your detailed guide, @tiroessler, this work for me and helped a lot! <3