deployment-docker-compose.md

author: Martin Weise
Docker Compose
TL;DR
If you have Docker already installed on your system, you can install DBRepo with:
curl -sSL https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/raw/dev/install.sh | bash
Requirements
Hardware
For this small, local, test deployment any modern hardware would suffice, we recommend a dedicated virtual machine with the following settings.
!!! tip "Resource Consumption"
Note that most of the vCPU and RAM resources will be needed for starting the infrastructure, this is because of
Docker. During operation and especially idle times, the deployment will use significantly less resources.
- 8 vCPU cores
- 16GB RAM memory
- 100GB SSD storage
Software
We only test the Docker Compose deployment with the official Docker engine installed on a Debian-based operating system. Other software deployments (e.g. Docker Desktop on Windows) are not recommended and not tested.
Architecture
Overview
The repository is designed as a service-based architecture to ensure scalability and the utilization of various technologies. The conceptualized microservices operate the basic database operations, data versioning as well as findability, accessability, interoperability and reuseability (FAIR).
Notes
Please note that we only save the state of the databases as well as the Broker Service since RabbitMQ maintains state inside the container.
Deployment
We maintain a rapid prototype deployment option through Docker Compose (v2.17.0 and newer). This deployment creates the core infrastructure and a single Docker container for all user-generated databases.
curl -sSL https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/raw/dev/install.sh | sudo bash
View the logs:
docker compose logs -f
You should now be able to view the front end at http://localhost:80.
Please be warned that the default configuration is not intended for public deployments. It is only intended to have a
running system within minutes to play around within the system and explore features. It is strongly advised to change
the default .env
environment variables.
!!! warning "Known security issues with the default configuration"
The system is auto-configured for a small, local, test deployment and is *not* secure! You need to make modifications
in various places to make it secure:
* **Authentication Service**:
a. You need to use your own instance or configure a secure instance using a (self-signed) certificate.
Additionally, when serving from a non-default Authentication Service, you need to put it into the
`JWT_ISSUER` environment variable (`.env`).
b. You need to change the default admin user `fda` password in Realm
master > Users > fda > Credentials > Reset password.
c. You need to change the client secrets for the clients `dbrepo-client` and `broker-client`. Do this in Realm
dbrepo > Clients > dbrepo-client > Credentials > Client secret > Regenerate. Do the same for the
broker-client.
d. You need to regenerate the public key of the `RS256` algorithm which is shared with all services to verify
the signature of JWT tokens. Add your securely generated private key in Realm
dbrepo > Realm settings > Keys > Providers > Add provider > rsa.
* **Broker Service**: by default, this service is configured with an administrative user that has major privileges.
You need to change the password of the user *fda* in Admin > Update this user > Password. We found this
[simple guide](https://onlinehelp.coveo.com/en/ces/7.0/administrator/changing_the_rabbitmq_administrator_password.htm)
to be very useful.
* **Search Database**: by default, this service is configured to require authentication with an administrative user
that is allowed to write into the indizes. Following
this [simple guide](https://www.elastic.co/guide/en/elasticsearch/reference/8.7/reset-password.html), this can be
achieved using the command line.
* **Gateway Service**: by default, no HTTPS is used that protects the services behind. You need to provide a trusted
SSL/TLS certificate in the configuration file or use your own proxy in front of the Gateway Service. See this
[simple guide](http://nginx.org/en/docs/http/configuring_https_servers.html) on how to install a SSL/TLS
certificate on NGINX.
Upgrade Guide
1.2 to 1.3
In case you have a previous deployment from version 1.2, shut down the containers and back them up manually. You can do
this by using the busybox
image. Replace deadbeef
with your container name or hash:
export NAME=dbrepo-userdb-xyz
docker run --rm --volumes-from $NAME -v /home/$USER/backup:/backup busybox tar pcvfz /backup/$NAME.tar.gz /var/lib/mysql
!!! danger "Wipe all traces of DBRepo from your system"
To erase all traces of DBRepo from your computer or virtual machine, the following commands delete all containers,
volumes and networks that are present, execute the following **dangerous** command. It will **wipe** all information
about DBRepo from your system (excluding the images).
```console
docker container stop $(docker container ls -aq -f name=^/dbrepo-.*) || true
docker container rm $(docker container ls -aq -f name=^/dbrepo-.*) || true
docker volume rm $(docker volume ls -q -f name=^dbrepo-.*) || true
docker network rm $(docker network ls -q -f name=^dbrepo-.*) || true
```
You can restore the volume after downloading the new 1.3 images and creating the infrastructure:
export NAME=dbrepo-userdb-xyz
export PORT=12345
docker container create -h $NAME --name $NAME -p $PORT:3306 -e MARIADB_ROOT_PASSWORD=mariadb --network userdb -v /backup mariadb:10.5
docker run --rm --volumes-from $NAME -v /home/$USER/backup/.tar.gz:/backup/$NAME.tar.gz busybox sh -c 'cd /backup && tar xvfz /backup/$NAME.tar.gz && cp -r /backup/var/lib/mysql/* /var/lib/mysql'
Future releases will be backwards compatible and will come with migration scripts.
Limitations
!!! info "Alternative Deployments"
Alternatively, you can also deploy DBRepo with [Helm](../deployment-helm/) in your virtual machine instead.