diff --git a/.docs/deployment-docker-compose.md b/.docs/deployment-docker-compose.md index 4e78dad081fe0cd6fdf065afbdd1408c97b719a4..761feccfc17587db9d26517a6cf7a689a90a2b47 100644 --- a/.docs/deployment-docker-compose.md +++ b/.docs/deployment-docker-compose.md @@ -61,7 +61,7 @@ since RabbitMQ maintains state inside the container. We maintain a rapid prototype deployment option through Docker Compose (v2.17.0 and newer). This deployment creates the core infrastructure and a single Docker container for all user-generated databases. -=== ":simple-linux: Linux" +=== "Linux" Download and install [Docker Engine](https://docs.docker.com/desktop/install/linux-install/) for your Linux distribution. Although the installation might work, we *do not* recommend Docker Desktop. @@ -74,7 +74,7 @@ core infrastructure and a single Docker container for all user-generated databas curl -sSL https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/raw/dev/install.sh | bash -=== ":simple-windows: Windows" +=== "Windows" Open `cmd.exe` as administrator and install WSL2 and the Debian subsystem: @@ -126,6 +126,14 @@ In case the deployment is unsuccessful, we have explanations on their origin and : *Solution*: Update your local Docker image cache by executing `docker compose pull`, it automatically downloads all Docker images that have updates. Then apply the new images with `docker compose up -d`. +**Error response from daemon: Error starting userland proxy: listen tcp4 0.0.0.0:xyz: bind: address already in use** + +: *Origin*: Your deployment machine (e.g. laptop, virtual machine) has the port `xyz` already assigned. Some service + or application is already listening to this port. +: *Solution*: This service or application needs to be stopped. You can find out the service or application via + `sudo netstat -tulpn` (sudo is necessary for the process id) and then stop the service or application + gracefully or force a stop via `kill -15 PID` (not recommended). + ## Security !!! warning "Known security issues with the default configuration" diff --git a/.docs/deployment-helm.md b/.docs/deployment-helm.md index eb4c146606610c3fa75e281505a54afac14c2008..d87f53e453ead005f8fce802de66c15c98502f0e 100644 --- a/.docs/deployment-helm.md +++ b/.docs/deployment-helm.md @@ -4,19 +4,21 @@ author: Martin Weise ## TL;DR -To install DBRepo in your existing cluster, download the sample [`values.yaml`](https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-deployment/-/raw/dev/charts/dbrepo-core/values.yaml?inline=false) +To install DBRepo in your existing cluster, download the sample [`values.yaml`](https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-deployment/-/raw/master/charts/dbrepo-core/values.yaml?inline=false) for your deployment and update the variables, especially `hostname`. ```shell helm upgrade --install dbrepo \ -n dbrepo \ - "oci://dbrepo.azurecr.io/helm/dbrepo-core" \ + "oci://s210.dl.hpc.tuwien.ac.at/dbrepo/helm/dbrepo-core" \ --values ./values.yaml \ - --version "0.1.4" \ + --version "1.4.0" \ --create-namespace \ --cleanup-on-fail ``` +This chart is also on [Artifact Hub](https://artifacthub.io/packages/helm/dbrepo/dbrepo-core). + ## Dependencies Our chart depends on seven other charts which will be automatically resolved when installing our `dbrepo-core` chart: diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5449795b63878b147639efb194a572a548f9fb2a..fa879b20a73b3a403168b95f28369cccbfb861a5 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -447,7 +447,7 @@ release-latest: - echo "$CI_REGISTRY2_PASSWORD" | docker login --username "$CI_REGISTRY2_USER" --password-stdin $CI_REGISTRY2_URL script: - "ifconfig eth0 mtu 1450 up" - - "apk add make" + - "apk add make bash" - TAG=latest make release release-1.3: @@ -466,7 +466,7 @@ release-1.3: - echo "$CI_REGISTRY2_PASSWORD" | docker login --username "$CI_REGISTRY2_USER" --password-stdin $CI_REGISTRY2_URL script: - "ifconfig eth0 mtu 1450 up" - - "apk add make" + - "apk add make bash" - "TAG=1.3.0 make release" release-1.4: @@ -485,7 +485,7 @@ release-1.4: - echo "$CI_REGISTRY2_PASSWORD" | docker login --username "$CI_REGISTRY2_USER" --password-stdin $CI_REGISTRY2_URL script: - "ifconfig eth0 mtu 1450 up" - - "apk add make" + - "apk add make bash" - "TAG=${VERSION} make release" build-api-latest: diff --git a/.gitlab/logo.png b/.gitlab/logo.png new file mode 100644 index 0000000000000000000000000000000000000000..d34412dab6e6be0ae66e8c9f35f42c0a7bcf437e Binary files /dev/null and b/.gitlab/logo.png differ diff --git a/Makefile b/Makefile index 5f39f77cc78a3c4fd1c84119d3c474cf65adf61d..79f8d5e1041aee04e0b0af36fe1097030cc62f07 100644 --- a/Makefile +++ b/Makefile @@ -28,9 +28,7 @@ build-analyse-service: bash ./dbrepo-analyse-service/build.sh build-docker: - docker build --network=host -t dbrepo-metadata-service:build --target build dbrepo-metadata-service - docker build --network=host -t dbrepo-data-service:build --target build dbrepo-data-service - docker compose build --parallel + bash ./bin/build-docker.sh build-frontend: yarn --cwd ./dbrepo-ui install --legacy-peer-deps @@ -220,7 +218,7 @@ test-clients: test: test-backend test-frontend teardown: - ./.scripts/teardown.sh + ./bin/teardown.sh docs: build-docker docker compose up -d || docker compose down diff --git a/README.md b/README.md index 4b38972edd9b06e8acf2e9d2aff3157c68caa8de..92d200dc1520c6007edfff1bb4e8af75f230c654 100644 --- a/README.md +++ b/README.md @@ -2,8 +2,9 @@ [](https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/commits/master) [](https://opensource.org/licenses/Apache-2.0) [](https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/tags) +[](https://artifacthub.io/packages/helm/dbrepo/dbrepo-core) -<img src="./dbrepo-ui/static/logo.png" alt="DBREPO — Repository for Data in Databases" width="200" /> + ## tl;dr @@ -19,11 +20,53 @@ curl -sSL https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-service Find a system description, component documentation and endpoint documentation online: https://www.ifs.tuwien.ac.at/infrastructures/dbrepo/. -## Contribute +## Development Contributions are always welcome and encouraged, please read the [contribution overview](./CONTRIBUTING.md) and contact [Prof. Andreas Rauber](http://www.ifs.tuwien.ac.at/~andi/) or [Martin Weise](https://ec.tuwien.ac.at/~weise/). +### Build + +Install the build dependencies under Debian +12 ([Instructions for Docker Engine](https://docs.docker.com/engine/install/debian/#install-using-the-repository)): + +```console +$ apt install -y bash maven openjdk-17-jdk nodejs && npm install --global yarn +$ node --version +v18.19.0 +``` + +Build the Docker containers: + +```console +./bin/build-docker.sh +``` + +### Test + +Install the [build dependencies](#build) as they also cover the test dependencies. + +Test the backend and frontend: + +```console +./bin/test.sh +``` + +## Run + +After [building the docker containers](#build) you can run them using the default `docker-compose.yml` in the root of +the sourcecode directory. This starts all services in the background (as daemons hence the `-d` flag). + +```console +$ docker compose up -d +``` + +Optionally view all logs in real-time: + +```console +$ docker compose logs -f +``` + ## Acknowledgements We want to thank the following organizations: diff --git a/bin/build-docker.sh b/bin/build-docker.sh new file mode 100644 index 0000000000000000000000000000000000000000..0fd30f989cf1710b0493f2f294212f04763d4c1c --- /dev/null +++ b/bin/build-docker.sh @@ -0,0 +1,4 @@ +#!/bin/bash +docker build --network=host -t dbrepo-metadata-service:build --target build dbrepo-metadata-service +docker build --network=host -t dbrepo-data-service:build --target build dbrepo-data-service +docker compose build --parallel \ No newline at end of file diff --git a/.scripts/teardown.sh b/bin/teardown.sh similarity index 100% rename from .scripts/teardown.sh rename to bin/teardown.sh diff --git a/bin/test.sh b/bin/test.sh new file mode 100644 index 0000000000000000000000000000000000000000..765ecaf6b7e2741a5e18d33f29071309d053de16 --- /dev/null +++ b/bin/test.sh @@ -0,0 +1,12 @@ +#!/bin/bash +mvn -f ./dbrepo-metadata-service/pom.xml clean install -DskipTests +# test java services +mvn -f ./dbrepo-metadata-service/pom.xml clean test verify +mvn -f ./dbrepo-data-service/pom.xml clean test verify +# test python services +bash ./dbrepo-analyse-service/test.sh +bash ./dbrepo-search-service/test.sh +# test ui +yarn --cwd ./dbrepo-ui install +yarn --cwd ./dbrepo-ui run test:unit +yarn --cwd ./dbrepo-ui run coverage \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 3eef0cf8c1c1a1892fa8b49f1e268badf854d363..ae8084fafc40d7832f7bf97b9965008a4ea55fd6 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -7,8 +7,8 @@ docs_dir: .docs nav: - Home: index.md - Deployment: - - Kubernetes: deployment-helm.md - Docker Compose: deployment-docker-compose.md + - Kubernetes: deployment-helm.md - System: - Overview: system.md - Services: