Added more documentation

.docs/DEVELOPMENT.md → .docs/dev-guide-app.md

-# Development Guide
+# Application Developer Guide
 
 ## Dependencies
 
-Local development depends on the following packages:
+Local development depends on the following packages for Debian 12:
 
-* [Apache Maven](https://maven.apache.org/) 3+
-* [Java JDK](https://openjdk.org/) 17 (LTS)
-* [Docker Engine](https://docs.docker.com/engine/install/) 24+
+```shell
+apt install -y maven openjdk-17-jdk make
+```
 
-Optional but recommended:
+Required tools with their own installing guides:
 
-* [GNU Make](https://www.gnu.org/software/make/) 4+
+* [Docker Engine](https://docs.docker.com/engine/install/) 24+
+* [Minikube](https://minikube.sigs.k8s.io/docs/start/) 1.32.0
 
 ## Getting Started
 
@@ -25,11 +26,13 @@ mvn -f ./dbrepo-metadata-service/pom.xml clean install -DskipTests
 
 We practice test-driven development and require contributors to test their code with at least 90% code coverage.
 
-## Code Documentation
-
-Before creating a merge request, make sure you:
+```shell
+make test
+```
 
-- [x] Generate the [Swagger Docs](#swagger-docs) endpoint documentation
+The Java-based services have the coverage reports generated by `JaCoCo` in the `report/site/` subdirectory, the
+Python-based services have the coverage reports generated by `coverage` in the `.coverage` SQLite3 database
+and `coverage.txt` log file respectively.
 
 ### Swagger Docs
 
@@ -41,20 +44,20 @@ bash .swagger/swagger-generate.sh
 
 ### Branching Strategy
 
-<p align="center">
-<img src="../.gitlab/branching-strategy.png" alt="Branching strategy from the master-dev-feature branches and release branches." width="732" height="391" /><br/>
-<i><strong>Figure 1.</strong> Branching strategy of the source code development.</i>
-</p>
+<figure markdown>
+![Branching strategy from the master-dev-feature branches and release branches](images/branching-strategy.png)
+<figcaption>Figure 1: Branching strategy of the source code development.</figcaption>
+</figure>
 
 ### CI/CD
 
 We get compute resources in-kind from [dataLAB](https://www.it.tuwien.ac.at/en/services/network-and-servers/datalab)
 to run our pipeline:
 
-<p align="center">
-<img src="../.gitlab/gitlab-runner.png" alt="Gitlab runner configuration in the cluster" width="732" height="262" /><br/>
-<i><strong>Figure 2.</strong> Gitlab runner configuration in the cluster.</i>
-</p>
+<figure markdown>
+![Gitlab runner configuration in the cluster](images/gitlab-runner.png)
+<figcaption>Figure 2: Gitlab runner configuration in the cluster.</figcaption>
+</figure>
 
 Minikube cluster with 6vCPU and 28GB RAM. The CI pipeline is configured as follows in the `config.toml`:
 

.docs/dev-guide-infra.md

+# Infrastructure Developer Guide
+
+## tl;dr
+
+```shell
+make cluster-start cluster-image-pull cluster-install
+```
+
+## Dependencies
+
+Local development depends on the following packages for Debian 12:
+
+```shell
+apt install -y make
+```
+
+Required tools with their own installing guides:
+
+* [Docker Engine](https://docs.docker.com/engine/install/) 24+
+* [Minikube](https://minikube.sigs.k8s.io/docs/start/) 1.32.0
+
+## Getting Started
+
+Start the local development cluster with the Docker driver (takes at least 8 vCPUs and 12GB RAM). It installs a Minikube
+single-node Kubernetes cluster with enabled Ingress and Dashboard
+
+```shell
+make cluster-start
+```
+
+Build the local images with `make build-docker` and copy them to the cluster image cache:
+
+```shell
+make cluster-image-pull
+```
+
+Build and install the Helm chart:
+
+```shell
+make cluster-install
+```
+
+## Debug
+
+Open the Minikube (Kubernetes) Dashboard:
+
+```shell
+make cluster-dashboard
+```
+
+<figure markdown>
+![Minikube Dashboard](images/screenshots/minikube-dashboard.png)
+<figcaption>Figure 1: Minikube Dashboard</figcaption>
+</figure>
+
+Optionally enable the Prometheus metrics addon with:
+
+```shell
+minikube addons enable metrics-server
+```
+
+## Test
+
+Test if the Helm chart raises errors on start (the script aborts after 5 minutes automatically if some pods are not
+starting or erroneous).
+
+```shell
+make cluster-test
+```
+
+## Uninstall
+
+To uninstall DBRepo from the local Minikube cluster, removing all data:
+
+```shell
+make cluster-uninstall
+```
+

.docs/dev-overview.md

+# Overview
+
+## Guides
+
+* The [application developer guide](../dev-guide-app) guides you through the steps on how to build DBRepo from
+  scratch and customize the application.
+* The [infrastructure developer guide](../dev-guide-infra) guides you through the steps on how to build and customize
+  the operation environment.
+
+## Organization
+
+* Monthly sprints with patch-releases (i.e. `1.4.2` in February, `1.4.3` in March, ...).
+* Branching from `dev` for feature development, one release branch per patch (i.e. `release-1.4.2` for release version
+  `1.4.2`).
+
+## Roadmap
+
+- [x] Q1: Python library, versioning in every component, bumping frontend versions, i18n
+- [ ] Q2: Kubernetes deployment guidelines for OpenShift
+- [ ] Q3: TBD
+- [ ] Q4: Release of 2.0.0
\ No newline at end of file

Makefile

@@ -241,12 +241,16 @@ helm-build:
 	helm package ./helm-charts/dbrepo --destination ./build
 
 cluster-start:
-	minikube start --driver="docker" --memory="24g" --cpus="8" # 2 CPUs for Control Plane + 6
+	minikube start --driver="docker" --memory="12g" --cpus="8" # 2 CPUs for Control Plane + 6
 	minikube addons disable metrics-server
 	minikube addons enable ingress && minikube addons enable dashboard
 	./helm-charts/dbrepo/hack/add-hosts.sh
 	#CERT_MANAGER_VERSION=1.14.4 ./helm-charts/dbrepo/hack/install-cert-manager.sh
 
+cluster-test: cluster-start cluster-image-pull cluster-install
+	bash ./helm-charts/dbrepo/test.sh
+	minikube stop
+
 cluster-stop:
 	minikube stop
 

helm-charts/dbrepo/hack/add-hosts.sh

 #!/bin/bash
+cat /etc/hosts | grep "dbrepo.local"
+if [ "$?" -ne 0 ]; then
   echo "$(minikube ip)  dbrepo.local" | sudo tee -a /etc/hosts
+fi
\ No newline at end of file

helm-charts/dbrepo/test.sh

+#!/bin/bash
+NAMESPACE=dbrepo
+
+echo "Waiting for containers to start ..."
+SUCCESS=0
+for i in 0 1 2 3 4 5 6 7 8 9; do
+  RES=$(kubectl -n ${NAMESPACE} get pods | awk 'NR>1 {print $3}' | grep -qF "[^Running|^Completed]")
+  if [ "$?" -ne 0 ]; then
+    if [ $SUCCESS -eq 0 ]; then
+      echo "INFO: all pods started or completed: wait another iteration ..."
+      sleep 30
+      SUCCESS=1
+      continue
+    fi
+    echo "INFO: all pods started or completed"
+    exit 0
+  fi
+  echo "Waiting ..."
+  sleep 30
+done
+echo "ERROR: some pods did not successfully complete or are still running"
+exit 1
\ No newline at end of file

helm-charts/dbrepo/values.yaml

@@ -30,8 +30,7 @@ metadataDb:
     loadBalancerIP: ""
     loadBalancerSourceRanges: [ ]
   persistence:
-    enabled: false
-    size: 10Gi
+    enabled: true
   replicaCount: 1 # uneven 3,5,7
 
 authService:
@@ -92,7 +91,7 @@ authDb:
     loadBalancerIP: ""
     loadBalancerSourceRanges: [ ]
   persistence:
-    enabled: false
+    enabled: true
     size: 10Gi
 
 dataDb:
@@ -149,7 +148,7 @@ dataDb:
       persistentVolumeClaim:
         claimName: data-db-shared
   persistence:
-    enabled: false
+    enabled: true
     size: 10Gi
   replicaCount: 1 # uneven
 
@@ -168,7 +167,7 @@ searchdb:
   sysctlInit:
     enabled: true
   persistence:
-    enabled: false
+    enabled: true
     size: 10Gi
   service:
     type: ClusterIP
@@ -491,7 +490,7 @@ ingress:
   enabled: true
   className: nginx
   tls:
-    enabled: false
+    enabled: true
     secretName: ingress-cert
   annotations:
     basic: {}

mkdocs.yml

@@ -36,6 +36,10 @@ nav:
       - Authentication Service: usage-authentication.md
       - Storage Service: usage-storage.md
       - Upload Service: usage-upload.md
+  - Development:
+    - Overview: dev-overview.md
+    - Application Guide: dev-guide-app.md
+    - Infrastructure Guide: dev-guide-infra.md
   - publications.md
   - contact.md
 extra_css:
@@ -86,6 +90,8 @@ markdown_extensions:
   - admonition
   - pymdownx.details
   - pymdownx.superfences
+  - pymdownx.tasklist:
+      custom_checkbox: true
   - pymdownx.tabbed:
       alternate_style: true
   - toc: