diff --git a/docs/deployment-docker-compose.md b/docs/deployment-docker-compose.md index ecf0aaa2f53e20b758ff11751d89600aa018c3ad..5263968a57eec9d779d7f5e165c479e587ebb1c7 100644 --- a/docs/deployment-docker-compose.md +++ b/docs/deployment-docker-compose.md @@ -26,6 +26,16 @@ technologies. The conceptualized microservices operate the basic database operat Alternatively, you can also deploy DBRepo with [Helm](../deployment-helm/) in your virtual machine instead. +## Environment Values + +| Key | Type | Default | Description | +|------------------------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `DBREPO_CLIENT_ID` | string | `dbrepo-client` | Client ID of the keycloak client for API communication. | +| `DBREPO_CLIENT_SECRET` | string | `MUwRc7yfXSJwX8AdRMWaQC3Nep1VjwgG` | Client secret of the keycloak client, this should be changed in the admin console of keycloak. | +| `JWT_ISSUER` | string | `http://localhost/api/auth/realms/dbrepo` | The issuer in the JWT `iss` field of the (decoded) token. Public deployments with hostnames other than localhost need to change that. The issuer always has the form `<PROTOCOL>://<HOSTNAME>/api/auth/realms/dbrepo`, e.g. change PROTOCOL to https for SSL/TLS deployments. | +| `JWT_PUBKEY` | string | `MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB` | Public key that can verify the JWT signature, this should be changed. | +| `JWT_CERT` | string | `MIICmzCCAYMCBgGG3GWyBTANBgkqhkiG9w0BAQsFADARMQ8wDQYDVQQDDAZkYnJlcG8wHhcNMjMwMzEzMTkxMzE3WhcNMzMwMzEzMTkxNDU3WjARMQ8wDQYDVQQDDAZkYnJlcG8wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCqqcdDYFZZb28M0tEJzEP77FmD/Xqioyj9zWX6VwUSOMAgmMmn8eqs9hT9T0a+q4YTo9tUW1PNbUpwprA5b4Uk04DcIajxDVMUR/PjcHytmkqwVskq9AZW/Vngdoo+8tSbuIybwe/3Vwt266hbHpDcM97a+DXcYooRl7tQWCEX7RP27wQrMD9epDQ6IgKayZg9vC9/03dsIqwH9jXQRiZlFvwiEKhX2aY7lPGBaCK414JO00K/Z49iov9TRa/IYVbSt5qwgrx6DcqsBSPwOnI6A85UGfeUEZ/7coVJiL7RvBlsllapsL9eWTbQajVh94k9Ei3sibEPbtH+U2OAM78zAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAASnN1Cuif1sdfEK2kWAURSXGJCohCROLWdKFjaeHPRaEfpbFJsgxW0Yj3nwX5O3bUlOWoTyENwnXSsXMQsqnNi+At32CKaKO8+AkhAbgQL9F0B+KeJwmYv3cUj5N/LYkJjBvZBzUZ4Ugu5dcxH0k7AktLAIwimkyEnxTNolOA3UyrGGpREr8MCKWVr10RFuOpF/0CsJNNwbHXzalO9D756EUcRWZ9VSg6QVNso0YYRKTnILWDn9hcTRnqGy3SHo3anFTqQZ+BB57YbgFWy6udC0LYRB3zdp6zNti87eu/VEymiDY/mmo1AB8Tm0b6vxFz4AKcL3ax5qS6YnZ9efSzk=IJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB` | Public key that can verify the JWT signature, this should be changed. | + ## Requirements ### Hardware diff --git a/docs/deployment-helm.md b/docs/deployment-helm.md index 210d5f461a82c655ec17d86e45c735235d735373..5bbac98ef346f18652a922ff368c2301ac89228b 100644 --- a/docs/deployment-helm.md +++ b/docs/deployment-helm.md @@ -35,6 +35,7 @@ helm upgrade --install dbrepo \ | `replicaCount` | int | `1` | Number of replicas (pods) to launch. | | `nameOverride` | string | `""` | A name in place of the chart name for `app:` labels. | | `fullnameOverride` | string | `""` | A name to substitute for the full names of resources. | +| `imagePullPolicy` | string | `Always` | Override for the image pull policy, e.g. `IfNotPresent` | | `adminEmail` | string | `noreply@example.com` | E-mail address for OAI-PMH metadata. | | `repositoryName` | string | `Database Repository` | Repository name for OAI-PMH metadata. | | `hostname` | string | `example.com` | Domain name for the deployment, should not contain `https://` or any path. | diff --git a/docs/system-services-metadata.md b/docs/system-services-metadata.md index 41b5a89468c2515dbbcfc6717618f00da4ead55c..15e3cc7f217448e22b317d5ccf589886f956eefa 100644 --- a/docs/system-services-metadata.md +++ b/docs/system-services-metadata.md @@ -6,22 +6,22 @@ author: Martin Weise !!! debug "Debug Information" - * Ports: 9092/tcp - * Info: `http://:9092/actuator/info` - * Health: `http://:9092/actuator/health` - * Prometheus: `http://:9092/actuator/prometheus` - * Swagger UI: `http://:9092/swagger-ui/index.html` <a href="/infrastructures/dbrepo/latest/swagger/database" target="_blank">:fontawesome-solid-square-up-right: view online</a> + * Ports: 9099/tcp + * Info: `http://:9099/actuator/info` + * Health: `http://:9099/actuator/health` + * Prometheus: `http://:9099/actuator/prometheus` + * Swagger UI: `http://:9099/swagger-ui/index.html` <a href="/infrastructures/dbrepo/latest/swagger/metadata" target="_blank">:fontawesome-solid-square-up-right: view online</a> -It creates the databases inside a Docker container and the Query Store. Currently, we only -support [MariaDB](https://mariadb.org/) images that allow table versioning with low programmatic effort. +This microservice manages: -!!! debug "Debug Information" +* Databases +* Users +* Metadata (PID, OAI-PMH) +* Queries, views +* Tables - * Ports: 9096/tcp - * Info: `http://:9096/actuator/info` - * Health: `http://:9096/actuator/health` - * Prometheus: `http://:9096/actuator/prometheus` - * Swagger UI: `http://:9096/swagger-ui/index.html` <a href="/infrastructures/dbrepo/latest/swagger/identifier" target="_blank">:fontawesome-solid-square-up-right: view online</a> +It creates the databases inside a Docker container and the Query Store. Currently, we only +support [MariaDB](https://mariadb.org/) images that allow table versioning with low programmatic effort. This microservice is responsible for creating and resolving a *persistent identifier* (PID) attached to a query to obtain the metadata attached to it and allow re-execution of a query. We store both the query and hashes of the query @@ -29,45 +29,11 @@ and result set to allow equality checks of the originally obtained result set an the reference implementation we currently only use a numerical id column and plan to integrate *digital object identifier* (DOI) through our institutional library soon. -!!! debug "Debug Information" - - * Ports: 9099/tcp - * Info: `http://:9099/actuator/info` - * Health: `http://:9099/actuator/health` - * Prometheus: `http://:9099/actuator/prometheus` - * Swagger UI: `http://:9099/swagger-ui/index.html` <a href="/infrastructures/dbrepo/latest/swagger/metadata" target="_blank">:fontawesome-solid-square-up-right: view online</a> - This service provides an OAI-PMH endpoint for metadata crawler. -!!! debug "Debug Information" - - * Ports: 9093/tcp - * Info: `http://:9093/actuator/info` - * Health: `http://:9093/actuator/health` - * Prometheus: `http://:9093/actuator/prometheus` - * Swagger UI: `http://:9093/swagger-ui/index.html` <a href="/infrastructures/dbrepo/latest/swagger/query" target="_blank">:fontawesome-solid-square-up-right: view online</a> - It provides an interface to insert data into the tables created by the Table Service. It also allows for view-only, paginated and versioned query execution to the raw data and consumes messages in the message queue from the Broker Service. -!!! debug "Debug Information" - - * Ports: 9094/tcp - * Info: `http://:9094/actuator/info` - * Health: `http://:9094/actuator/health` - * Prometheus: `http://:9094/actuator/prometheus` - * Swagger UI: `http://:9094/swagger-ui/index.html` <a href="/infrastructures/dbrepo/latest/swagger/table" target="_blank">:fontawesome-solid-square-up-right: view online</a> - This microservice handles table operations inside a database that is managed by the Database Service. We use [Hibernate](https://hibernate.org/orm/) for schema and data ingest operations. - -!!! debug "Debug Information" - - * Ports: 9098/tcp - * Info: `http://:9098/actuator/info` - * Health: `http://:9098/actuator/health` - * Prometheus: `http://:9098/actuator/prometheus` - * Swagger UI: `http://:9098/swagger-ui/index.html` <a href="/infrastructures/dbrepo/latest/swagger/user" target="_blank">:fontawesome-solid-square-up-right: view online</a> - -This microservice handles user information. diff --git a/docs/system-services-upload.md b/docs/system-services-upload.md new file mode 100644 index 0000000000000000000000000000000000000000..40f9d3d61c3c2a122df5e5addb1e895f217563c2 --- /dev/null +++ b/docs/system-services-upload.md @@ -0,0 +1,27 @@ +--- +author: Martin Weise +--- + +# Upload Service + +!!! debug "Debug Information" + + * Ports: 1080/tcp + +Upload files using one of the official the TUSd clients: + +* [NodeJS / JavaScript](https://github.com/tus/tus-js-client) +* [Java](https://github.com/tus/tus-java-client) +* [Python](https://github.com/tus/tus-py-client) + +## Example (Python) + +Uploads a file `file.csv` in 200 byte chunks. + +```python +#!/bin/env python3 +from tusclient import client +my_client = client.TusClient('http://localhost/api/upload/files') +uploader = my_client.uploader('/path/to/file.csv', chunk_size=200) +uploader.upload() +``` diff --git a/mkdocs.yml b/mkdocs.yml index 15100d6483a0001f460a1b9b9636fc1a7da11055..8bf20165eca7c6ff4d71309def3b0237f180903d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -19,6 +19,7 @@ nav: - Broker Service: system-services-broker.md - Gateway Service: system-services-gateway.md - Metadata Service: system-services-metadata.md + - Upload Service: system-services-upload.md - Databases: - Data Database: system-databases-data.md - Metadata Database: system-databases-metadata.md