From b9f456a234a280c26089c1c6d968379b24c6f4a5 Mon Sep 17 00:00:00 2001
From: Martin Weise <martin.weise@tuwien.ac.at>
Date: Fri, 6 Jan 2023 06:35:08 +0100
Subject: [PATCH] Updated system endpoints
---
docs/contact.md | 1 +
docs/index.md | 6 +-
docs/operation.md | 15 -----
docs/publications.md | 6 +-
docs/system.md | 133 +++++++++++++------------------------------
5 files changed, 48 insertions(+), 113 deletions(-)
diff --git a/docs/contact.md b/docs/contact.md
index 651ff19..c0d5d49 100644
--- a/docs/contact.md
+++ b/docs/contact.md
@@ -33,5 +33,6 @@ A-1040 Vienna, Austria
- Eva Gergely
- Kirill Stytsenko
- Josef Taha
+- Tobias Grantner
- Raman Ganguly
- Andreas Rauber
diff --git a/docs/index.md b/docs/index.md
index 651294b..ee1c343 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -24,9 +24,9 @@ evolving, allows reproducing of query results and supports findable-, accessible
## More Information
-- Public demonstration instance [https://dbrepo.ossdip.at](https://dbrepo.ossdip.at)
-- Join our [mailing list](https://lists.univie.ac.at/mailman/listinfo/fairdata_dbrepo) to receive bi-monthly updates
- about the progress, new features and events!
+- Demonstration instance [https://dbrepo1.ec.tuwien.ac.at](https://dbrepo1.ec.tuwien.ac.at)
+- Sandbox instance [https://dbrepo2.ec.tuwien.ac.at](https://dbrepo2.ec.tuwien.ac.at)
+- System description [https://doi.org/10.2218/ijdc.v17i1.825](https://doi.org/10.2218/ijdc.v17i1.825)
<style>
video {
diff --git a/docs/operation.md b/docs/operation.md
index 4b14568..6371ff0 100644
--- a/docs/operation.md
+++ b/docs/operation.md
@@ -10,18 +10,3 @@ hide:
!!! abstract "Abstract"
We compiled a short [getting started guide](/getting-started) that quickly allows a stand-alone test-deployment of the infrastructure on your server without the need of infrastructure configuration.
-
-## External Systems
-
-We thought the easiest way to make documentation of the endpoints available was through the interactive in-code
-documentation of Swagger:
-
-* [Analyse Service](/swagger/analyse)
-* [Authentication Service](/swagger/authentication)
-* [Container Service](/swagger/container)
-* [Database Service](/swagger/database)
-* [Identifier Service](/swagger/identifier)
-* [Metadata Service](/swagger/metadata)
-* [Query Service](/swagger/query)
-* [Table Service](/swagger/table)
-* [Units Service](/swagger/units)
\ No newline at end of file
diff --git a/docs/publications.md b/docs/publications.md
index d2ccead..242d49a 100644
--- a/docs/publications.md
+++ b/docs/publications.md
@@ -11,7 +11,11 @@ hide:
##### 2022
-1. Weise, M., Staudinger, M., Michlits, C., Gergely, E., Stytsenko, K., Ganguly, R., & Rauber, A. (2022).
+1. Ekaputra, F. E., Weise, M., Flicker, K., Salleh, M. R., Rahman, N. A., Miksa, T., & Rauber, A. (2022). Towards A
+ Data Repository for Educational Factories. *Proceedings of the 8th International Conference on Data and Software
+ Engineering*, pp. 149-154. DOI: [10.1109/ICoDSE56892.2022.9971958](https://doi.org/10.1109/ICoDSE56892.2022.9971958).
+
+2. Weise, M., Staudinger, M., Michlits, C., Gergely, E., Stytsenko, K., Ganguly, R., & Rauber, A. (2022).
DBRepo: a Semantic Digital Repository for Relational Databases. *International Journal of Digital Curation*,
17(1), 11. DOI: [10.2218/ijdc.v17i1.825](https://doi.org/10.2218/ijdc.v17i1.825)
diff --git a/docs/system.md b/docs/system.md
index 11f9b59..5ede2eb 100644
--- a/docs/system.md
+++ b/docs/system.md
@@ -13,8 +13,16 @@ The repository is designed as a microservice architecture to ensure scalability
technologies. The conceptualized microservices operate the basic database operations, data versioning as well as
*findability*, *accessability*, *interoperability* and *reuseability* (FAIR).
+<figure markdown>
+
+<figcaption>Microservice cloud architecture</figcaption>
+</figure>
+
+
## Services
+View the docker images for the documentation of the service.
+
### Discovery Service
This microservice allows service discovery and registration of containers that provide services. It configures
@@ -24,6 +32,7 @@ services.
!!! debug "Debug Information"
* Port(s): 9090
+ * Docker Image: [dbrepo/discovery-service](https://hub.docker.com/repository/docker/dbrepo/discovery-service)
* Swagger: not configured
### Gateway Service
@@ -34,99 +43,47 @@ the [Spring Cloud Gateway](https://spring.io/projects/spring-cloud-gateway) to r
!!! debug "Debug Information"
* Port(s): 9095
+ * Docker Image: [dbrepo/gateway-service](https://hub.docker.com/repository/docker/dbrepo/gateway-service)
* Swagger: not configured
-<figure markdown>
-
-<figcaption>Microservice cloud architecture</figcaption>
-</figure>
-
### Authentication Service
Very specific to the deployment of the organization. In our reference implementation we implement a *security assertion
markup language* (SAML) service provider and use our institutional SAML identity provider for obtaining account data
through an encrypted channel.
-The Authentication Service configures [Spring Boot Starter Security](https://spring.io/guides/gs/securing-web/)
-with [Java JWT](https://github.com/auth0/java-jwt) for internal authentication once the user details are known in the
-metadata database. By default, a token is valid for 24 hours and is used on all HTTP API endpoints.
-
-For the **HTTP API**, obtaining a new token can be done via, e.g. cURL.
-
-```console
-curl -X POST -d '{"username":"username","password":"password"}' -H "Content-Type: application/json" https://dbrepo.ossdip.at/api/auth
-```
-
-Call a secured method by setting the JWT Token as [Bearer Token](https://www.rfc-editor.org/rfc/rfc6750.html) via,
-e.g. cURL.
-
-```console
-curl -X PUT -H "Authorization: Bearer TOKEN" -H "Content-Type: application/json" https://dbrepo.ossdip.at/api/auth
-```
-
-For the **AMQP API**, the Authentication Service also creates a dedicated user at the [Broker Service](#broker-service)
-that has permissions for writing and configuring the RabbitMQ queues that feed into the database owned by this user.
-
!!! debug "Debug Information"
* Port(s): 9097
- * Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9097/swagger-ui/index.html)
- * Swagger API .json: [/v3/api-docs/authentication-service](http://dbrepo.ossdip.at:9097/v3/api-docs/authentication-service)
- * Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9097/v3/api-docs.yaml)
+ * Docker Image: [dbrepo/authentication-service](https://hub.docker.com/repository/docker/dbrepo/authentication-service)
+ * Swagger UI: [/swagger/authentication](/swagger/authentication)
### Metadata Database
It is the core component of the project. It is a relational database that contains metadata about all researcher databases
created in the database repository like column names, check expressions, value enumerations or key/value constraints and
relevant data for citing data sets. Additionally, the concept, e.g. URI of units of measurements of numerical columns is
-stored in the Metadata Database in order to provide semantic knowledge context. We
-use [PostgreSQL](https://www.postgresql.org/) for its rich capabilities in the reference implementation.
-
-The default credentials are `postgres:postgres` for the database `fda`. Connect to the database via, e.g. *psql*.
+stored in the Metadata Database in order to provide semantic knowledge context. We use MariaDB for its rich capabilities
+in the reference implementation.
-```console
-psql -d fda -h localhost -p 5432 -U postgres -W
-```
+The default credentials are `root:dbrepo` for the database `fda`. Connect to the database via the JDBC connector.
!!! debug "Debug Information"
- * Port(s): 5432
- * Swagger: not configured
+ * Port(s): 3306
-### Unit Service
+### Semantics Service
It is designed to map terms in the domain of units of measurement to controlled vocabulary, modelled in
the [ontology of units of measure](https://github.com/HajoRijgersberg/OM). This service validates researcher provided in
units and provides a *uniform resource identifier* (URI) to the related concept, which will be stored in the system.
Furthermore, there is a method for auto-completing text and listing a description as well as commonly used unit symbols.
-The Unit Service reads units of measurement from [`om-2.ttl`](https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/blob/master/fda-units-service/onto/om-2.ttl)
-and registers a unit. It is used to assign a unit of measurement to a table column.
-
-For the **HTTP API**, the Unit Service assigns a unit of measurement via, e.g. cURL. First the list of concepts can be
-queried.
-
-```console
-curl -X POST -d '{"offset":0,"ustring":"met"}' https://dbrepo.ossdip.at/api/units/suggest
-```
-
-Then the concept needs to be saved into the metadata database.
-
-```console
-curl -X POST -d '{"name":"metre","uri":...}' https://dbrepo.ossdip.at/api/units/saveconcept
-```
-
-Then the column can be assigned to a table column.
-
-```console
-curl -X POST -d '{"cdbid":1,"tid":1,"cid":1,"uri":...}' https://dbrepo.ossdip.at/api/units/savecolumnsconcept
-```
-
!!! debug "Debug Information"
* Port(s): 5010
- * Swagger UI: [/swagger-ui/](http://dbrepo.ossdip.at:9097/swagger-ui/)
- * Swagger API .json: [/api-units.json](http://dbrepo.ossdip.at:9097/api-units.json)
+ * Docker Image: [dbrepo/semantics-service](https://hub.docker.com/repository/docker/dbrepo/semantics-service)
+ * Swagger UI: [/swagger/semantics](/swagger/semantics)
### Identifier Service
@@ -139,9 +96,8 @@ identifier* (DOI) through our institutional library soon.
!!! debug "Debug Information"
* Port(s): 9096
- * Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9096/swagger-ui/index.html)
- * Swagger API .json: [/v3/api-docs/identifier-service](http://dbrepo.ossdip.at:9096/v3/api-docs/identifier-service)
- * Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9096/v3/api-docs.yaml)
+ * Docker Image: [dbrepo/identifier-service](https://hub.docker.com/repository/docker/dbrepo/identifier-service)
+ * Swagger UI: [/swagger/identifier](/swagger/identifier)
### Search Service
@@ -153,13 +109,14 @@ with each save operation on databases in the metadata database. The database nam
to e.g. match the term "Airquality"
```console
-curl http://dbrepo.ossdip.at:9200/databaseindex/_search?q=name:Airquality
+curl http://localhost:9200/databaseindex/_search?q=name:Airquality
```
!!! debug "Debug Information"
* Port(s): 9200, 9600
- * ElasticSearch: [/databaseindex](http://dbrepo.ossdip.at:9000/databaseindex)
+ * Docker Image: [elasticsearch](https://hub.docker.com/_/elasticsearch)
+ * ElasticSearch Index
### Container Service
@@ -168,9 +125,8 @@ It is responsible for Docker container lifecycle operations and updating the loc
!!! debug "Debug Information"
* Port(s): 9091
- * Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9091/swagger-ui/index.html)
- * Swagger API .json: [/v3/api-docs/container-service](http://dbrepo.ossdip.at:9091/v3/api-docs/container-service)
- * Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9091/v3/api-docs.yaml)
+ * Docker Image: [dbrepo/container-service](https://hub.docker.com/repository/docker/dbrepo/container-service)
+ * Swagger UI: [/swagger/container](/swagger/container)
### Database Service
@@ -180,9 +136,8 @@ support [MariaDB](https://mariadb.org/) images that allow table versioning with
!!! debug "Debug Information"
* Port(s): 9092
- * Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9092/swagger-ui/index.html)
- * Swagger API .json: [/v3/api-docs/database-service](http://dbrepo.ossdip.at:9092/v3/api-docs/database-service)
- * Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9092/v3/api-docs.yaml)
+ * Docker Image: [dbrepo/database-service](https://hub.docker.com/repository/docker/dbrepo/database-service)
+ * Swagger UI: [/swagger/database](/swagger/database)
### Table Service
@@ -192,29 +147,20 @@ use [Hibernate](https://hibernate.org/orm/) for schema and data ingest operation
!!! debug "Debug Information"
* Port(s): 9094
- * Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9094/swagger-ui/index.html)
- * Swagger API .json: [/v3/api-docs/table-service](http://dbrepo.ossdip.at:9094/v3/api-docs/table-service)
- * Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9094/v3/api-docs.yaml)
+ * Docker Image: [dbrepo/table-service](https://hub.docker.com/repository/docker/dbrepo/table-service)
+ * Swagger UI: [/swagger/table](/swagger/table)
### Broker Service
It holds exchanges and topics responsible for holding AMQP messages for later consumption. We
-use [RabbitMQ](https://www.rabbitmq.com/) in the reference implementation.
-
-For the **HTTP API**, the Broker Service offers an endpoint to manage the AMQP users and their permissions on exchanges
-and queues. This endpoint is reachable via the Gateway Service or at port 9098 directly. Internally this service just
-parses commands to the [`rabbitmqctl`](https://www.rabbitmq.com/rabbitmqctl.8.html).
-
-For the **AMQP API**, the Broker Service can declare exchanges and queues. The AMQP endpoint listens to port 5672 for
+use [RabbitMQ](https://www.rabbitmq.com/) in the reference implementation. The AMQP endpoint listens to port 5672 for
regular declares and offers a management interface at port 15672.
!!! debug "Debug Information"
* Port(s): 9098, 5672, 15672
- * Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9098/swagger-ui/index.html)
- * Swagger API .json: [/v3/api-docs/broker-service](http://dbrepo.ossdip.at:9098/v3/api-docs/broker-service)
- * Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9098/v3/api-docs.yaml)
- * RabbitMQ Management: [/#](http://dbrepo.ossdip.at:15672/#)
+ * Docker Image: [dbrepo/broker-service](https://hub.docker.com/repository/docker/dbrepo/broker-service)
+ * RabbitMQ Management Plugin
### Query Service
@@ -225,10 +171,8 @@ Service.
!!! debug "Debug Information"
* Port(s): 9093
- * Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9093/swagger-ui/index.html)
- * Swagger API .json: [/v3/api-docs/query-service](http://dbrepo.ossdip.at:9093/v3/api-docs/query-service)
- * Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9093/v3/api-docs.yaml)
-
+ * Docker Image: [dbrepo/query-service](https://hub.docker.com/repository/docker/dbrepo/query-service)
+ * Swagger UI: [/swagger/query](/swagger/query)
### FAIR Portal
It provides a *graphical user interface* (GUI) for a researcher to interact with the database repository's API.
@@ -236,7 +180,8 @@ It provides a *graphical user interface* (GUI) for a researcher to interact with
!!! debug "Debug Information"
* Port(s): 3000
- * GUI: [/#](http://dbrepo.ossdip.at:3000)
+ * Docker Image: [dbrepo/ui](https://hub.docker.com/repository/docker/dbrepo/ui)
+ * FAIR Portal
### Analyse Service
@@ -248,8 +193,8 @@ numerical columns.
!!! debug "Debug Information"
* Port(s): 5000
- * Swagger UI: [/swagger-ui/](http://dbrepo.ossdip.at:5000/swagger-ui/)
- * Swagger API .json: [/api-analyze.json](http://dbrepo.ossdip.at:5000/api-analyze.json)
+ * Docker Image: [dbrepo/analyse-service](https://hub.docker.com/repository/docker/dbrepo/analyse-service)
+ * Swagger UI: [/swagger/analyse](/swagger/analyse)
## Database
--
GitLab