Skip to content
Snippets Groups Projects
Unverified Commit b9f456a2 authored by Martin Weise's avatar Martin Weise
Browse files

Updated system endpoints

parent 2ee00075
No related branches found
No related tags found
No related merge requests found
Pipeline #3363 passed
...@@ -33,5 +33,6 @@ A-1040 Vienna, Austria ...@@ -33,5 +33,6 @@ A-1040 Vienna, Austria
- Eva Gergely - Eva Gergely
- Kirill Stytsenko - Kirill Stytsenko
- Josef Taha - Josef Taha
- Tobias Grantner
- Raman Ganguly - Raman Ganguly
- Andreas Rauber - Andreas Rauber
...@@ -24,9 +24,9 @@ evolving, allows reproducing of query results and supports findable-, accessible ...@@ -24,9 +24,9 @@ evolving, allows reproducing of query results and supports findable-, accessible
## More Information ## More Information
- Public demonstration instance [https://dbrepo.ossdip.at](https://dbrepo.ossdip.at) - Demonstration instance [https://dbrepo1.ec.tuwien.ac.at](https://dbrepo1.ec.tuwien.ac.at)
- Join our [mailing list](https://lists.univie.ac.at/mailman/listinfo/fairdata_dbrepo) to receive bi-monthly updates - Sandbox instance [https://dbrepo2.ec.tuwien.ac.at](https://dbrepo2.ec.tuwien.ac.at)
about the progress, new features and events! - System description [https://doi.org/10.2218/ijdc.v17i1.825](https://doi.org/10.2218/ijdc.v17i1.825)
<style> <style>
video { video {
......
...@@ -10,18 +10,3 @@ hide: ...@@ -10,18 +10,3 @@ hide:
!!! abstract "Abstract" !!! abstract "Abstract"
We compiled a short [getting started guide](/getting-started) that quickly allows a stand-alone test-deployment of the infrastructure on your server without the need of infrastructure configuration. We compiled a short [getting started guide](/getting-started) that quickly allows a stand-alone test-deployment of the infrastructure on your server without the need of infrastructure configuration.
## External Systems
We thought the easiest way to make documentation of the endpoints available was through the interactive in-code
documentation of Swagger:
* [Analyse Service](/swagger/analyse)
* [Authentication Service](/swagger/authentication)
* [Container Service](/swagger/container)
* [Database Service](/swagger/database)
* [Identifier Service](/swagger/identifier)
* [Metadata Service](/swagger/metadata)
* [Query Service](/swagger/query)
* [Table Service](/swagger/table)
* [Units Service](/swagger/units)
\ No newline at end of file
...@@ -11,7 +11,11 @@ hide: ...@@ -11,7 +11,11 @@ hide:
##### 2022 ##### 2022
1. Weise, M., Staudinger, M., Michlits, C., Gergely, E., Stytsenko, K., Ganguly, R., & Rauber, A. (2022). 1. Ekaputra, F. E., Weise, M., Flicker, K., Salleh, M. R., Rahman, N. A., Miksa, T., & Rauber, A. (2022). Towards A
Data Repository for Educational Factories. *Proceedings of the 8th International Conference on Data and Software
Engineering*, pp. 149-154. DOI: [10.1109/ICoDSE56892.2022.9971958](https://doi.org/10.1109/ICoDSE56892.2022.9971958).
2. Weise, M., Staudinger, M., Michlits, C., Gergely, E., Stytsenko, K., Ganguly, R., & Rauber, A. (2022).
DBRepo: a Semantic Digital Repository for Relational Databases. *International Journal of Digital Curation*, DBRepo: a Semantic Digital Repository for Relational Databases. *International Journal of Digital Curation*,
17(1), 11. DOI: [10.2218/ijdc.v17i1.825](https://doi.org/10.2218/ijdc.v17i1.825) 17(1), 11. DOI: [10.2218/ijdc.v17i1.825](https://doi.org/10.2218/ijdc.v17i1.825)
......
...@@ -13,8 +13,16 @@ The repository is designed as a microservice architecture to ensure scalability ...@@ -13,8 +13,16 @@ The repository is designed as a microservice architecture to ensure scalability
technologies. The conceptualized microservices operate the basic database operations, data versioning as well as technologies. The conceptualized microservices operate the basic database operations, data versioning as well as
*findability*, *accessability*, *interoperability* and *reuseability* (FAIR). *findability*, *accessability*, *interoperability* and *reuseability* (FAIR).
<figure markdown>
![Microservice cloud architecture](/images/interaction-gateway.svg)
<figcaption>Microservice cloud architecture</figcaption>
</figure>
## Services ## Services
View the docker images for the documentation of the service.
### Discovery Service ### Discovery Service
This microservice allows service discovery and registration of containers that provide services. It configures This microservice allows service discovery and registration of containers that provide services. It configures
...@@ -24,6 +32,7 @@ services. ...@@ -24,6 +32,7 @@ services.
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 9090 * Port(s): 9090
* Docker Image: [dbrepo/discovery-service](https://hub.docker.com/repository/docker/dbrepo/discovery-service)
* Swagger: not configured * Swagger: not configured
### Gateway Service ### Gateway Service
...@@ -34,99 +43,47 @@ the [Spring Cloud Gateway](https://spring.io/projects/spring-cloud-gateway) to r ...@@ -34,99 +43,47 @@ the [Spring Cloud Gateway](https://spring.io/projects/spring-cloud-gateway) to r
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 9095 * Port(s): 9095
* Docker Image: [dbrepo/gateway-service](https://hub.docker.com/repository/docker/dbrepo/gateway-service)
* Swagger: not configured * Swagger: not configured
<figure markdown>
![Microservice cloud architecture](/images/interaction-gateway.svg)
<figcaption>Microservice cloud architecture</figcaption>
</figure>
### Authentication Service ### Authentication Service
Very specific to the deployment of the organization. In our reference implementation we implement a *security assertion Very specific to the deployment of the organization. In our reference implementation we implement a *security assertion
markup language* (SAML) service provider and use our institutional SAML identity provider for obtaining account data markup language* (SAML) service provider and use our institutional SAML identity provider for obtaining account data
through an encrypted channel. through an encrypted channel.
The Authentication Service configures [Spring Boot Starter Security](https://spring.io/guides/gs/securing-web/)
with [Java JWT](https://github.com/auth0/java-jwt) for internal authentication once the user details are known in the
metadata database. By default, a token is valid for 24 hours and is used on all HTTP API endpoints.
For the **HTTP API**, obtaining a new token can be done via, e.g. cURL.
```console
curl -X POST -d '{"username":"username","password":"password"}' -H "Content-Type: application/json" https://dbrepo.ossdip.at/api/auth
```
Call a secured method by setting the JWT Token as [Bearer Token](https://www.rfc-editor.org/rfc/rfc6750.html) via,
e.g. cURL.
```console
curl -X PUT -H "Authorization: Bearer TOKEN" -H "Content-Type: application/json" https://dbrepo.ossdip.at/api/auth
```
For the **AMQP API**, the Authentication Service also creates a dedicated user at the [Broker Service](#broker-service)
that has permissions for writing and configuring the RabbitMQ queues that feed into the database owned by this user.
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 9097 * Port(s): 9097
* Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9097/swagger-ui/index.html) * Docker Image: [dbrepo/authentication-service](https://hub.docker.com/repository/docker/dbrepo/authentication-service)
* Swagger API .json: [/v3/api-docs/authentication-service](http://dbrepo.ossdip.at:9097/v3/api-docs/authentication-service) * Swagger UI: [/swagger/authentication](/swagger/authentication)
* Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9097/v3/api-docs.yaml)
### Metadata Database ### Metadata Database
It is the core component of the project. It is a relational database that contains metadata about all researcher databases It is the core component of the project. It is a relational database that contains metadata about all researcher databases
created in the database repository like column names, check expressions, value enumerations or key/value constraints and created in the database repository like column names, check expressions, value enumerations or key/value constraints and
relevant data for citing data sets. Additionally, the concept, e.g. URI of units of measurements of numerical columns is relevant data for citing data sets. Additionally, the concept, e.g. URI of units of measurements of numerical columns is
stored in the Metadata Database in order to provide semantic knowledge context. We stored in the Metadata Database in order to provide semantic knowledge context. We use MariaDB for its rich capabilities
use [PostgreSQL](https://www.postgresql.org/) for its rich capabilities in the reference implementation. in the reference implementation.
The default credentials are `postgres:postgres` for the database `fda`. Connect to the database via, e.g. *psql*.
```console The default credentials are `root:dbrepo` for the database `fda`. Connect to the database via the JDBC connector.
psql -d fda -h localhost -p 5432 -U postgres -W
```
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 5432 * Port(s): 3306
* Swagger: not configured
### Unit Service ### Semantics Service
It is designed to map terms in the domain of units of measurement to controlled vocabulary, modelled in It is designed to map terms in the domain of units of measurement to controlled vocabulary, modelled in
the [ontology of units of measure](https://github.com/HajoRijgersberg/OM). This service validates researcher provided in the [ontology of units of measure](https://github.com/HajoRijgersberg/OM). This service validates researcher provided in
units and provides a *uniform resource identifier* (URI) to the related concept, which will be stored in the system. units and provides a *uniform resource identifier* (URI) to the related concept, which will be stored in the system.
Furthermore, there is a method for auto-completing text and listing a description as well as commonly used unit symbols. Furthermore, there is a method for auto-completing text and listing a description as well as commonly used unit symbols.
The Unit Service reads units of measurement from [`om-2.ttl`](https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/blob/master/fda-units-service/onto/om-2.ttl)
and registers a unit. It is used to assign a unit of measurement to a table column.
For the **HTTP API**, the Unit Service assigns a unit of measurement via, e.g. cURL. First the list of concepts can be
queried.
```console
curl -X POST -d '{"offset":0,"ustring":"met"}' https://dbrepo.ossdip.at/api/units/suggest
```
Then the concept needs to be saved into the metadata database.
```console
curl -X POST -d '{"name":"metre","uri":...}' https://dbrepo.ossdip.at/api/units/saveconcept
```
Then the column can be assigned to a table column.
```console
curl -X POST -d '{"cdbid":1,"tid":1,"cid":1,"uri":...}' https://dbrepo.ossdip.at/api/units/savecolumnsconcept
```
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 5010 * Port(s): 5010
* Swagger UI: [/swagger-ui/](http://dbrepo.ossdip.at:9097/swagger-ui/) * Docker Image: [dbrepo/semantics-service](https://hub.docker.com/repository/docker/dbrepo/semantics-service)
* Swagger API .json: [/api-units.json](http://dbrepo.ossdip.at:9097/api-units.json) * Swagger UI: [/swagger/semantics](/swagger/semantics)
### Identifier Service ### Identifier Service
...@@ -139,9 +96,8 @@ identifier* (DOI) through our institutional library soon. ...@@ -139,9 +96,8 @@ identifier* (DOI) through our institutional library soon.
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 9096 * Port(s): 9096
* Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9096/swagger-ui/index.html) * Docker Image: [dbrepo/identifier-service](https://hub.docker.com/repository/docker/dbrepo/identifier-service)
* Swagger API .json: [/v3/api-docs/identifier-service](http://dbrepo.ossdip.at:9096/v3/api-docs/identifier-service) * Swagger UI: [/swagger/identifier](/swagger/identifier)
* Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9096/v3/api-docs.yaml)
### Search Service ### Search Service
...@@ -153,13 +109,14 @@ with each save operation on databases in the metadata database. The database nam ...@@ -153,13 +109,14 @@ with each save operation on databases in the metadata database. The database nam
to e.g. match the term "Airquality" to e.g. match the term "Airquality"
```console ```console
curl http://dbrepo.ossdip.at:9200/databaseindex/_search?q=name:Airquality curl http://localhost:9200/databaseindex/_search?q=name:Airquality
``` ```
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 9200, 9600 * Port(s): 9200, 9600
* ElasticSearch: [/databaseindex](http://dbrepo.ossdip.at:9000/databaseindex) * Docker Image: [elasticsearch](https://hub.docker.com/_/elasticsearch)
* ElasticSearch Index
### Container Service ### Container Service
...@@ -168,9 +125,8 @@ It is responsible for Docker container lifecycle operations and updating the loc ...@@ -168,9 +125,8 @@ It is responsible for Docker container lifecycle operations and updating the loc
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 9091 * Port(s): 9091
* Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9091/swagger-ui/index.html) * Docker Image: [dbrepo/container-service](https://hub.docker.com/repository/docker/dbrepo/container-service)
* Swagger API .json: [/v3/api-docs/container-service](http://dbrepo.ossdip.at:9091/v3/api-docs/container-service) * Swagger UI: [/swagger/container](/swagger/container)
* Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9091/v3/api-docs.yaml)
### Database Service ### Database Service
...@@ -180,9 +136,8 @@ support [MariaDB](https://mariadb.org/) images that allow table versioning with ...@@ -180,9 +136,8 @@ support [MariaDB](https://mariadb.org/) images that allow table versioning with
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 9092 * Port(s): 9092
* Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9092/swagger-ui/index.html) * Docker Image: [dbrepo/database-service](https://hub.docker.com/repository/docker/dbrepo/database-service)
* Swagger API .json: [/v3/api-docs/database-service](http://dbrepo.ossdip.at:9092/v3/api-docs/database-service) * Swagger UI: [/swagger/database](/swagger/database)
* Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9092/v3/api-docs.yaml)
### Table Service ### Table Service
...@@ -192,29 +147,20 @@ use [Hibernate](https://hibernate.org/orm/) for schema and data ingest operation ...@@ -192,29 +147,20 @@ use [Hibernate](https://hibernate.org/orm/) for schema and data ingest operation
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 9094 * Port(s): 9094
* Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9094/swagger-ui/index.html) * Docker Image: [dbrepo/table-service](https://hub.docker.com/repository/docker/dbrepo/table-service)
* Swagger API .json: [/v3/api-docs/table-service](http://dbrepo.ossdip.at:9094/v3/api-docs/table-service) * Swagger UI: [/swagger/table](/swagger/table)
* Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9094/v3/api-docs.yaml)
### Broker Service ### Broker Service
It holds exchanges and topics responsible for holding AMQP messages for later consumption. We It holds exchanges and topics responsible for holding AMQP messages for later consumption. We
use [RabbitMQ](https://www.rabbitmq.com/) in the reference implementation. use [RabbitMQ](https://www.rabbitmq.com/) in the reference implementation. The AMQP endpoint listens to port 5672 for
For the **HTTP API**, the Broker Service offers an endpoint to manage the AMQP users and their permissions on exchanges
and queues. This endpoint is reachable via the Gateway Service or at port 9098 directly. Internally this service just
parses commands to the [`rabbitmqctl`](https://www.rabbitmq.com/rabbitmqctl.8.html).
For the **AMQP API**, the Broker Service can declare exchanges and queues. The AMQP endpoint listens to port 5672 for
regular declares and offers a management interface at port 15672. regular declares and offers a management interface at port 15672.
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 9098, 5672, 15672 * Port(s): 9098, 5672, 15672
* Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9098/swagger-ui/index.html) * Docker Image: [dbrepo/broker-service](https://hub.docker.com/repository/docker/dbrepo/broker-service)
* Swagger API .json: [/v3/api-docs/broker-service](http://dbrepo.ossdip.at:9098/v3/api-docs/broker-service) * RabbitMQ Management Plugin
* Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9098/v3/api-docs.yaml)
* RabbitMQ Management: [/#](http://dbrepo.ossdip.at:15672/#)
### Query Service ### Query Service
...@@ -225,10 +171,8 @@ Service. ...@@ -225,10 +171,8 @@ Service.
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 9093 * Port(s): 9093
* Swagger UI: [/swagger-ui/index.html](http://dbrepo.ossdip.at:9093/swagger-ui/index.html) * Docker Image: [dbrepo/query-service](https://hub.docker.com/repository/docker/dbrepo/query-service)
* Swagger API .json: [/v3/api-docs/query-service](http://dbrepo.ossdip.at:9093/v3/api-docs/query-service) * Swagger UI: [/swagger/query](/swagger/query)
* Swagger API .yaml: [/v3/api-docs.yaml](http://dbrepo.ossdip.at:9093/v3/api-docs.yaml)
### FAIR Portal ### FAIR Portal
It provides a *graphical user interface* (GUI) for a researcher to interact with the database repository's API. It provides a *graphical user interface* (GUI) for a researcher to interact with the database repository's API.
...@@ -236,7 +180,8 @@ It provides a *graphical user interface* (GUI) for a researcher to interact with ...@@ -236,7 +180,8 @@ It provides a *graphical user interface* (GUI) for a researcher to interact with
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 3000 * Port(s): 3000
* GUI: [/#](http://dbrepo.ossdip.at:3000) * Docker Image: [dbrepo/ui](https://hub.docker.com/repository/docker/dbrepo/ui)
* FAIR Portal
### Analyse Service ### Analyse Service
...@@ -248,8 +193,8 @@ numerical columns. ...@@ -248,8 +193,8 @@ numerical columns.
!!! debug "Debug Information" !!! debug "Debug Information"
* Port(s): 5000 * Port(s): 5000
* Swagger UI: [/swagger-ui/](http://dbrepo.ossdip.at:5000/swagger-ui/) * Docker Image: [dbrepo/analyse-service](https://hub.docker.com/repository/docker/dbrepo/analyse-service)
* Swagger API .json: [/api-analyze.json](http://dbrepo.ossdip.at:5000/api-analyze.json) * Swagger UI: [/swagger/analyse](/swagger/analyse)
## Database ## Database
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment