Skip to content
Snippets Groups Projects
Verified Commit bd42561a authored by Martin Weise's avatar Martin Weise
Browse files

Updated secure install

parent 8272fcd6
Branches
Tags
4 merge requests!310Dev,!308Dev,!307Dev,!304Dev
Showing with 133 additions and 87 deletions
# general # general
BASE_URL= BASE_URL=https://example.com
ADMIN_EMAIL=support@example.com ADMIN_EMAIL=support@example.com
# password for the identity service admin user # password for the identity service admin user
IDENTITY_SERVICE_ADMIN_PASSWORD=admin IDENTITY_SERVICE_ADMIN_PASSWORD=admin
...@@ -11,7 +11,6 @@ DATA_DB_PASSWORD=dbrepo ...@@ -11,7 +11,6 @@ DATA_DB_PASSWORD=dbrepo
AUTH_DB_PASSWORD=dbrepo AUTH_DB_PASSWORD=dbrepo
SEARCH_DB_PASSWORD=dbrepo SEARCH_DB_PASSWORD=dbrepo
# storage service # storage service
S3_ACCESS_KEY_ID=seaweedfsadmin
S3_SECRET_ACCESS_KEY=seaweedfsadmin S3_SECRET_ACCESS_KEY=seaweedfsadmin
# internal admin user, requires a change of the value of auth_ldap.dn_lookup_bind.password in dist/rabbitmq.conf # internal admin user, requires a change of the value of auth_ldap.dn_lookup_bind.password in dist/rabbitmq.conf
SYSTEM_PASSWORD=admin SYSTEM_PASSWORD=admin
.docs/images/screenshots/auth-service-ldap-1.png

138 KiB

.docs/images/screenshots/auth-service-ldap-2.png

249 KiB

...@@ -36,26 +36,55 @@ SSL/TLS certificate is recommended. Follow the [secure install](#secure-install) ...@@ -36,26 +36,55 @@ SSL/TLS certificate is recommended. Follow the [secure install](#secure-install)
## Secure Installation ## Secure Installation
1. Execute the install script to download only the environment and save it to `dist`. Execute the install script to download only the environment and save it to `dist`.
```shell ```shell
curl -sSL https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/raw/release-1.4.5/install.sh | DOWNLOAD_ONLY=1 bash curl -sSL https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/raw/release-1.4.5/install.sh | DOWNLOAD_ONLY=1 bash
``` ```
2. Call the helper script to regenerate the client secret of the `dbrepo-client` and set it as value of the ### Static Configuration
Call the helper script to regenerate the client secret of the `dbrepo-client` and set it as value of the
`AUTH_SERVICE_CLIENT_SECRET` variable in the `.env` file. `AUTH_SERVICE_CLIENT_SECRET` variable in the `.env` file.
```bash ```bash
curl -sSL "https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/raw/release-1.4.5/.scripts/reg-client-secret.sh" | bash curl -sSL "https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/raw/release-1.4.5/.scripts/reg-client-secret.sh" | bash
``` ```
3. Finally, update the rest of the default secrets in the `.env` file to secure passwords. You can use `openssl` for Update the rest of the default secrets in the `.env` file to secure passwords. You can use `openssl` for that, e.g.
that, e.g. `openssl rand -hex 16`. `openssl rand -hex 16`. Set `auth_ldap.dn_lookup_bind.password` in `dist/rabbitmq.conf` to the value of
`SYSTEM_PASSWORD`.
### Runtime Configuration
The [Auth Service](../api/auth-service) can be configured easily when DBRepo is running. Start DBRepo temporarily:
```shell
docker compose up -d
```
Log into the Auth Service with the default credentials `admin` and the value of `AUTH_SERVICE_ADMIN_PASSWORD`
(c.f. Figure 1) and select the "dbrepo" realm :material-numeric-1-circle-outline:. In the sidebar, select the
"User federation" :material-numeric-2-circle-outline: and from the provider list, select the "Identity Service" provider
:material-numeric-3-circle-outline:.
<figure markdown>
![](images/screenshots/auth-service-ldap-1.png){ .img-border }
<figcaption>Figure 1: Select the Identity Service provider.</figcaption>
</figure>
Set `auth_ldap.dn_lookup_bind.password` in `dist/rabbitmq.conf` to the value of `SYSTEM_PASSWORD`. If you plan to change the default admin username (c.f. Figure 2), modify the Bind DN :material-numeric-1-circle-outline:
but this is optional. Change the Bind credentials to the desired password :material-numeric-2-circle-outline: from
the variable `IDENTITY_SERVICE_ADMIN_PASSWORD` in `.env`.
4. To secure your deployment traffic with SSL/TLS, tell the Gateway Service to use your certificate secret (e.g. <figure markdown>
from Let's Encrypt): ![](images/screenshots/auth-service-ldap-2.png){ .img-border }
<figcaption>Figure 2: Update the Identity Service admin user credentials.</figcaption>
</figure>
Also, update the JWT key according to the
[Keycloak documentation](https://www.keycloak.org/docs/24.0.1/server_admin/index.html#rotating-keys). To secure your
deployment traffic with SSL/TLS, tell the Gateway Service to use your certificate secret (e.g. from Let's Encrypt):
```yaml title="docker-compose.yml" ```yaml title="docker-compose.yml"
services: services:
...@@ -86,6 +115,17 @@ SSL/TLS certificate is recommended. Follow the [secure install](#secure-install) ...@@ -86,6 +115,17 @@ SSL/TLS certificate is recommended. Follow the [secure install](#secure-install)
} }
``` ```
### Apply the Configuration
Restart the configured DBRepo system to apply the static and runtime configuration:
```shell
docker compose down
docker compose up -d
```
The secure installation is now finished!
## Troubleshooting ## Troubleshooting
In case the deployment is unsuccessful, we have explanations on their origin and solutions to the most common errors: In case the deployment is unsuccessful, we have explanations on their origin and solutions to the most common errors:
......
...@@ -8,13 +8,22 @@ fancy () { ...@@ -8,13 +8,22 @@ fancy () {
printf "This is a utility script to re-generate the client secret of the %s client.\n" $(fancy dbrepo-client) printf "This is a utility script to re-generate the client secret of the %s client.\n" $(fancy dbrepo-client)
fancy "Your credentials are never transmitted outside your machine!\n\n" fancy "Your credentials are never transmitted outside your machine!\n\n"
read -rp "Username: " USERNAME read -rp "Username: " USERNAME
read -rp "Password: " PASSWORD read -rsp "Password: " PASSWORD
# get admin token # get admin token
ADMIN_ACCESS_TOKEN=$(curl -fsSL -X POST -d "username=${USERNAME}&password=${PASSWORD}&grant_type=password&client_id=admin-cli" http://localhost/api/auth/realms/master/protocol/openid-connect/token | jq -r .access_token) ADMIN_ACCESS_TOKEN=$(curl -fsSL -X POST -d "username=${USERNAME}&password=${PASSWORD}&grant_type=password&client_id=admin-cli" http://localhost/api/auth/realms/master/protocol/openid-connect/token | jq -r .access_token)
printf "\nSuccessfully obtained admin token." if [ -z $ADMIN_ACCESS_TOKEN ]; then
printf "\n\nFailed to obtain admin token, credentials may not be correct."
exit 1
fi
printf "\n\nSuccessfully obtained admin token."
# re-generate client secret # re-generate client secret
SECRET=$(curl -fsSL -X POST -H "Authorization: Bearer ${ADMIN_ACCESS_TOKEN}" http://localhost/api/auth/admin/realms/dbrepo/clients/6b7ef364-4132-4831-b4e2-b6e9e9dc63ee/client-secret | jq -r .value) SECRET=$(curl -fsSL -X POST -H "Authorization: Bearer ${ADMIN_ACCESS_TOKEN}" http://localhost/api/auth/admin/realms/dbrepo/clients/6b7ef364-4132-4831-b4e2-b6e9e9dc63ee/client-secret | jq -r .value)
if [ -z $SECRET ]; then
printf "\n\nFailed to re-generate client secret."
exit 1
fi
printf "\nSuccessfully re-generated client secret: %s" $(fancy $SECRET) printf "\nSuccessfully re-generated client secret: %s" $(fancy $SECRET)
...@@ -178,7 +178,6 @@ template = { ...@@ -178,7 +178,6 @@ template = {
} }
swagger = Swagger(app, config=swagger_config, template=template) swagger = Swagger(app, config=swagger_config, template=template)
app.config["GATEWAY_SERVICE_ENDPOINT"] = os.getenv("GATEWAY_SERVICE_ENDPOINT", "http://localhost")
app.config["JWT_ALGORITHM"] = "HS256" app.config["JWT_ALGORITHM"] = "HS256"
app.config["JWT_PUBKEY"] = '-----BEGIN PUBLIC KEY-----\n' + os.getenv("JWT_PUBKEY", app.config["JWT_PUBKEY"] = '-----BEGIN PUBLIC KEY-----\n' + os.getenv("JWT_PUBKEY",
"MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB") + '\n-----END PUBLIC KEY-----' "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB") + '\n-----END PUBLIC KEY-----'
......
...@@ -2174,7 +2174,7 @@ ...@@ -2174,7 +2174,7 @@
} ], } ],
"org.keycloak.storage.UserStorageProvider" : [ { "org.keycloak.storage.UserStorageProvider" : [ {
"id" : "c109d473-5ce1-4032-af7b-02e5442f5c07", "id" : "c109d473-5ce1-4032-af7b-02e5442f5c07",
"name" : "openldap", "name" : "Identity Service",
"providerId" : "ldap", "providerId" : "ldap",
"subComponents" : { "subComponents" : {
"org.keycloak.storage.ldap.mappers.LDAPStorageMapper" : [ { "org.keycloak.storage.ldap.mappers.LDAPStorageMapper" : [ {
......
...@@ -50,9 +50,9 @@ logging: ...@@ -50,9 +50,9 @@ logging:
org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver: debug org.springframework.web.servlet.mvc.method.annotation.ExceptionHandlerExceptionResolver: debug
dbrepo: dbrepo:
endpoints: endpoints:
metadataService: "${METADATA_SERVICE_ENDPOINT:http://gateway-service}" metadataService: "${METADATA_SERVICE_ENDPOINT:http://metadata-service:8080}"
storageService: "${S3_ENDPOINT:http://gateway-service/api/storage}" storageService: "${S3_ENDPOINT:http://storage-service:9000}"
authService: "${AUTH_SERVICE_ENDPOINT:http://gateway-service/api/auth}" authService: "${AUTH_SERVICE_ENDPOINT:http://auth-service:8080}"
s3: s3:
accessKeyId: "${S3_ACCESS_KEY_ID:seaweedfsadmin}" accessKeyId: "${S3_ACCESS_KEY_ID:seaweedfsadmin}"
secretAccessKey: "${S3_SECRET_ACCESS_KEY:seaweedfsadmin}" secretAccessKey: "${S3_SECRET_ACCESS_KEY:seaweedfsadmin}"
......
...@@ -3,7 +3,7 @@ application: ...@@ -3,7 +3,7 @@ application:
version: '@project.version@' version: '@project.version@'
spring: spring:
datasource: datasource:
url: "jdbc:mariadb://${METADATA_HOST:metadata-db}:3306/${METADATA_DB:dbrepo}${METADATA_JDBC_EXTRA_ARGS}" url: "jdbc:mariadb://${METADATA_HOST:metadata-db}:${METADATA_PORT:3306}/${METADATA_DB:dbrepo}${METADATA_JDBC_EXTRA_ARGS}"
driver-class-name: org.mariadb.jdbc.Driver driver-class-name: org.mariadb.jdbc.Driver
username: "${METADATA_USERNAME:root}" username: "${METADATA_USERNAME:root}"
password: "${METADATA_DB_PASSWORD:dbrepo}" password: "${METADATA_DB_PASSWORD:dbrepo}"
...@@ -65,16 +65,16 @@ dbrepo: ...@@ -65,16 +65,16 @@ dbrepo:
username: "${SYSTEM_USERNAME:admin}" username: "${SYSTEM_USERNAME:admin}"
password: "${SYSTEM_PASSWORD:admin}" password: "${SYSTEM_PASSWORD:admin}"
endpoints: endpoints:
searchService: "${SEARCH_SERVICE_ENDPOINT:http://gateway-service}" analyseService: "${ANALYSE_SERVICE_ENDPOINT:http://analyse-service:8080}"
analyseService: "${ANALYSE_SERVICE_ENDPOINT:http://gateway-service}" searchService: "${SEARCH_SERVICE_ENDPOINT:http://search-service:8080}"
dataService: "${DATA_SERVICE_ENDPOINT:http://data-service:8080}" dataService: "${DATA_SERVICE_ENDPOINT:http://data-service:8080}"
brokerService: "${BROKER_SERVICE_ENDPOINT:http://gateway-service/admin/broker}" brokerService: "${BROKER_SERVICE_ENDPOINT:http://broker-service:15672}"
authService: "${AUTH_SERVICE_ENDPOINT:http://gateway-service/api/auth}" authService: "${AUTH_SERVICE_ENDPOINT:http://auth-service:8080}"
storageService: "${S3_ENDPOINT:http://gateway-service/api/storage}" storageService: "${S3_ENDPOINT:http://storage-service:9000}"
rorService: "${ROR_ENDPOINT:https://api.ror.org}" rorService: "${ROR_ENDPOINT:https://api.ror.org}"
crossRefService: "${CROSSREF_ENDPOINT:http://data.crossref.org}" crossRefService: "${CROSSREF_ENDPOINT:http://data.crossref.org}"
pid: pid:
base: "${PID_BASE:http://localhost/pid/}" base: "${BASE_URL:http://localhost}/pid/"
jwt: jwt:
public_key: "${JWT_PUBKEY:MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB}" public_key: "${JWT_PUBKEY:MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB}"
keycloak: keycloak:
......
...@@ -192,14 +192,14 @@ template = { ...@@ -192,14 +192,14 @@ template = {
} }
swagger = Swagger(app, config=swagger_config, template=template) swagger = Swagger(app, config=swagger_config, template=template)
app.config["GATEWAY_SERVICE_ENDPOINT"] = os.getenv("GATEWAY_SERVICE_ENDPOINT", "http://localhost") app.config["METADATA_SERVICE_ENDPOINT"] = os.getenv("METADATA_SERVICE_ENDPOINT", "http://metadata-service:8080")
app.config["JWT_ALGORITHM"] = "HS256" app.config["JWT_ALGORITHM"] = "HS256"
app.config["JWT_PUBKEY"] = '-----BEGIN PUBLIC KEY-----\n' + os.getenv("JWT_PUBKEY", app.config["JWT_PUBKEY"] = '-----BEGIN PUBLIC KEY-----\n' + os.getenv("JWT_PUBKEY",
"MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB") + '\n-----END PUBLIC KEY-----' "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB") + '\n-----END PUBLIC KEY-----'
app.config["AUTH_SERVICE_ENDPOINT"] = os.getenv("AUTH_SERVICE_ENDPOINT", "http://localhost/api/auth") app.config["AUTH_SERVICE_ENDPOINT"] = os.getenv("AUTH_SERVICE_ENDPOINT", "http://auth-service:8080/api/auth")
app.config["AUTH_SERVICE_CLIENT"] = os.getenv("AUTH_SERVICE_CLIENT", "dbrepo-client") app.config["AUTH_SERVICE_CLIENT"] = os.getenv("AUTH_SERVICE_CLIENT", "dbrepo-client")
app.config["AUTH_SERVICE_CLIENT_SECRET"] = os.getenv("AUTH_SERVICE_CLIENT_SECRET", "MUwRc7yfXSJwX8AdRMWaQC3Nep1VjwgG") app.config["AUTH_SERVICE_CLIENT_SECRET"] = os.getenv("AUTH_SERVICE_CLIENT_SECRET", "MUwRc7yfXSJwX8AdRMWaQC3Nep1VjwgG")
app.config["OPENSEARCH_HOST"] = os.getenv('OPENSEARCH_HOST', 'localhost') app.config["OPENSEARCH_HOST"] = os.getenv('OPENSEARCH_HOST', 'search-db')
app.config["OPENSEARCH_PORT"] = os.getenv('OPENSEARCH_PORT', '9200') app.config["OPENSEARCH_PORT"] = os.getenv('OPENSEARCH_PORT', '9200')
app.config["OPENSEARCH_USERNAME"] = os.getenv('OPENSEARCH_USERNAME', 'admin') app.config["OPENSEARCH_USERNAME"] = os.getenv('OPENSEARCH_USERNAME', 'admin')
app.config["OPENSEARCH_PASSWORD"] = os.getenv('OPENSEARCH_PASSWORD', 'admin') app.config["OPENSEARCH_PASSWORD"] = os.getenv('OPENSEARCH_PASSWORD', 'admin')
......
...@@ -3,7 +3,6 @@ The opensearch_client.py is used by the different API endpoints in routes.py to ...@@ -3,7 +3,6 @@ The opensearch_client.py is used by the different API endpoints in routes.py to
""" """
from json import dumps, load from json import dumps, load
import logging import logging
import re
from dbrepo.api.dto import Database from dbrepo.api.dto import Database
from flask import current_app from flask import current_app
...@@ -12,7 +11,7 @@ from collections.abc import MutableMapping ...@@ -12,7 +11,7 @@ from collections.abc import MutableMapping
from opensearchpy import OpenSearch, TransportError, RequestError from opensearchpy import OpenSearch, TransportError, RequestError
from omlib.measure import om from omlib.measure import om
from omlib.constants import SI, OM_IDS from omlib.constants import OM_IDS
from omlib.omconstants import OM from omlib.omconstants import OM
from omlib.unit import Unit from omlib.unit import Unit
......
...@@ -2,6 +2,7 @@ import json ...@@ -2,6 +2,7 @@ import json
import os import os
import logging import logging
from typing import List from typing import List
from flask import current_app
import opensearchpy.exceptions import opensearchpy.exceptions
from dbrepo.RestClient import RestClient from dbrepo.RestClient import RestClient
...@@ -40,7 +41,7 @@ class App: ...@@ -40,7 +41,7 @@ class App:
""" """
The client to communicate with the OpenSearch database. The client to communicate with the OpenSearch database.
""" """
gateway_endpoint: str = None metadata_service_endpoint: str = None
search_host: str = None search_host: str = None
search_port: int = None search_port: int = None
search_username: str = None search_username: str = None
...@@ -48,11 +49,11 @@ class App: ...@@ -48,11 +49,11 @@ class App:
search_instance: OpenSearch = None search_instance: OpenSearch = None
def __init__(self): def __init__(self):
self.gateway_endpoint = os.getenv("GATEWAY_SERVICE_ENDPOINT", "http://localhost") self.metadata_service_endpoint = current_app.config["METADATA_SERVICE_ENDPOINT"]
self.search_host = os.getenv("OPENSEARCH_HOST", "localhost") self.search_host = current_app.config["OPENSEARCH_HOST"]
self.search_port = int(os.getenv("OPENSEARCH_PORT", "9200")) self.search_port = int(current_app.config["OPENSEARCH_PORT"])
self.search_username = os.getenv("OPENSEARCH_USERNAME", "admin") self.search_username = current_app.config["OPENSEARCH_USERNAME"]
self.search_password = os.getenv("OPENSEARCH_PASSWORD", "admin") self.search_password = current_app.config["OPENSEARCH_PASSWORD"]
def _instance(self) -> OpenSearch: def _instance(self) -> OpenSearch:
""" """
...@@ -101,7 +102,7 @@ class App: ...@@ -101,7 +102,7 @@ class App:
return True return True
def fetch_databases(self) -> List[Database]: def fetch_databases(self) -> List[Database]:
client = RestClient(endpoint=self.gateway_endpoint) client = RestClient(endpoint=self.metadata_service_endpoint)
databases = [] databases = []
for database in client.get_databases(): for database in client.get_databases():
databases.append(client.get_database(database_id=database.id)) databases.append(client.get_database(database_id=database.id))
......
...@@ -116,19 +116,19 @@ services: ...@@ -116,19 +116,19 @@ services:
- "${SHARED_VOLUME:-/tmp}:/tmp" - "${SHARED_VOLUME:-/tmp}:/tmp"
environment: environment:
ADMIN_EMAIL: "${ADMIN_EMAIL:-noreply@localhost}" ADMIN_EMAIL: "${ADMIN_EMAIL:-noreply@localhost}"
ANALYSE_SERVICE_ENDPOINT: "${ANALYSE_SERVICE_ENDPOINT:-http://gateway-service}" ANALYSE_SERVICE_ENDPOINT: "${ANALYSE_SERVICE_ENDPOINT:-http://analyse-service:8080}"
AUTH_SERVICE_ADMIN: ${AUTH_SERVICE_ADMIN:-admin} AUTH_SERVICE_ADMIN: ${AUTH_SERVICE_ADMIN:-admin}
AUTH_SERVICE_ADMIN_PASSWORD: ${AUTH_SERVICE_ADMIN_PASSWORD:-admin} AUTH_SERVICE_ADMIN_PASSWORD: ${AUTH_SERVICE_ADMIN_PASSWORD:-admin}
AUTH_SERVICE_CLIENT: ${AUTH_SERVICE_CLIENT:-dbrepo-client} AUTH_SERVICE_CLIENT: ${AUTH_SERVICE_CLIENT:-dbrepo-client}
AUTH_SERVICE_CLIENT_SECRET: ${AUTH_SERVICE_CLIENT_SECRET:-MUwRc7yfXSJwX8AdRMWaQC3Nep1VjwgG} AUTH_SERVICE_CLIENT_SECRET: ${AUTH_SERVICE_CLIENT_SECRET:-MUwRc7yfXSJwX8AdRMWaQC3Nep1VjwgG}
AUTH_SERVICE_ENDPOINT: ${AUTH_SERVICE_ENDPOINT:-http://gateway-service/api/auth} AUTH_SERVICE_ENDPOINT: ${AUTH_SERVICE_ENDPOINT:-http://auth-service:8080}
BASE_URL: "${BASE_URL:-http://localhost}" BASE_URL: "${BASE_URL:-http://localhost}"
BROKER_EXCHANGE_NAME: ${BROKER_EXCHANGE_NAME:-dbrepo} BROKER_EXCHANGE_NAME: ${BROKER_EXCHANGE_NAME:-dbrepo}
BROKER_QUEUE_NAME: ${BROKER_QUEUE_NAME:-dbrepo} BROKER_QUEUE_NAME: ${BROKER_QUEUE_NAME:-dbrepo}
BROKER_HOST: "${BROKER_ENDPOINT:-broker-service}" BROKER_HOST: "${BROKER_ENDPOINT:-broker-service}"
BROKER_PASSWORD: ${BROKER_PASSWORD:-admin} BROKER_PASSWORD: ${BROKER_PASSWORD:-admin}
BROKER_PORT: ${BROKER_PORT:-5672} BROKER_PORT: ${BROKER_PORT:-5672}
BROKER_SERVICE_ENDPOINT: ${BROKER_SERVICE_ENDPOINT:-http://gateway-service/admin/broker} BROKER_SERVICE_ENDPOINT: ${BROKER_SERVICE_ENDPOINT:-http://broker-service:15672}
BROKER_USERNAME: ${BROKER_USERNAME:-admin} BROKER_USERNAME: ${BROKER_USERNAME:-admin}
BROKER_VIRTUALHOST: "${BROKER_VIRTUALHOST:-dbrepo}" BROKER_VIRTUALHOST: "${BROKER_VIRTUALHOST:-dbrepo}"
CROSSREF_ENDPOINT: "${CROSSREF_ENDPOINT:-http://data.crossref.org}" CROSSREF_ENDPOINT: "${CROSSREF_ENDPOINT:-http://data.crossref.org}"
...@@ -138,14 +138,14 @@ services: ...@@ -138,14 +138,14 @@ services:
JWT_PUBKEY: "${JWT_PUBKEY:-MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB}" JWT_PUBKEY: "${JWT_PUBKEY:-MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB}"
LOG_LEVEL: ${LOG_LEVEL:-info} LOG_LEVEL: ${LOG_LEVEL:-info}
METADATA_DB: "${METADATA_DB:-dbrepo}" METADATA_DB: "${METADATA_DB:-dbrepo}"
METADATA_DB_PASSWORD: "${METADATA_DB_PASSWORD:-dbrepo}"
METADATA_HOST: "${METADATA_HOST:-metadata-db}" METADATA_HOST: "${METADATA_HOST:-metadata-db}"
METADATA_JDBC_EXTRA_ARGS: "${METADATA_JDBC_EXTRA_ARGS:-}" METADATA_JDBC_EXTRA_ARGS: "${METADATA_JDBC_EXTRA_ARGS:-}"
METADATA_PORT: "${METADATA_PORT:-3306}"
METADATA_USERNAME: root METADATA_USERNAME: root
METADATA_DB_PASSWORD: "${METADATA_DB_PASSWORD:-dbrepo}"
PID_BASE: ${PID_BASE:-http://localhost/pid/}
REPOSITORY_NAME: "${REPOSITORY_NAME:-Database Repository}" REPOSITORY_NAME: "${REPOSITORY_NAME:-Database Repository}"
ROR_ENDPOINT: "${ROR_ENDPOINT:-https://api.ror.org}" ROR_ENDPOINT: "${ROR_ENDPOINT:-https://api.ror.org}"
SEARCH_SERVICE_ENDPOINT: "${SEARCH_SERVICE_ENDPOINT:-http://gateway-service}" SEARCH_SERVICE_ENDPOINT: "${SEARCH_SERVICE_ENDPOINT:-http://search-service:8080}"
S3_ACCESS_KEY_ID: "${S3_ACCESS_KEY_ID:-seaweedfsadmin}" S3_ACCESS_KEY_ID: "${S3_ACCESS_KEY_ID:-seaweedfsadmin}"
S3_BUCKET: "${S3_BUCKET:-dbrepo}" S3_BUCKET: "${S3_BUCKET:-dbrepo}"
S3_ENDPOINT: "${S3_ENDPOINT:-http://storage-service:9000}" S3_ENDPOINT: "${S3_ENDPOINT:-http://storage-service:9000}"
...@@ -184,7 +184,6 @@ services: ...@@ -184,7 +184,6 @@ services:
AUTH_SERVICE_CLIENT: ${AUTH_SERVICE_CLIENT:-dbrepo-client} AUTH_SERVICE_CLIENT: ${AUTH_SERVICE_CLIENT:-dbrepo-client}
AUTH_SERVICE_CLIENT_SECRET: ${AUTH_SERVICE_CLIENT:-MUwRc7yfXSJwX8AdRMWaQC3Nep1VjwgG} AUTH_SERVICE_CLIENT_SECRET: ${AUTH_SERVICE_CLIENT:-MUwRc7yfXSJwX8AdRMWaQC3Nep1VjwgG}
AUTH_SERVICE_ENDPOINT: ${AUTH_SERVICE_ENDPOINT:-http://auth-service:8080} AUTH_SERVICE_ENDPOINT: ${AUTH_SERVICE_ENDPOINT:-http://auth-service:8080}
GATEWAY_SERVICE_ENDPOINT: ${GATEWAY_SERVICE_ENDPOINT:-http://gateway-service}
JWT_PUBKEY: "${JWT_PUBKEY:-MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB}" JWT_PUBKEY: "${JWT_PUBKEY:-MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB}"
S3_ACCESS_KEY_ID: "${S3_ACCESS_KEY_ID:-seaweedfsadmin}" S3_ACCESS_KEY_ID: "${S3_ACCESS_KEY_ID:-seaweedfsadmin}"
S3_BUCKET: "${S3_BUCKET:-dbrepo}" S3_BUCKET: "${S3_BUCKET:-dbrepo}"
...@@ -269,7 +268,7 @@ services: ...@@ -269,7 +268,7 @@ services:
AUTH_SERVICE_CLIENT_SECRET: ${AUTH_SERVICE_CLIENT_SECRET:-MUwRc7yfXSJwX8AdRMWaQC3Nep1VjwgG} AUTH_SERVICE_CLIENT_SECRET: ${AUTH_SERVICE_CLIENT_SECRET:-MUwRc7yfXSJwX8AdRMWaQC3Nep1VjwgG}
AUTH_SERVICE_ENDPOINT: ${AUTH_SERVICE_ENDPOINT:-http://auth-service:8080} AUTH_SERVICE_ENDPOINT: ${AUTH_SERVICE_ENDPOINT:-http://auth-service:8080}
COLLECTION: ${COLLECTION:-['database','table','column','identifier','unit','concept','user','view']} COLLECTION: ${COLLECTION:-['database','table','column','identifier','unit','concept','user','view']}
GATEWAY_SERVICE_ENDPOINT: ${GATEWAY_SERVICE_ENDPOINT:-http://gateway-service} METADATA_SERVICE_ENDPOINT: ${METADATA_SERVICE_ENDPOINT:-http://metadata-service:8080}
OPENSEARCH_HOST: ${OPENSEARCH_HOST:-search-db} OPENSEARCH_HOST: ${OPENSEARCH_HOST:-search-db}
OPENSEARCH_PORT: ${OPENSEARCH_PORT:-9200} OPENSEARCH_PORT: ${OPENSEARCH_PORT:-9200}
OPENSEARCH_USERNAME: ${SEARCH_DB_USERNAME:-admin} OPENSEARCH_USERNAME: ${SEARCH_DB_USERNAME:-admin}
...@@ -402,7 +401,7 @@ services: ...@@ -402,7 +401,7 @@ services:
context: ./dbrepo-search-service/init context: ./dbrepo-search-service/init
network: host network: host
environment: environment:
GATEWAY_SERVICE_ENDPOINT: ${GATEWAY_SERVICE_ENDPOINT:-http://gateway-service} METADATA_SERVICE_ENDPOINT: ${METADATA_SERVICE_ENDPOINT:-http://metadata-service:8080}
OPENSEARCH_HOST: ${OPENSEARCH_HOST:-search-db} OPENSEARCH_HOST: ${OPENSEARCH_HOST:-search-db}
OPENSEARCH_PORT: ${OPENSEARCH_PORT:-9200} OPENSEARCH_PORT: ${OPENSEARCH_PORT:-9200}
OPENSEARCH_USERNAME: ${SEARCH_DB_USERNAME:-admin} OPENSEARCH_USERNAME: ${SEARCH_DB_USERNAME:-admin}
...@@ -504,7 +503,7 @@ services: ...@@ -504,7 +503,7 @@ services:
BROKER_VIRTUALHOST: "${BROKER_VIRTUALHOST:-dbrepo}" BROKER_VIRTUALHOST: "${BROKER_VIRTUALHOST:-dbrepo}"
CONNECTION_TIMEOUT: ${CONNECTION_TIMEOUT:-60000} CONNECTION_TIMEOUT: ${CONNECTION_TIMEOUT:-60000}
EXCHANGE_NAME: ${EXCHANGE_NAME:-dbrepo} EXCHANGE_NAME: ${EXCHANGE_NAME:-dbrepo}
METADATA_SERVICE_ENDPOINT: ${METADATA_SERVICE_ENDPOINT:-http://gateway-service} METADATA_SERVICE_ENDPOINT: ${METADATA_SERVICE_ENDPOINT:-http://metadata-service:8080}
GRANT_DEFAULT_READ: "${GRANT_DEFAULT_READ:-SELECT}" GRANT_DEFAULT_READ: "${GRANT_DEFAULT_READ:-SELECT}"
GRANT_DEFAULT_WRITE: "${GRANT_DEFAULT_WRITE:-SELECT, CREATE, CREATE VIEW, CREATE ROUTINE, CREATE TEMPORARY TABLES, LOCK TABLES, INDEX, TRIGGER, INSERT, UPDATE, DELETE}" GRANT_DEFAULT_WRITE: "${GRANT_DEFAULT_WRITE:-SELECT, CREATE, CREATE VIEW, CREATE ROUTINE, CREATE TEMPORARY TABLES, LOCK TABLES, INDEX, TRIGGER, INSERT, UPDATE, DELETE}"
JWT_PUBKEY: "${JWT_PUBKEY:-MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB}" JWT_PUBKEY: "${JWT_PUBKEY:-MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAqqnHQ2BWWW9vDNLRCcxD++xZg/16oqMo/c1l+lcFEjjAIJjJp/HqrPYU/U9GvquGE6PbVFtTzW1KcKawOW+FJNOA3CGo8Q1TFEfz43B8rZpKsFbJKvQGVv1Z4HaKPvLUm7iMm8Hv91cLduuoWx6Q3DPe2vg13GKKEZe7UFghF+0T9u8EKzA/XqQ0OiICmsmYPbwvf9N3bCKsB/Y10EYmZRb8IhCoV9mmO5TxgWgiuNeCTtNCv2ePYqL/U0WvyGFW0reasIK8eg3KrAUj8DpyOgPOVBn3lBGf+3KFSYi+0bwZbJZWqbC/Xlk20Go1YfeJPRIt7ImxD27R/lNjgDO/MwIDAQAB}"
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment