Updated changelog

.docker/.env

-APP_VERSION=1.9.2
+APP_VERSION=1.9.3
 FLUENTBIT_VERSION=4.0.0
 GRAFANA_VERSION=11.4.0
 MARIADB_VERSION=11.3.2

.docker/docker-compose.yml

@@ -158,7 +158,8 @@ services:
       KEYCLOAK_DATABASE_USER: "${AUTH_DB_USERNAME:-keycloak}"
       KEYCLOAK_DATABASE_PASSWORD: "${AUTH_DB_PASSWORD:-dbrepo}"
       KEYCLOAK_HOSTNAME: "${BASE_URL:-http://localhost}"
-      KEYCLOAK_HOSTNAME_ADMIN: "${BASE_URL:-http://localhost}:8080"
+      KEYCLOAK_HOSTNAME_ADMIN: "${BASE_URL:-http://localhost}/admin/"
+      KEYCLOAK_PROXY_HEADERS: "xforwarded"
       METADATA_SERVICE_ENDPOINT: "${METADATA_SERVICE_ENDPOINT:-http://metadata-service:8080}"
       SYSTEM_USERNAME: "${SYSTEM_USERNAME:-admin}"
       SYSTEM_PASSWORD: "${SYSTEM_PASSWORD:-admin}"

.docs/api/open-api.md

----
-author: Martin Weise
----
-
-![OpenAPI 3.1](https://img.shields.io/badge/OpenAPI-3.1-leaf)
-
-## tl;dr
-
-[:simple-swagger:  View Swagger-UI](../../rest/){ .md-button .md-button--primary tabindex=-1 }
-
-## Overview
-
-All services are documented using the [OpenAPI 3.1](https://www.openapis.org/) documentation standard.
\ No newline at end of file

.docs/api/python.md

----
-author: Martin Weise
----
-
-[![PyPI - Version](https://img.shields.io/pypi/v/dbrepo)](https://pypi.org/project/dbrepo/){ tabindex=-1 }
-
-## tl;dr
-
-[:fontawesome-solid-cube:  View Docs](../../python){ .md-button .md-button--primary }
-
-## Overview
-
-The DBRepo Python library is using some of the most pupular and maintained Python packages for Data Scientists under the
-hood. For example: [`requests`](https://requests.readthedocs.io/) to interact with the HTTP API
-endpoints, [`pandas`](https://pandas.pydata.org/) for data operations and [`pydantic`](https://docs.pydantic.dev/) for
-information representation from/to the HTTP API.
-
-## Installing
-
-:octicons-tag-16:{ title="Minimum version" } 1.4.2
-
-```console
-$ python -m pip install dbrepo
-```
-
-To use DBRepo in your Jupyter notebook, install the `dbrepo` library` directly in a code cell and type:
-
-```jupyter
-!pip install dbrepo
-```
-
-This package supports Python 3.11+.
-
-## Quickstart
-
-Get public data from a table as pandas `DataFrame`:
-
-```python
-from dbrepo.RestClient import RestClient
-
-client = RestClient(endpoint="https://dbrepo1.ec.tuwien.ac.at")
-# Get a small data slice of just three rows
-df = client.get_table_data(database_id=7, table_id=13, page=0, size=3, df=True)
-print(df)
-#     x_coord         component   unit  ... value stationid meantype
-# 0  16.52617  Feinstaub (PM10)  µg/m³  ...  21.0   01:0001      HMW
-# 1  16.52617  Feinstaub (PM10)  µg/m³  ...  23.0   01:0001      HMW
-# 2  16.52617  Feinstaub (PM10)  µg/m³  ...  26.0   01:0001      HMW
-#
-# [3 rows x 12 columns]
-```
-
-Import data into a table:
-
-```python
-import pandas as pd
-from dbrepo.RestClient import RestClient
-
-client = RestClient(endpoint="https://dbrepo1.ec.tuwien.ac.at", username="foo",
-                    password="bar")
-df = pd.DataFrame(data={'x_coord': 16.52617, 'component': 'Feinstaub (PM10)',
-                        'unit': 'µg/m³', ...})
-client.import_table_data(database_id=7, table_id=13, file_name_or_data_frame=df)
-```
-
-## Supported Features & Best-Practices
-
-- Manage user
-  account ([docs](https://www.ifs.tuwien.ac.at/infrastructures/dbrepo/1.4.5/api/#create-user-account))
-- Manage
-  databases ([docs](https://www.ifs.tuwien.ac.at/infrastructures/dbrepo//usage-overview/#create-database))
-- Manage database access &
-  visibility ([docs](https://www.ifs.tuwien.ac.at/infrastructures/dbrepo/1.4.5/api/#create-database))
-- Import
-  dataset ([docs](https://www.ifs.tuwien.ac.at/infrastructures/dbrepo/1.4.5/api/#import-dataset))
-- Create persistent
-  identifiers ([docs](https://www.ifs.tuwien.ac.at/infrastructures/dbrepo/1.4.5/api/#assign-database-pid))
-- Execute
-  queries ([docs](https://www.ifs.tuwien.ac.at/infrastructures/dbrepo/1.4.5/api/#export-subset))
-- Get data from tables/views/subsets
-
-## Configure
-
-All credentials can optionally be set/overridden with environment variables. This is especially useful when sharing 
-Jupyter Notebooks by creating an invisible `.env` file and loading it:
-
-``` title=".env"
-REST_API_ENDPOINT="https://dbrepo1.ec.tuwien.ac.at"
-REST_API_USERNAME="foo"
-REST_API_PASSWORD="bar"
-REST_API_SECURE="True"
-AMQP_API_HOST="https://dbrepo1.ec.tuwien.ac.at"
-AMQP_API_PORT="5672"
-AMQP_API_USERNAME="foo"
-AMQP_API_PASSWORD="bar"
-AMQP_API_VIRTUAL_HOST="dbrepo"
-REST_UPLOAD_ENDPOINT="https://dbrepo1.ec.tuwien.ac.at/api/upload/files"
-```
-
-You can disable logging by setting the log level to e.g. `INFO`:
-
-```python
-from dbrepo.RestClient import RestClient
-import logging
-logging.getLogger().setLevel(logging.INFO)
-...
-client = RestClient(...)
-```
-
-## Future
-
-- Searching
-
-## Links
-
-This information is also mirrored on [PyPI](https://pypi.org/project/dbrepo/).
\ No newline at end of file

.docs/examples/manufacturing.md

----
-author: Martin Weise
----
-
-## tl;dr
-
-tbd
-
-## Description
-
-TBD
-
-## Solution
-
-TBD
-
-## DBRepo Features
-
-- [x] TBD

.env

-APP_VERSION=1.9.2
+APP_VERSION=1.9.3
 FLUENTBIT_VERSION=4.0.0
 GRAFANA_VERSION=11.4.0
 MARIADB_VERSION=11.3.2

.gitignore

@@ -75,7 +75,6 @@ venv/
 # Docs
 swagger.json
 container.md
-database.md
 query.md
 table.md
 .git.bfg-report/

.gitlab-ci.yml

@@ -12,9 +12,9 @@ variables:
   REGCLIENT_VERSION: "0.8.2"
   BUN_VERSION: "1.1.40"
   DOC_VERSION: "1.9"
-  APP_VERSION: "1.9.2"
-  CHART_VERSION: "1.9.2"
-  SUPPORTED_VERSIONS: "1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2"
+  APP_VERSION: "1.9.3"
+  CHART_VERSION: "1.9.3"
+  SUPPORTED_VERSIONS: "1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3"
   SUPPORTED_ARCH: "linux/amd64"
   MAINTAINED_SERVICES: "analyse-service, auth-service-init, dashboard-service, dashboard-service-init, data-service, metadata-service, search-service, search-service-init, storage-service-init, ui"
   CACHE_FALLBACK_KEY: "${CI_DEFAULT_BRANCH}"
@@ -149,9 +149,9 @@ lint-open-api-version:
     - 'apk add --no-cache bash wget'
     - 'wget https://github.com/mikefarah/yq/releases/download/v${VERSION}/${BINARY} -O /usr/bin/yq && chmod +x /usr/bin/yq'
   script:
-    - yq '.externalDocs.url' ./.docs/.openapi/api.base.yaml | grep -o "${DOC_VERSION}" > ./openapi-doc-version.txt
+    - yq '.externalDocs.url' ./docs/.openapi/api.base.yaml | grep -o "${DOC_VERSION}" > ./openapi-doc-version.txt
     - diff ./openapi-doc-version.txt ./doc-version.txt
-    - yq '.info.version' ./.docs/.openapi/api.base.yaml | grep -o "${APP_VERSION}" > ./openapi-app-version.txt
+    - yq '.info.version' ./docs/.openapi/api.base.yaml | grep -o "${APP_VERSION}" > ./openapi-app-version.txt
     - diff ./openapi-app-version.txt ./app-version.txt
 
 build-metadata-service:
@@ -509,7 +509,7 @@ test-storage-service-init:
   before_script:
     - "apk --no-cache add s3cmd bash libxml2-utils"
     - "docker compose build dbrepo-storage-service-init"
-    - "mkdir -p /app/"
+    - "mkdir -p /app/config /var/log/app/service/storage"
     - "echo ${CI_S3_CONFIG} | base64 -d > /app/.s3cfg"
   script:
     - "bash ./dbrepo-storage-service/init/tests/test_integration_expire.sh"
@@ -575,6 +575,7 @@ release-images:
   script:
     - "make release-images"
     - "APP_VERSION=$DOC_VERSION make release-images"
+    - "make release-jupyter-image"
 
 release-helm:
   stage: release
@@ -617,12 +618,12 @@ release-docs:
   script:
     - "make gen-lib-doc gen-docs-doc package-config"
     - "cp -r ./lib/python/docs/build/html ./final/${DOC_VERSION}/python" # sphinx
-    - "cp .docs/.openapi/api.yaml ./final/${DOC_VERSION}/rest/api.yaml" # openapi
-    - "cp .docs/.openapi/swagger-ui.html ./final/${DOC_VERSION}/rest/index.html" # openapi
-    - "cp .docs/.openapi/custom.css ./final/${DOC_VERSION}/rest/custom.css" # openapi
+    - "cp ./docs/.openapi/api.yaml ./final/${DOC_VERSION}/rest/api.yaml" # openapi
+    - "cp ./docs/.openapi/swagger-ui.html ./final/${DOC_VERSION}/rest/index.html" # openapi
+    - "cp ./docs/.openapi/custom.css ./final/${DOC_VERSION}/rest/custom.css" # openapi
     - "cp -r ./site/* ./final/${DOC_VERSION}" # mkdocs
     - "cp .docker/dist.tar.gz ./final/${APP_VERSION}/dist.tar.gz" # dist
-    - "cp .docs/index.html.tpl ./final/index.html" # redirect patch docs
+    - "cp ./docs/index.html.tpl ./final/index.html" # redirect patch docs
     - sed -i "s/DOC_VERSION/$DOC_VERSION/g" ./final/index.html
     - "bash ./.gitlab/gen-badge.sh"
     - eval $(ssh-agent -s)
@@ -633,7 +634,7 @@ release-docs:
     - tar czf ./final.tar.gz ./final
     - "scp -oHostKeyAlgorithms=+ssh-rsa -oPubkeyAcceptedAlgorithms=+ssh-rsa final.tar.gz $CI_DOC_USER@$CI_DOC_IP:final.tar.gz"
     - "scp -oHostKeyAlgorithms=+ssh-rsa -oPubkeyAcceptedAlgorithms=+ssh-rsa versions.json $CI_DOC_USER@$CI_DOC_IP:/system/user/ifs/infrastructures/public_html/dbrepo/versions.json"
-    - "scp -oHostKeyAlgorithms=+ssh-rsa -oPubkeyAcceptedAlgorithms=+ssh-rsa .docs/index.html.tpl $CI_DOC_USER@$CI_DOC_IP:/system/user/ifs/infrastructures/public_html/dbrepo/index.html"
+    - "scp -oHostKeyAlgorithms=+ssh-rsa -oPubkeyAcceptedAlgorithms=+ssh-rsa ./docs/index.html.tpl $CI_DOC_USER@$CI_DOC_IP:/system/user/ifs/infrastructures/public_html/dbrepo/index.html"
     - 'ssh -oHostKeyAlgorithms=+ssh-rsa -oPubkeyAcceptedAlgorithms=+ssh-rsa $CI_DOC_USER@$CI_DOC_IP "rm -rf /system/user/ifs/infrastructures/public_html/dbrepo/${DOC_VERSION}; tar xzf ./final.tar.gz; rm -f ./final.tar.gz; cp -r ./final/* /system/user/ifs/infrastructures/public_html/dbrepo; rm -rf ./final"'
 
 release-libs:

.jupyter/.env

-# Set your credentials here
-REST_API_ENDPOINT="https://dbrepo.example.com"
-REST_API_USERNAME="foo"
-REST_API_PASSWORD="bar"
\ No newline at end of file

.jupyter/.ipynb_checkpoints/starter-checkpoint.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "3ca0c15a13bd6882",
+   "metadata": {},
+   "source": [
+    "# DBRepo Starter Notebook\n",
+    "\n",
+    "## Environment\n",
+    "\n",
+    "Set the REST API endpoint and your username + password in the `.env` file. This ensures it is not revealed in the code."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "4eb6c2470f464173",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2025-06-04T07:08:17.369993Z",
+     "start_time": "2025-06-04T07:08:17.362901Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "2025-06-04 09:13:12,070 dotenv.main  WARNING python-dotenv could not parse statement starting at line 3\n",
+      "2025-06-04 09:13:12,071 dotenv.main  WARNING python-dotenv could not parse statement starting at line 4\n"
+     ]
+    }
+   ],
+   "source": [
+    "import dotenv\n",
+    "from pandas.core.interchange.dataframe_protocol import DataFrame\n",
+    "%load_ext dotenv\n",
+    "%dotenv"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2cbd7a83a800b879",
+   "metadata": {},
+   "source": [
+    "The code above loads the values from the `.env` file into this Jupyter Notebook. Alternatively you can set all parameters directly in the method:\n",
+    " \n",
+    "```python\n",
+    "client = RestClient(endpoint='https://test.dbrepo.tuwien.ac.at', username='foo', password='bar')\n",
+    "```\n",
+    "\n",
+    "## Global Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "319e797ebb89b17b",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2025-06-04T07:08:17.403777Z",
+     "start_time": "2025-06-04T07:08:17.399725Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from dbrepo.RestClient import RestClient\n",
+    "\n",
+    "client = RestClient()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dc1ee2ab9378917",
+   "metadata": {},
+   "source": [
+    "## Tables\n",
+    "\n",
+    "Load data from any table into a `pandas` DataFrame."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "initial_id",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2025-06-04T07:08:18.062219Z",
+     "start_time": "2025-06-04T07:08:17.505168Z"
+    }
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div>\n",
+       "<style scoped>\n",
+       "    .dataframe tbody tr th:only-of-type {\n",
+       "        vertical-align: middle;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe tbody tr th {\n",
+       "        vertical-align: top;\n",
+       "    }\n",
+       "\n",
+       "    .dataframe thead th {\n",
+       "        text-align: right;\n",
+       "    }\n",
+       "</style>\n",
+       "<table border=\"1\" class=\"dataframe\">\n",
+       "  <thead>\n",
+       "    <tr style=\"text-align: right;\">\n",
+       "      <th></th>\n",
+       "      <th>acronym</th>\n",
+       "      <th>organization</th>\n",
+       "      <th>ror</th>\n",
+       "      <th>lat</th>\n",
+       "      <th>lng</th>\n",
+       "      <th>url</th>\n",
+       "    </tr>\n",
+       "  </thead>\n",
+       "  <tbody>\n",
+       "    <tr>\n",
+       "      <th>0</th>\n",
+       "      <td>dataLab</td>\n",
+       "      <td>TU Wien</td>\n",
+       "      <td>https://ror.org/04d836q62</td>\n",
+       "      <td>48.19952574</td>\n",
+       "      <td>16.36693332</td>\n",
+       "      <td>https://s34.datalab.tuwien.ac.at/</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>1</th>\n",
+       "      <td>DS-IFS</td>\n",
+       "      <td>TU Wien</td>\n",
+       "      <td>https://ror.org/04d836q62</td>\n",
+       "      <td>48.19889447</td>\n",
+       "      <td>16.36987750</td>\n",
+       "      <td>https://dbrepo1.ec.tuwien.ac.at</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>2</th>\n",
+       "      <td>TUDa</td>\n",
+       "      <td>TU Darmstadt</td>\n",
+       "      <td>https://ror.org/05n911h24</td>\n",
+       "      <td>49.87486230</td>\n",
+       "      <td>8.65636633</td>\n",
+       "      <td>https://tudatabases.ulb.tu-darmstadt.de</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>3</th>\n",
+       "      <td>UHH</td>\n",
+       "      <td>Universität Hamburg</td>\n",
+       "      <td>https://ror.org/00g30e956</td>\n",
+       "      <td>53.56304986</td>\n",
+       "      <td>9.98838274</td>\n",
+       "      <td>https://dbrepo.fdm.uni-hamburg.de</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>4</th>\n",
+       "      <td>UPCE</td>\n",
+       "      <td>University of the Philippines Diliman</td>\n",
+       "      <td>https://ror.org/03tbh6y23</td>\n",
+       "      <td>14.65490814</td>\n",
+       "      <td>121.07216843</td>\n",
+       "      <td>None</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>5</th>\n",
+       "      <td>USM</td>\n",
+       "      <td>Universiti Sains Malaysia</td>\n",
+       "      <td>https://ror.org/02rgb2k63</td>\n",
+       "      <td>5.14588662</td>\n",
+       "      <td>100.49289291</td>\n",
+       "      <td>https://atmosfera.usm.my/dbrepo/</td>\n",
+       "    </tr>\n",
+       "    <tr>\n",
+       "      <th>6</th>\n",
+       "      <td>UTeM</td>\n",
+       "      <td>Technical University of Malaysia Malacca</td>\n",
+       "      <td>https://ror.org/01xb6rs26</td>\n",
+       "      <td>2.31371743</td>\n",
+       "      <td>102.32115732</td>\n",
+       "      <td>https://dbrepo.utem.edu.my/</td>\n",
+       "    </tr>\n",
+       "  </tbody>\n",
+       "</table>\n",
+       "</div>"
+      ],
+      "text/plain": [
+       "   acronym                              organization  \\\n",
+       "0  dataLab                                   TU Wien   \n",
+       "1   DS-IFS                                   TU Wien   \n",
+       "2     TUDa                              TU Darmstadt   \n",
+       "3      UHH                       Universität Hamburg   \n",
+       "4     UPCE     University of the Philippines Diliman   \n",
+       "5      USM                 Universiti Sains Malaysia   \n",
+       "6     UTeM  Technical University of Malaysia Malacca   \n",
+       "\n",
+       "                          ror          lat           lng  \\\n",
+       "0   https://ror.org/04d836q62  48.19952574   16.36693332   \n",
+       "1   https://ror.org/04d836q62  48.19889447   16.36987750   \n",
+       "2   https://ror.org/05n911h24  49.87486230    8.65636633   \n",
+       "3   https://ror.org/00g30e956  53.56304986    9.98838274   \n",
+       "4   https://ror.org/03tbh6y23  14.65490814  121.07216843   \n",
+       "5   https://ror.org/02rgb2k63   5.14588662  100.49289291   \n",
+       "6   https://ror.org/01xb6rs26   2.31371743  102.32115732   \n",
+       "\n",
+       "                                       url  \n",
+       "0        https://s34.datalab.tuwien.ac.at/  \n",
+       "1          https://dbrepo1.ec.tuwien.ac.at  \n",
+       "2  https://tudatabases.ulb.tu-darmstadt.de  \n",
+       "3        https://dbrepo.fdm.uni-hamburg.de  \n",
+       "4                                     None  \n",
+       "5         https://atmosfera.usm.my/dbrepo/  \n",
+       "6              https://dbrepo.utem.edu.my/  "
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "deployments = client.get_table_data('71183815-021d-40d1-884e-3e58bd43b169', '0b533238-e73e-43ec-a773-04ed75349b7a')\n",
+    "display(deployments)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "39095623-65a4-4de9-bbab-419946e5c428",
+   "metadata": {},
+   "source": [
+    "Import data from a `pandas` DataFrame."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f85f8c85-16f9-4b42-88af-e5a50b4729e4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# optional: update table data from `pandas` DataFrame\n",
+    "# df = DataFrame()\n",
+    "# client.import_table_data('71183815-021d-40d1-884e-3e58bd43b169', '0b533238-e73e-43ec-a773-04ed75349b7a', df)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9098ea798a2ab598",
+   "metadata": {},
+   "source": [
+    "## Views\n",
+    "\n",
+    "Load data from any view into a `pandas` DataFrame."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ddd433035e632df9",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2025-06-04T07:08:18.155004278Z",
+     "start_time": "2024-12-14T07:22:44.708283Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "beton_acc = client.get_view_data(database_id, 70)\n",
+    "beton_acc\n",
+    "\n",
+    "# optional: export view data as .csv\n",
+    "# beton_acc.to_csv('beton_acc.csv')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "54b9b1f732219331",
+   "metadata": {},
+   "source": [
+    "## Subsets"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f165e5e3861a0dda",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2025-06-04T07:08:18.180071407Z",
+     "start_time": "2024-12-14T07:22:45.197517Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "subset = client.create_subset(database_id, \"SELECT m.mix, date_production, b.name, ta.k_ac_mm_root_day, ta.test_procedure, ta.mean_depth_t0_mm FROM mix m JOIN betonvariante b ON b.id = m.betonvariante_id JOIN test_result_carb_acc ta ON m.id = ta.mix_id\")\n",
+    "subset"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

.jupyter/Dockerfile

+FROM docker.io/jupyter/datascience-notebook:python-3.11 AS runtime
+LABEL org.opencontainers.image.authors="martin.weise@tuwien.ac.at"
+
+COPY ./environment.yml ./environment.yml
\ No newline at end of file

.jupyter/Pipfile

-[[source]]
-url = "https://pypi.org/simple"
-verify_ssl = true
-name = "pypi"
-
-[packages]
-dbrepo = "*"
-notebook = "*"
-python-dotenv = "*"
-
-[requires]
-python_version = "3.11"
\ No newline at end of file

.jupyter/default.ipynb

-{
- "cells": [
-  {
-   "metadata": {},
-   "cell_type": "markdown",
-   "source": [
-    "# DBRepo Demo Notebook\n",
-    "\n",
-    "## Environment\n",
-    "\n",
-    "Set the REST API endpoint and your username + password in the `.env` file. This ensures it is not revealed in the code."
-   ],
-   "id": "3ca0c15a13bd6882"
-  },
-  {
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-12-14T07:22:43.939456Z",
-     "start_time": "2024-12-14T07:22:43.577273Z"
-    }
-   },
-   "cell_type": "code",
-   "source": [
-    "import dotenv\n",
-    "from pandas.core.interchange.dataframe_protocol import DataFrame\n",
-    "%load_ext dotenv\n",
-    "%dotenv"
-   ],
-   "id": "4eb6c2470f464173",
-   "outputs": [],
-   "execution_count": 1
-  },
-  {
-   "metadata": {},
-   "cell_type": "markdown",
-   "source": [
-    "The code above loads the values from the `.env` file into this Jupyter Notebook. Alternatively you can set all parameters directly in the method:\n",
-    " \n",
-    "```python\n",
-    "client = RestClient(endpoint='https://test.dbrepo.tuwien.ac.at', username='foo', password='bar')\n",
-    "```\n",
-    "\n",
-    "## Global Variables & Imports\n",
-    "\n",
-    "Set the database id for convenience."
-   ],
-   "id": "2cbd7a83a800b879"
-  },
-  {
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-12-14T07:22:44.325683Z",
-     "start_time": "2024-12-14T07:22:43.946474Z"
-    }
-   },
-   "cell_type": "code",
-   "source": [
-    "from dbrepo.RestClient import RestClient\n",
-    "\n",
-    "database_id = 37\n",
-    "client = RestClient()"
-   ],
-   "id": "319e797ebb89b17b",
-   "outputs": [],
-   "execution_count": 2
-  },
-  {
-   "metadata": {},
-   "cell_type": "markdown",
-   "source": [
-    "## Tables\n",
-    "\n",
-    "Load data from any table, uncomment where needed."
-   ],
-   "id": "dc1ee2ab9378917"
-  },
-  {
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-12-14T07:22:44.666058Z",
-     "start_time": "2024-12-14T07:22:44.409680Z"
-    }
-   },
-   "cell_type": "code",
-   "source": [
-    "# test_result_chloride_ageing = client.get_table_data(database_id, 615)\n",
-    "# test_result_chloride = client.get_table_data(database_id, 614)\n",
-    "# test_result_carb_nat = client.get_table_data(database_id, 613)\n",
-    "# test_result_carb_acc = client.get_table_data(database_id, 612)\n",
-    "# source_mat_analysis = client.get_table_data(database_id, 611)\n",
-    "# source_mat = client.get_table_data(database_id, 610)\n",
-    "# pruefstelle = client.get_table_data(database_id, 609)\n",
-    "# mix_recipe = client.get_table_data(database_id, 607)\n",
-    "# mix = client.get_table_data(database_id, 606)\n",
-    "betonvariante = client.get_table_data(database_id, 604)\n",
-    "betonvariante\n",
-    "\n",
-    "# optional: export table data as .csv\n",
-    "# betonvariante.to_csv('betonvariante.csv')\n",
-    "\n",
-    "# optional: update table data from .csv\n",
-    "# client.import_table_data(database_id, 604, 'my_data.csv')\n",
-    "\n",
-    "# optional: update table data from `pandas` DataFrame\n",
-    "# df = DataFrame()\n",
-    "# client.import_table_data(database_id, 604, df)"
-   ],
-   "id": "initial_id",
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "  id name\n",
-       "0  1    s\n",
-       "1  2    d\n",
-       "2  3    p\n",
-       "3  4    c"
-      ],
-      "text/html": [
-       "<div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>id</th>\n",
-       "      <th>name</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>0</th>\n",
-       "      <td>1</td>\n",
-       "      <td>s</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1</th>\n",
-       "      <td>2</td>\n",
-       "      <td>d</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2</th>\n",
-       "      <td>3</td>\n",
-       "      <td>p</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3</th>\n",
-       "      <td>4</td>\n",
-       "      <td>c</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "</div>"
-      ]
-     },
-     "execution_count": 3,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "execution_count": 3
-  },
-  {
-   "metadata": {},
-   "cell_type": "markdown",
-   "source": [
-    "## Views\n",
-    "\n",
-    "Load data from any view, uncomment where needed."
-   ],
-   "id": "9098ea798a2ab598"
-  },
-  {
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-12-14T07:22:45.110019Z",
-     "start_time": "2024-12-14T07:22:44.708283Z"
-    }
-   },
-   "cell_type": "code",
-   "source": [
-    "beton_acc = client.get_view_data(database_id, 70)\n",
-    "beton_acc\n",
-    "\n",
-    "# optional: export view data as .csv\n",
-    "# beton_acc.to_csv('beton_acc.csv')"
-   ],
-   "id": "ddd433035e632df9",
-   "outputs": [
-    {
-     "data": {
-      "text/plain": [
-       "                  mix date_production name      k_ac_mm_root_day  \\\n",
-       "0  20230918-N01-XC2-D      2023-09-18    d  1.590000000000000000   \n",
-       "1  20230918-N03-XC1-S      2023-09-18    s  2.070000000000000000   \n",
-       "2  20230919-N03-XC1-D      2023-09-19    d  1.390000000000000000   \n",
-       "3  20230920-N03-XC2-D      2023-09-20    d  1.260000000000000000   \n",
-       "4  20230918-N04-XC1-S      2023-09-18    s  2.090000000000000000   \n",
-       "5  20230919-N06-XC1-S      2023-09-19    s  1.070000000000000000   \n",
-       "6  20230920-N06-XC2-D      2023-09-20    d  0.770000000000000000   \n",
-       "\n",
-       "  test_procedure      mean_depth_t0_mm  \n",
-       "0        KB-TVFA  0.750000000000000000  \n",
-       "1        KB-TVFA  0.250000000000000000  \n",
-       "2        KB-TVFA  0.200000000000000000  \n",
-       "3        KB-TVFA  0.300000000000000000  \n",
-       "4        KB-TVFA  0.300000000000000000  \n",
-       "5        KB-TVFA                 0E-18  \n",
-       "6        KB-TVFA                 0E-18  "
-      ],
-      "text/html": [
-       "<div>\n",
-       "<style scoped>\n",
-       "    .dataframe tbody tr th:only-of-type {\n",
-       "        vertical-align: middle;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe tbody tr th {\n",
-       "        vertical-align: top;\n",
-       "    }\n",
-       "\n",
-       "    .dataframe thead th {\n",
-       "        text-align: right;\n",
-       "    }\n",
-       "</style>\n",
-       "<table border=\"1\" class=\"dataframe\">\n",
-       "  <thead>\n",
-       "    <tr style=\"text-align: right;\">\n",
-       "      <th></th>\n",
-       "      <th>mix</th>\n",
-       "      <th>date_production</th>\n",
-       "      <th>name</th>\n",
-       "      <th>k_ac_mm_root_day</th>\n",
-       "      <th>test_procedure</th>\n",
-       "      <th>mean_depth_t0_mm</th>\n",
-       "    </tr>\n",
-       "  </thead>\n",
-       "  <tbody>\n",
-       "    <tr>\n",
-       "      <th>0</th>\n",
-       "      <td>20230918-N01-XC2-D</td>\n",
-       "      <td>2023-09-18</td>\n",
-       "      <td>d</td>\n",
-       "      <td>1.590000000000000000</td>\n",
-       "      <td>KB-TVFA</td>\n",
-       "      <td>0.750000000000000000</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>1</th>\n",
-       "      <td>20230918-N03-XC1-S</td>\n",
-       "      <td>2023-09-18</td>\n",
-       "      <td>s</td>\n",
-       "      <td>2.070000000000000000</td>\n",
-       "      <td>KB-TVFA</td>\n",
-       "      <td>0.250000000000000000</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>2</th>\n",
-       "      <td>20230919-N03-XC1-D</td>\n",
-       "      <td>2023-09-19</td>\n",
-       "      <td>d</td>\n",
-       "      <td>1.390000000000000000</td>\n",
-       "      <td>KB-TVFA</td>\n",
-       "      <td>0.200000000000000000</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>3</th>\n",
-       "      <td>20230920-N03-XC2-D</td>\n",
-       "      <td>2023-09-20</td>\n",
-       "      <td>d</td>\n",
-       "      <td>1.260000000000000000</td>\n",
-       "      <td>KB-TVFA</td>\n",
-       "      <td>0.300000000000000000</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>4</th>\n",
-       "      <td>20230918-N04-XC1-S</td>\n",
-       "      <td>2023-09-18</td>\n",
-       "      <td>s</td>\n",
-       "      <td>2.090000000000000000</td>\n",
-       "      <td>KB-TVFA</td>\n",
-       "      <td>0.300000000000000000</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>5</th>\n",
-       "      <td>20230919-N06-XC1-S</td>\n",
-       "      <td>2023-09-19</td>\n",
-       "      <td>s</td>\n",
-       "      <td>1.070000000000000000</td>\n",
-       "      <td>KB-TVFA</td>\n",
-       "      <td>0E-18</td>\n",
-       "    </tr>\n",
-       "    <tr>\n",
-       "      <th>6</th>\n",
-       "      <td>20230920-N06-XC2-D</td>\n",
-       "      <td>2023-09-20</td>\n",
-       "      <td>d</td>\n",
-       "      <td>0.770000000000000000</td>\n",
-       "      <td>KB-TVFA</td>\n",
-       "      <td>0E-18</td>\n",
-       "    </tr>\n",
-       "  </tbody>\n",
-       "</table>\n",
-       "</div>"
-      ]
-     },
-     "execution_count": 4,
-     "metadata": {},
-     "output_type": "execute_result"
-    }
-   ],
-   "execution_count": 4
-  },
-  {
-   "metadata": {},
-   "cell_type": "markdown",
-   "source": "## Subsets",
-   "id": "54b9b1f732219331"
-  },
-  {
-   "metadata": {
-    "ExecuteTime": {
-     "end_time": "2024-12-14T07:22:46.566518Z",
-     "start_time": "2024-12-14T07:22:45.197517Z"
-    }
-   },
-   "cell_type": "code",
-   "source": [
-    "subset = client.create_subset(database_id, \"SELECT m.mix, date_production, b.name, ta.k_ac_mm_root_day, ta.test_procedure, ta.mean_depth_t0_mm FROM mix m JOIN betonvariante b ON b.id = m.betonvariante_id JOIN test_result_carb_acc ta ON m.id = ta.mix_id\")\n",
-    "subset"
-   ],
-   "id": "f165e5e3861a0dda",
-   "outputs": [
-    {
-     "ename": "ResponseCodeError",
-     "evalue": "Failed to create subset: response code: 200 is not 201 (CREATED): [{\"mix\":\"20230918-N01-XC2-D\",\"date_production\":\"2023-09-18\",\"name\":\"d\",\"k_ac_mm_root_day\":\"1.590000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0.750000000000000000\"},{\"mix\":\"20230918-N03-XC1-S\",\"date_production\":\"2023-09-18\",\"name\":\"s\",\"k_ac_mm_root_day\":\"2.070000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0.250000000000000000\"},{\"mix\":\"20230919-N03-XC1-D\",\"date_production\":\"2023-09-19\",\"name\":\"d\",\"k_ac_mm_root_day\":\"1.390000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0.200000000000000000\"},{\"mix\":\"20230920-N03-XC2-D\",\"date_production\":\"2023-09-20\",\"name\":\"d\",\"k_ac_mm_root_day\":\"1.260000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0.300000000000000000\"},{\"mix\":\"20230918-N04-XC1-S\",\"date_production\":\"2023-09-18\",\"name\":\"s\",\"k_ac_mm_root_day\":\"2.090000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0.300000000000000000\"},{\"mix\":\"20230919-N06-XC1-S\",\"date_production\":\"2023-09-19\",\"name\":\"s\",\"k_ac_mm_root_day\":\"1.070000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0E-18\"},{\"mix\":\"20230920-N06-XC2-D\",\"date_production\":\"2023-09-20\",\"name\":\"d\",\"k_ac_mm_root_day\":\"0.770000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0E-18\"}]",
-     "output_type": "error",
-     "traceback": [
-      "\u001B[0;31m---------------------------------------------------------------------------\u001B[0m",
-      "\u001B[0;31mResponseCodeError\u001B[0m                         Traceback (most recent call last)",
-      "Cell \u001B[0;32mIn[5], line 1\u001B[0m\n\u001B[0;32m----> 1\u001B[0m subset \u001B[38;5;241m=\u001B[39m \u001B[43mclient\u001B[49m\u001B[38;5;241;43m.\u001B[39;49m\u001B[43mcreate_subset\u001B[49m\u001B[43m(\u001B[49m\u001B[43mdatabase_id\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;124;43m\"\u001B[39;49m\u001B[38;5;124;43mSELECT m.mix, date_production, b.name, ta.k_ac_mm_root_day, ta.test_procedure, ta.mean_depth_t0_mm FROM mix m JOIN betonvariante b ON b.id = m.betonvariante_id JOIN test_result_carb_acc ta ON m.id = ta.mix_id\u001B[39;49m\u001B[38;5;124;43m\"\u001B[39;49m\u001B[43m)\u001B[49m\n\u001B[1;32m      2\u001B[0m subset\n",
-      "File \u001B[0;32m~/Projects/fda-services/.jupyter/venv/lib/python3.11/site-packages/dbrepo/RestClient.py:1590\u001B[0m, in \u001B[0;36mRestClient.create_subset\u001B[0;34m(self, database_id, query, page, size, timestamp)\u001B[0m\n\u001B[1;32m   1588\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m response\u001B[38;5;241m.\u001B[39mstatus_code \u001B[38;5;241m==\u001B[39m \u001B[38;5;241m503\u001B[39m:\n\u001B[1;32m   1589\u001B[0m     \u001B[38;5;28;01mraise\u001B[39;00m ServiceError(\u001B[38;5;124mf\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mFailed to create subset: failed to establish connection with data database\u001B[39m\u001B[38;5;124m'\u001B[39m)\n\u001B[0;32m-> 1590\u001B[0m \u001B[38;5;28;01mraise\u001B[39;00m ResponseCodeError(\u001B[38;5;124mf\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mFailed to create subset: response code: \u001B[39m\u001B[38;5;132;01m{\u001B[39;00mresponse\u001B[38;5;241m.\u001B[39mstatus_code\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m is not \u001B[39m\u001B[38;5;124m'\u001B[39m\n\u001B[1;32m   1591\u001B[0m                         \u001B[38;5;124mf\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124m201 (CREATED): \u001B[39m\u001B[38;5;132;01m{\u001B[39;00mresponse\u001B[38;5;241m.\u001B[39mtext\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m'\u001B[39m)\n",
-      "\u001B[0;31mResponseCodeError\u001B[0m: Failed to create subset: response code: 200 is not 201 (CREATED): [{\"mix\":\"20230918-N01-XC2-D\",\"date_production\":\"2023-09-18\",\"name\":\"d\",\"k_ac_mm_root_day\":\"1.590000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0.750000000000000000\"},{\"mix\":\"20230918-N03-XC1-S\",\"date_production\":\"2023-09-18\",\"name\":\"s\",\"k_ac_mm_root_day\":\"2.070000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0.250000000000000000\"},{\"mix\":\"20230919-N03-XC1-D\",\"date_production\":\"2023-09-19\",\"name\":\"d\",\"k_ac_mm_root_day\":\"1.390000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0.200000000000000000\"},{\"mix\":\"20230920-N03-XC2-D\",\"date_production\":\"2023-09-20\",\"name\":\"d\",\"k_ac_mm_root_day\":\"1.260000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0.300000000000000000\"},{\"mix\":\"20230918-N04-XC1-S\",\"date_production\":\"2023-09-18\",\"name\":\"s\",\"k_ac_mm_root_day\":\"2.090000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0.300000000000000000\"},{\"mix\":\"20230919-N06-XC1-S\",\"date_production\":\"2023-09-19\",\"name\":\"s\",\"k_ac_mm_root_day\":\"1.070000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0E-18\"},{\"mix\":\"20230920-N06-XC2-D\",\"date_production\":\"2023-09-20\",\"name\":\"d\",\"k_ac_mm_root_day\":\"0.770000000000000000\",\"test_procedure\":\"KB-TVFA\",\"mean_depth_t0_mm\":\"0E-18\"}]"
-     ]
-    }
-   ],
-   "execution_count": 5
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 2
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython2",
-   "version": "2.7.6"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}

.jupyter/environment.yml

+name: main
+dependencies:
+  - dbrepo=1.9.1
+  - python-dotenv
\ No newline at end of file

Makefile

 .PHONY: all
 
-APP_VERSION ?= 1.9.2
-CHART_VERSION ?= 1.9.2
+APP_VERSION ?= 1.9.3
+CHART_VERSION ?= 1.9.3
 REPOSITORY_URL ?= registry.datalab.tuwien.ac.at/dbrepo
 
 .PHONY: all

dbrepo-analyse-service/Pipfile

@@ -21,7 +21,7 @@ numpy = "*"
 pandas = "*"
 minio = "*"
 pydantic = "*"
-dbrepo = {path = "./lib/dbrepo-1.9.2.tar.gz"}
+dbrepo = {path = "./lib/dbrepo-1.9.3.tar.gz"}
 opensearch-py = "*"
 ecs_logging = "*"