WIP

Signed-off-by: Martin Weise <martin.weise@tuwien.ac.at>

.docs/concepts/data-versioning.md

@@ -6,34 +6,23 @@ Data is getting bigger and so are expectations of data provisioning in regards t
 after quality check and not in snapshot intervals), cost-effectiveness (i.e. no duplication of data), transparent,
 precise citation and many more.
 
-[System-versioned](https://mariadb.com/kb/en/system-versioned-tables/) tables in MariaDB are improved data structures
-that keep track of historical data. For each entry in a system-versioned table, a time period is maintained that denotes
-the validity time span of this tuple from its start to end. Tuples in system-versioned tables are not *actually*
-modified, they are marked as (in-)valid in time periods.
-
-<figure markdown>
-
-| ID | Sensor | Temp | Start | End |
-|----|--------|------|-------|-----|
-| 1  | A      | 23.1 | t1    |     |
-| 2  | B      | 25.8 | t2    |     |
-
+System-versioned tables in MariaDB are improved data structures that keep track of historical data. For each entry in a
+system-versioned table, a time period is maintained that denotes the validity time span of this tuple from its start to
+end. Tuples in system-versioned tables are not *actually* modified, they are marked as (in-)valid in time periods
+(c.f. [Fig. 1](#fig1)).
+
+<figure id="fig1" markdown>
+![](../images/data-versioning.png)
+<figcaption>Fig. 1: Data versioning in MariaDB system-versioned tables.</figcaption>
 </figure>
 
 Assuming that Sensor A was calibrated wrong and an updated measurement is passed to the system-versioned table, the
 table contents show that the old row with Temp 23.1 is not deleted, but marked as valid in time span (t1, t3). The
 updated row with Temp 22.1 is marked as valid from time span t3 onwards.
 
-<figure markdown>
-
-| ID | Sensor | Temp | Start | End |
-|----|--------|------|-------|-----|
-| 1  | A      | 23.1 | t1    | t3  |
-| 2  | B      | 25.8 | t2    |     |
-| 1  | A      | 22.1 | t3    |     |
-
-</figure>
+## Further Reading
 
-System-versioned tables are part of the SQL:2011 standard and have been adopted by many database management system
-vendors: MariaDB (10.5 and higher), Google BigQuery, IBM DB2 (12 and higher), SQL Server (2016 and higher), Azure SQL,
-PostgreSQL with [temporal tables extension](https://github.com/nearform/temporal_tables), etc.
+System-versioned tables are part of the current ISO/IEC 9075-2:2023 ("SQL") standard and have been adopted by many
+database management system vendors, e.g. MariaDB (10.5 and higher), Google BigQuery, IBM DB2 (12 and higher),
+Cockroach DB, SAP HANA, SQL Server (2016 and higher), Azure SQL, PostgreSQL
\ No newline at end of file
+with [temporal tables extension](https://github.com/nearform/temporal_tables), etc.
\ No newline at end of file

.docs/index.md

@@ -16,16 +16,16 @@ author: Martin Weise
 
 Documentation for version: [v1.7.3](https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-services/-/releases).
 
-DBRepo is a repository for data in databases that are used from the beginning until the end of a research 
-project supporting data evolution, -citation and -versioning. It implements the query store of the 
-[RDA WGDC](https://doi.org/10.1162/99608f92.be565013) on precisely identifying arbitrary subsets of data.
+DBRepo is a repository for data in databases that cover the entire data life cycle supporting data evolution, -citation 
+and -versioning. It implements the query store of the [RDA WGDC](https://doi.org/10.1162/99608f92.be565013) on precisely
+identifying arbitrary subsets of data.
 
 ## Why use DBRepo?
 
-* **Built-in search** makes your dataset searchable without extra effort: most metadata is generated 
-  automatically for data in your databases.
+* **Built-in search** makes your dataset searchable without extra effort: metadata is generated automatically for data
+  in your databases.
 * **Citable datasets** adopting the recommendations of the RDA-WGDC, arbitrary subsets can be precisely, persistently 
-  identified using system-versioned tables of MariaDB and the DataCite schema for minting DOIs.
+  identified using [data versioning](concepts/data-versioning) of MariaDB and the DataCite schema for minting DOIs.
 * **Powerful API for Data Scientists** with our strongly typed Python Library, Data Scientists can import, export and
   work with data from Jupyter Notebook or Python script, optionally using Pandas DataFrames.
 * **Cloud Native** our lightweight Helm chart allows for installations on any cloud provider or private-cloud setting 

lib/java/dbrepo-core/src/main/java/at/ac/tuwien/ifs/dbrepo/core/test/BaseTest.java

@@ -244,6 +244,33 @@ public class BaseTest {
     public static final Identifier IDENTIFIER_7;
     public static final IdentifierDto IDENTIFIER_7_DTO;
     public static final IdentifierBriefDto IDENTIFIER_7_BRIEF_DTO;
+    public static final Table TABLE_1;
+    public static final TableDto TABLE_1_DTO;
+    public static final TableBriefDto TABLE_1_BRIEF_DTO;
+    public static final Table TABLE_2;
+    public static final TableDto TABLE_2_DTO;
+    public static final TableBriefDto TABLE_2_BRIEF_DTO;
+    public static final Table TABLE_3;
+    public static final TableDto TABLE_3_DTO;
+    public static final TableBriefDto TABLE_3_BRIEF_DTO;
+    public static final Table TABLE_4;
+    public static final TableDto TABLE_4_DTO;
+    public static final TableBriefDto TABLE_4_BRIEF_DTO;
+    public static final Table TABLE_5;
+    public static final TableDto TABLE_5_DTO;
+    public static final TableBriefDto TABLE_5_BRIEF_DTO;
+    public static final Table TABLE_6;
+    public static final TableDto TABLE_6_DTO;
+    public static final TableBriefDto TABLE_6_BRIEF_DTO;
+    public static final Table TABLE_7;
+    public static final TableDto TABLE_7_DTO;
+    public static final TableBriefDto TABLE_7_BRIEF_DTO;
+    public static final Table TABLE_8;
+    public static final TableDto TABLE_8_DTO;
+    public static final TableBriefDto TABLE_8_BRIEF_DTO;
+    public static final Table TABLE_9;
+    public static final TableDto TABLE_9_DTO;
+    public static final TableBriefDto TABLE_9_BRIEF_DTO;
 
     static {
         try {
@@ -322,6 +349,33 @@ public class BaseTest {
             IDENTIFIER_7 = objectMapper().readValue(new File(resourcesPath + "/identifier-7.json"), Identifier.class);
             IDENTIFIER_7_DTO = metadataMapper.identifierToIdentifierDto(IDENTIFIER_7);
             IDENTIFIER_7_BRIEF_DTO = metadataMapper.identifierToIdentifierBriefDto(IDENTIFIER_7);
+            TABLE_1 = objectMapper().readValue(new File(resourcesPath + "/table-1.json"), Table.class);
+            TABLE_1_DTO = metadataMapper.tableToTableDto(TABLE_1);
+            TABLE_1_BRIEF_DTO = metadataMapper.tableToTableBriefDto(TABLE_1);
+            TABLE_2 = objectMapper().readValue(new File(resourcesPath + "/table-2.json"), Table.class);
+            TABLE_2_DTO = metadataMapper.tableToTableDto(TABLE_2);
+            TABLE_2_BRIEF_DTO = metadataMapper.tableToTableBriefDto(TABLE_2);
+            TABLE_3 = objectMapper().readValue(new File(resourcesPath + "/table-3.json"), Table.class);
+            TABLE_3_DTO = metadataMapper.tableToTableDto(TABLE_3);
+            TABLE_3_BRIEF_DTO = metadataMapper.tableToTableBriefDto(TABLE_3);
+            TABLE_4 = objectMapper().readValue(new File(resourcesPath + "/table-4.json"), Table.class);
+            TABLE_4_DTO = metadataMapper.tableToTableDto(TABLE_4);
+            TABLE_4_BRIEF_DTO = metadataMapper.tableToTableBriefDto(TABLE_4);
+            TABLE_5 = objectMapper().readValue(new File(resourcesPath + "/table-5.json"), Table.class);
+            TABLE_5_DTO = metadataMapper.tableToTableDto(TABLE_5);
+            TABLE_5_BRIEF_DTO = metadataMapper.tableToTableBriefDto(TABLE_5);
+            TABLE_6 = objectMapper().readValue(new File(resourcesPath + "/table-6.json"), Table.class);
+            TABLE_6_DTO = metadataMapper.tableToTableDto(TABLE_6);
+            TABLE_6_BRIEF_DTO = metadataMapper.tableToTableBriefDto(TABLE_6);
+            TABLE_7 = objectMapper().readValue(new File(resourcesPath + "/table-7.json"), Table.class);
+            TABLE_7_DTO = metadataMapper.tableToTableDto(TABLE_7);
+            TABLE_7_BRIEF_DTO = metadataMapper.tableToTableBriefDto(TABLE_7);
+            TABLE_8 = objectMapper().readValue(new File(resourcesPath + "/table-8.json"), Table.class);
+            TABLE_8_DTO = metadataMapper.tableToTableDto(TABLE_8);
+            TABLE_8_BRIEF_DTO = metadataMapper.tableToTableBriefDto(TABLE_8);
+            TABLE_9 = objectMapper().readValue(new File(resourcesPath + "/table-9.json"), Table.class);
+            TABLE_9_DTO = metadataMapper.tableToTableDto(TABLE_9);
+            TABLE_9_BRIEF_DTO = metadataMapper.tableToTableBriefDto(TABLE_9);
         } catch (IOException e) {
             throw new RuntimeException("Failed to initialize core test library", e);
         }
@@ -608,47 +662,9 @@ public class BaseTest {
             .cid(CONTAINER_1.getId())
             .build();
 
-    public static final at.ac.tuwien.ifs.dbrepo.core.api.database.internal.CreateDatabaseDto DATABASE_1_CREATE_INTERNAL = at.ac.tuwien.ifs.dbrepo.core.api.database.internal.CreateDatabaseDto.builder()
-            .internalName(DATABASE_1.getInternalName())
-            .containerId(CONTAINER_1.getId())
-            .username(USER_1.getUsername())
-            .password(USER_1_PASSWORD)
-            .userId(USER_1.getId())
-            .privilegedUsername(CONTAINER_1_PRIVILEGED_USERNAME)
-            .privilegedPassword(CONTAINER_1_PRIVILEGED_PASSWORD)
-            .build();
-
-    public static final CreateDatabaseDto DATABASE_2_CREATE = CreateDatabaseDto.builder()
-            .name(DATABASE_2_NAME)
-            .isPublic(DATABASE_2_PUBLIC)
-            .cid(CONTAINER_1.getId())
-            .build();
-
-    public static final at.ac.tuwien.ifs.dbrepo.core.api.database.internal.CreateDatabaseDto DATABASE_2_CREATE_INTERNAL = at.ac.tuwien.ifs.dbrepo.core.api.database.internal.CreateDatabaseDto.builder()
-            .internalName(DATABASE_2.getInternalName())
-            .containerId(CONTAINER_1.getId())
-            .username(USER_1.getUsername())
-            .password(USER_1_PASSWORD)
-            .userId(USER_1.getId())
-            .privilegedUsername(CONTAINER_1_PRIVILEGED_USERNAME)
-            .privilegedPassword(CONTAINER_1_PRIVILEGED_PASSWORD)
-            .build();
-
-    public static final CreateDatabaseDto DATABASE_3_CREATE = CreateDatabaseDto.builder()
-            .name(DATABASE_3_NAME)
-            .isPublic(DATABASE_3_PUBLIC)
-            .cid(CONTAINER_1.getId())
-            .build();
-
-    public static final at.ac.tuwien.ifs.dbrepo.core.api.database.internal.CreateDatabaseDto DATABASE_3_CREATE_INTERNAL = at.ac.tuwien.ifs.dbrepo.core.api.database.internal.CreateDatabaseDto.builder()
-            .internalName(DATABASE_3.getInternalName())
-            .containerId(CONTAINER_1.getId())
-            .username(USER_1.getUsername())
-            .password(USER_1_PASSWORD)
-            .userId(USER_1.getId())
-            .privilegedUsername(CONTAINER_1_PRIVILEGED_USERNAME)
-            .privilegedPassword(CONTAINER_1_PRIVILEGED_PASSWORD)
-            .build();
+    public static final String CONTAINER_1_PRIVILEGED_USERNAME = "root";
+    @SuppressWarnings("java:S2068")
+    public static final String CONTAINER_1_PRIVILEGED_PASSWORD = "dbrepo";
 
     public static final Long TABLE_1_DATA_COUNT = 3L;
     @SuppressWarnings("java:S3599")
@@ -678,18 +694,18 @@ public class BaseTest {
 
     public static final SubsetDto QUERY_9_SUBSET_DTO = SubsetDto.builder()
             .tableId(TABLE_9.getId())
-            .columns(new LinkedList<>(List.of(COLUMN_9_2_ID, COLUMN_9_3_ID)))
+            .columns(new LinkedList<>(List.of(TABLE_9.getColumns().get(2).getId(), TABLE_9.getColumns().get(3).getId())))
             .filter(new LinkedList<>(List.of(FilterDto.builder()
-                    .columnId(COLUMN_9_1_ID)
-                    .operatorId(IMAGE_1_OPERATORS_2_ID)
+                    .columnId(TABLE_9.getColumns().get(1).getId())
+                    .operatorId(IMAGE_1 IMAGE_1_OPERATORS_2_ID)
                     .value("Fuji")
                     .type(FilterTypeDto.WHERE)
                     .build())))
             .build();
 
     public static final ViewDto QUERY_9_VIEW_DTO = ViewDto.builder()
-            .query(QUERY_9_STATEMENT)
-            .queryHash(QUERY_9_QUERY_HASH)
+            .query(QUERY_9_DTO.getQuery())
+            .queryHash(QUERY_9_DTO.getQueryHash())
             .owner(USER_1_BRIEF_DTO)
             .columns(new LinkedList<>(List.of(ViewColumnDto.builder()
                             .name("lat")