Skip to content
Snippets Groups Projects
Verified Commit 78862b71 authored by Martin Weise's avatar Martin Weise
Browse files

Updated docs

parent b5a30b76
No related branches found
No related tags found
3 merge requests!231CI: Remove build for log-service,!228Better error message handling in the frontend,!223Release of version 1.4.0
...@@ -23,12 +23,6 @@ We present a database repository system that allows researchers to ingest data i ...@@ -23,12 +23,6 @@ We present a database repository system that allows researchers to ingest data i
through common interfaces, provides efficient access to arbitrary subsets of data even when the underlying data store is through common interfaces, provides efficient access to arbitrary subsets of data even when the underlying data store is
evolving, allows reproducing of query results and supports findable-, accessible-, interoperable- and reusable data. evolving, allows reproducing of query results and supports findable-, accessible-, interoperable- and reusable data.
## Community
These institutions use DBRepo and integrated it into their repository infrastructure
<img src="images/logos.png" width="100%" alt="Logos of TU Darmstadt, TU Wien, University Malaya" />
## More Information ## More Information
- Demonstration instance [https://dbrepo1.ec.tuwien.ac.at](https://dbrepo1.ec.tuwien.ac.at) - Demonstration instance [https://dbrepo1.ec.tuwien.ac.at](https://dbrepo1.ec.tuwien.ac.at)
......
...@@ -38,32 +38,7 @@ the [Storage Service](../system-services-storage), analysis for data types and p ...@@ -38,32 +38,7 @@ the [Storage Service](../system-services-storage), analysis for data types and p
### Examples ### Examples
Given a [CSV-file](https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-datasets/-/raw/master/gps.csv) See the [usage page](../usage-analyse).
containing GPS-data `gps.csv` already uploaded in the `dbrepo-upload` bucket of the Storage Service with key `gps.csv`:
```shell
curl -X POST \
-d '{"filename":"gps.csv","separator":","}'
http://<hostname>:5000/api/analyse/determinedt
```
This results in the response:
```json
{
"columns": {
"ID": "bigint",
"KEY": "varchar",
"OBJECTID": "bigint",
"LBEZEICHNUNG": "varchar",
"LTYP": "bigint",
"LTYPTXT": "varchar",
"LAT": "decimal",
"LNG": "decimal"
},
"separator": ","
}
```
## Limitations ## Limitations
......
...@@ -33,42 +33,7 @@ The default configuration creates two buckets `dbrepo-upload`, `dbrepo-download` ...@@ -33,42 +33,7 @@ The default configuration creates two buckets `dbrepo-upload`, `dbrepo-download`
### Examples ### Examples
Upload a CSV-file into the `dbrepo-upload` bucket with the AWS CLI: See the [usage page](../usage-storage).
```console
$ aws --endpoint-url http://<hostname>:9000 \
s3 \
cp /path/to/file.csv \
s3://dbrepo-upload/
upload: /path/to/file.csv to s3://dbrepo-upload/file.csv
```
You can list the buckets:
```console
$ aws --endpoint-url http://<hostname>:9000 \
s3 \
ls
2023-12-03 16:23:15 dbrepo-download
2023-12-03 16:28:05 dbrepo-upload
```
And list the files in the bucket `dbrepo-upload` with:
```console
$ aws --endpoint-url http://<hostname>:9000 \
s3 \
ls \
dbrepo-upload
2023-12-03 16:28:05 535219 file.csv
```
Alternatively, you can use the middleware of the [User Interface](../system-other-ui/) to upload files.
Alternatively, you can use a S3-compatible client:
* [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) (generic Python implementation of S3)
* AWS SDK (tailored towards Amazon S3)
## Limitations ## Limitations
......
...@@ -20,16 +20,7 @@ We use the [TUS](https://tus.io/) open protocol for resumable file uploads which ...@@ -20,16 +20,7 @@ We use the [TUS](https://tus.io/) open protocol for resumable file uploads which
### Examples ### Examples
Upload a CSV-file into the `dbrepo-upload` bucket with the console See the [usage page](../usage-upload).
via `http://<hostname>/admin/storage/browser/dbrepo-upload`.
We recommend using a TUS-compatible client:
* [tus-java-client](https://github.com/tus/tus-java-client) (Java)
* [tus-js-client](https://github.com/tus/tus-js-client) (JavaScript/Node.js)
* [tusd](https://github.com/tus/tusd) (Go)
## Limitations ## Limitations
......
---
author: Martin Weise
---
# Analyse Service
Given a [CSV-file](https://gitlab.phaidra.org/fair-data-austria-db-repository/fda-datasets/-/raw/master/gps.csv)
containing GPS-data `gps.csv` already uploaded in the `dbrepo-upload` bucket of the Storage Service with key `gps.csv`:
```shell
curl -X POST \
-d '{"filename":"gps.csv","separator":","}'
http://<hostname>:5000/api/analyse/determinedt
```
This results in the response:
```json
{
"columns": {
"ID": "bigint",
"KEY": "varchar",
"OBJECTID": "bigint",
"LBEZEICHNUNG": "varchar",
"LTYP": "bigint",
"LTYPTXT": "varchar",
"LAT": "decimal",
"LNG": "decimal"
},
"separator": ","
}
```
\ No newline at end of file
---
author: Martin Weise
---
# Storage Service
Upload a CSV-file into the `dbrepo-upload` bucket with the AWS CLI:
```console
$ aws --endpoint-url http://<hostname>:9000 \
s3 \
cp /path/to/file.csv \
s3://dbrepo-upload/
upload: /path/to/file.csv to s3://dbrepo-upload/file.csv
```
You can list the buckets:
```console
$ aws --endpoint-url http://<hostname>:9000 \
s3 \
ls
2023-12-03 16:23:15 dbrepo-download
2023-12-03 16:28:05 dbrepo-upload
```
And list the files in the bucket `dbrepo-upload` with:
```console
$ aws --endpoint-url http://<hostname>:9000 \
s3 \
ls \
dbrepo-upload
2023-12-03 16:28:05 535219 file.csv
```
Alternatively, you can use the middleware of the [User Interface](../system-other-ui/) to upload files.
Alternatively, you can use a S3-compatible client:
* [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) (generic Python implementation of S3)
* AWS SDK (tailored towards Amazon S3)
...@@ -4,7 +4,17 @@ author: Martin Weise ...@@ -4,7 +4,17 @@ author: Martin Weise
# Upload Service # Upload Service
Uploads a file `file.csv` in 200 byte chunks. Upload a CSV-file into the `dbrepo-upload` bucket with the console
via `http://<hostname>/admin/storage/browser/dbrepo-upload`.
We recommend using a TUS-compatible client:
* [tus-py-client](https://github.com/tus/tus-py-client) (Python)
* [tus-java-client](https://github.com/tus/tus-java-client) (Java)
* [tus-js-client](https://github.com/tus/tus-js-client) (JavaScript/Node.js)
* [tusd](https://github.com/tus/tusd) (Go)
You can also upload a file `file.csv` in 200 byte chunks with Python:
=== "Python" === "Python"
......
...@@ -31,10 +31,13 @@ nav: ...@@ -31,10 +31,13 @@ nav:
- Search Database Dashboard: system-other-search-dashboard.md - Search Database Dashboard: system-other-search-dashboard.md
- Usage: - Usage:
- Overview: usage-overview.md - Overview: usage-overview.md
- Services:
- Analyse Service: usage-analyse.md
- Authentication Service: usage-auth.md - Authentication Service: usage-auth.md
- Broker Service: usage-broker.md - Broker Service: usage-broker.md
- Identifier Service: usage-identifier.md - Identifier Service: usage-identifier.md
- Upload Service: usage-upload.md - Upload Service: usage-upload.md
- Storage Service: usage-storage.md
- publications.md - publications.md
- contact.md - contact.md
extra_css: extra_css:
......
File moved
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment