diff --git a/workflow.ipynb b/workflow.ipynb
index e9fd0b16aefe0bc8476302a2b9970dfd1c1b6b38..7c7f0c0df91f79239ee5c1ae7e37e25e51681da1 100644
--- a/workflow.ipynb
+++ b/workflow.ipynb
@@ -9,9 +9,9 @@
"## Intro to Topic Modeling on a Data Set created by the Newseye Project\n",
"\n",
"The data analysed in the following by software libraries [Gensim](https://radimrehurek.com/gensim/) and [Mallet](http://mallet.cs.umass.edu/) were initially taken from the [ANNO](https://anno.onb.ac.at/infos_zeizs.htm) system of the Austrian National Library, which is and was involved in the [Newseye](https://www.newseye.eu/) project (2018-2021). The data was then re-OCRed through the Transkribus programme and the layout of the newspapers was also analysed. Significantly improved Optical Caracter Recognition coupled with a form of Article Separation makes it possible to build a data platform that enables a new and very different quality of search, addressing and also storage. After these enrichment processes, the data was imported into the Newseye platform, which is based on the Content Management System [Blacklight](https://blacklight-cms.net).\n",
- "The data used here is based on a general search on the basis of the search engine [Solr](https://solr.apache.org/) for articles on the word telegraph in the following time periods: 1864-1874, 1895-1901, 1911-1922.\n",
+ "The data used here is based on a general search on the basis of the search engine [Solr](https://solr.apache.org/) for articles on the word \"telegraph\" in the following time periods: 1864-1874, 1895-1901, 1911-1922.\n",
"The search was carried out in the following German-language newspapers, namely _Neue Freie Presse_, *Innsbrucker Nachrichten*, *Arbeiter Zeitung* and *Illustrierte Kronen Zeitung*. The resulting data package was exported as JSON and processed with regard to topic modelling.\n",
- "The total number of hits was 14949, of which the following is a breakdown by newspaper:\n",
+ "The total number of hits was 14,949, of which the following is a breakdown by newspaper:\n",
"\n",
"```\n",
" Neue Freie Presse 10,981\n",
@@ -29,7 +29,7 @@
"## Workflow of Processing Transcribus Data: From Scans to Topic Model Visualizations\n",
"\n",
"First, the necessary libraries and modules are imported and some variables are initiated. Note the use of \n",
- "the convenience module `tm_utils.py` containing helper functions for operations needed frequently. Above all, functions for saving and retrieving of calculation results are defined there."
+ "the convenience module `tm_utils.py` containing helper functions for operations needed frequently. Mainly functions for saving and retrieving of calculation results are defined there."
]
},
{
@@ -59,7 +59,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Some data exploration\n",
+ "## Preliminary Data Exploration\n",
"\n",
"First, we have a look at the `json` data coming out of `transkribus` by printing its length and some of\n",
"its content (this could also be done in `jupyterlab` by opening the `json` file)."
@@ -933,7 +933,13 @@
"* Most representative topics\n",
"* Distribution of topics\n",
"\n",
- "We only show an abridged version of the dominant topic list; our [Datasette](https://datasette.io/) instance would render all data, so that the direct link to actual source data (column \"Link\") would be functional.\n",
+ "We only show an abridged version of the dominant topic list; our [Datasette](https://datasette.io/) instance would render all data, so that the direct link to actual source data (column \"Link\") would be functional.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
"\n",
"### Most Dominant Topic per Document\n",
"\n",
@@ -1936,6 +1942,29 @@
"distr_top.columns = ['Dominant_Topic', 'Num_Documents', 'Perc_Documents', 'Topic_Keywords']\n",
"distr_top.head(28)"
]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Future Prospects\n",
+ "\n",
+ "A broad search result provides information on areas in which the term \"telegraph\" is used. The topics listed here were found after filtering by search for the term \"telegraph\" in the _Newseye_ platform articles (not open for the public). The term itself was removed from the corpus in preprocessing. \n",
+ "\n",
+ "On this basis it is then possible to search in more specific contexts and with more precise terms. \n",
+ "The results presented here will also be linked back to the data in a next step, so as to individual topics can be read and analysed at article level. The entry or basic foundation of a rough topic based on the topic modelling shown here can be refined in various ways, but remains as a sort of mind map enabling oneself to find one's way \n",
+ "along the topic within the researched corpus again and again. \n",
+ "\n",
+ "The exemplary workflow presented here is already serving as basis of an article and several lectures and presentations. \n",
+ "\n",
+ "This topic modelling workflow inspired by the _Newseye_ project will also be offered as a service of the Vienna University Library in collaboration with the Vienna University computer center in the context of the Transkribus service. The JupyterHub as service infrastructure has already been set up and the workflow is ready to be loaded with your own data or corpora. Please contact us if you are interested: [phaidra.topicmodel@univie.ac.at](mailto:phaidra.topicmodel@univie.ac.at)\n",
+ "\n",
+ "### Links\n",
+ "\n",
+ "* [Gensim](https://radimrehurek.com/gensim)\n",
+ "* [JupyterHub](https://jupyter.org/hub)\n",
+ "* [Littlest JupyterHub](https://tljh.jupyter.org/en/latest/)"
+ ]
}
],
"metadata": {
@@ -1955,7 +1984,11 @@
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
- }
+ },
+ "toc-autonumbering": false,
+ "toc-showcode": false,
+ "toc-showmarkdowntxt": true,
+ "toc-showtags": false
},
"nbformat": 4,
"nbformat_minor": 4