IASA 2019 Annual Conference

Many organisations have trouble getting an overview of their data. A particular challenge for the archival community is that their data spans a large time period, during which systems and policies for creating metadata have changed, and even the meaning of the metadata itself, as language evolves. Archivists often struggle to answer questions such as: How many hours of television do we have? What percentage of the material is digital? What content do we have relating to migration? This information is essential to manage archival processes such as digitisation, and to select relevant content for individual users (recommendations) and for exhibitions. Scholars and journalists accessing the archive also need an overview of the type and content of material available, and the completeness of its metadata. At the Netherlands Institute for Sound and Vision, we have developed a data science approach for creating overviews of archive collections. We use Jupyter Notebooks, 'live' documents containing code, visualisations and narrative text. With these notebooks, we provide an up-to-date 'shop window' overview to interested parties, while allowing users with programming skills to flexibly create their own overview. Most importantly, we embed knowledge of the archive history and processes in the notebooks, to correctly assemble data fragments into the big picture. In this paper, we describe how our approach has been successfully applied to create overviews for diverse archive collections, including our own. We will discuss how these overviews support users in their work, and present the results of evaluations with scholars and archivists.