Engaging the UK Web Archive for social science research

(Left) Spatial distribution of crawl frequencies of the Hyperlocal News collection at the UKWA, (Middle) Screenshot of recently UKWA faceted search interface, (Right) Distribution of search results in UKWA over time using Shine search interface - "immigrant" (blue), "immigration" (green), "migrant" (orange) and "migration" (blue).

Executive Summary

This project was the result of the ESRC’s National Centre for Research Methods funded placement fellowship: Research challenges for using the archived web for social research. Based at the UK Web Archive (UKWA) at the British Library, the placement fundamentally sought to engage with the web archive as a source for social science research. Over the course of the three month placement, the access constraints surrounding the UKWA’s mandated mission presented numerous challenges for researcher engagement. This report charts these challenges and constraints by first contextualising this project within the wider landscape of digital secondary/data use in the social sciences, web archival researcher engagement, and the body of work that has been undertaken by researchers specifically engaged with the UKWA over time. The beginnings of a case study for using the UKWA News and Hyperlocal News Collections is presented, alongside a vignette of similar research conducted using live Web resources for comparison.

A conceptual framework for scholarly engagement with the UKWA is presented through the lens of my own research experience of developing a case study using the Hyperlocal News collection. Through these conceptual devices, I reflect on the various processes associated with orientating, auditing and constructing a corpus for web archival research. The processes of contextualising the Hyperlocal News collection through mapping curator and crawling activities, and comparing the numerous interfaces for using the UKWA established the important sociotechnical impacts of both human and technical interventions on the archives. Observations regarding the challenges of doing social science research in this context are discussed, including the experience of data friction, the allure of computation and obstacles for contextualising web archives at scale. In conclusion, some recommendations for facilitating researcher engagement are briefly described including the ability to collate links from search results, a clear overview of documentation describing available resources for researcher use and the creation of a programme to facilitate collaborative, interdisciplinary web archival research at the UKWA.

Project Outputs

  • Research Note in Methods News, a publication of the ESRC National Centre for Research Methods, 2019, Volume 1 [Publication]