AI for Digital Research Services: Stanford Libraries AI Initiative Report

Librarians are trained in the arts of preparing a feast of knowledge. The library’s archives and collections are, in the words of Lorraine Daston, the provisions upon which they draw. The work of the digital library has been to attend to the essential and challenging infrastructural tasks of digital storage, preservation, information exchange, and online presentation. Now that the catalog is online and digital surrogates of physical artifacts are the initial point of contact with collections and archives, the instruments that librarians use to work directly with the content need to be updated as well. The tools and techniques of artificial intelligence (AI) now make it possible to build the fine instruments that augment the day-to-day work of librarians and the researchers they serve.
Researchers are already using AI techniques to get beyond the cover and into resources to unlock entirely new layers of information to mine. Some examples are given in Section 2 of this report. Over the past year (2018-19) Stanford Libraries staff participating in the Digital Research Soundings Group met as a community of practice, across subject domains and areas of expertise, to explore how to keep pace with research needs and develop new recipes for inquiry. Subsequently, library staff who participated in the Stanford Libraries AI Studio surfaced several opportunities to develop the essential tools of the library — descriptive metadata, collection development, resource integration, finding aids, and field guides — bringing them in line with the digital era. What we learned about how to mobilize this new technology in the library is covered in Section 3.
Machine learning is a rapidly developing capability that makes it possible to process tremendous amounts of data algorithmically to discover patterns. Deep learning (also known as multi-layered artificial neural networks), a sub-field of machine learning, amplifies the machine perception underlying language translation, transcription, OCR, and object identification in images, making it possible to analyze more and more complexity. Where we once thought of the digitized photograph as a digital object that can be duplicated, distributed, layered, annotated, and compared, we must now also recognize that individual image as a data set at the pixel level: a source of information that can be machine read. This paradigm shift complicates our understanding of the digital object from a library perspective, but it also opens up new possibilities for managing those objects. The opportunities and risks of applying AI in the library are addressed in Section 4.
Data, whether gathered from clicks on consumer web sites, 20th Century climate studies, or mined from ancient texts, is the fuel of AI. But AI is not magic, and it is not intelligent. Human expertise is required to select and prepare the data we consume. Libraries are discriminating stewards of knowledge and cultural heritage ideally suited to wield the power of AI to identify what is relevant and provide the intellectual nourishment that is so urgently needed. Section 5 offers strategies for exploring and explaining content in exciting new ways, putting the powerful classification capabilities of AI to work to augment the fundamental work of the library while helping to reveal new patterns and provide greater access, not only to what researchers have produced but how they have produced it.
“Beyond 100” has focused our attention on how Stanford Libraries seeks to anticipate the needs of tomorrow’s scholars. Michael Keller’s co-sponsorship of the first conference on artificial intelligence in libraries, “Fantastic Futures” (Oslo, December 2018), marks the place of AI on that agenda. The conference took place at a moment when many of the technology companies that develop and deploy artificial intelligence are coming under scrutiny due to algorithmic and data bias. This is a crucial time for libraries to take the lead in ensuring that information and the preservation of knowledge are in trusted hands.