Surprise machines
Revealing Harvard Art Museums’ image collection
Surprise Machines is a project of experimental museology that
sets out to visualize the entire image collection of the Harvard Art Museums,
with a view to opening up unexpected vistas on more than 200,000 objects usually
inaccessible to visitors. The project is part of the exhibition organized by
metaLAB (at) Harvard entitled Curatorial A(i)gents and explores the limits of
artificial intelligence to display a large set of images and create surprise
among visitors. To achieve this feeling of surprise, a choreographic interface
was designed to connect the audience’s movement with several unique views of the
collection.
Article outline
- 1.Introduction
- 2.Harvard Art Museums
- 3.Can machines curate?
- 4.How to map 200,000 images
- 5.Designing a post-pandemic, choreographic interface
- 6.A substantial drawback
- 7.Conclusions
-
Bibliography
References (50)
Bibliography
– American Museum of Natural History. (2022). Collectionscope [JavaScript]. [URL] (Original work published 2020)
– Barabási, A.-L., Bello, M., Kluge-Fabényi, J., Forde, K., Készman, J., Meirelles, I., Ratti, C. G., Ritchie, M., & Szántó, A. (2020). Hidden patterns: Visualizing networks at BarabásiLab (A. Stang & P. Weibel, Eds.). Hatje Cantz Verlag.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Benedetti, A. (2022, January 5). Browsing and visualizing collections of images. [URL]
– Birkle, C., & Däwes, B. (2019). “Old media don’t go away, they mutate”: An interview with Jeffrey
Schnapp. Amerikastudien/American Studies,
64
(1), 111–125. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Bludau, M.-J., Dörk, M., & Heidmann, F. (2021). Relational perspectives as situated visualizations of art
collections. Digital Scholarship in the Humanities,
36
(Supplement_2), ii17–ii29. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Börner, K., Maltese, A., Balliet, R. N., & Heimlich, J. (2016). Investigating aspects of data visualization literacy using 20
information visualizations and 273 science museum visitors. Information Visualization,
15
(3), 198–213. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Bostock, M., Ogievetsky, V., & Heer, J. (2011). D3: Data-Driven Documents. IEEE Transactions on Visualization and Computer Graphics,
17
(12), 2301–2309. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Champion, E. M. (2016). Digital humanities is text heavy, visualization light, and
simulation poor. Digital Scholarship in the Humanities, i25–i32. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Crawford, K., & Paglen, T. (2019). Excavating AI: the politics of images in machine learning training
sets. AI Now Institute. [URL]
– Crockett, D. (2019). IVPY: iconographic visualization inside computational
notebooks. International Journal for Digital Art History,
4
1, 3.60–3.79. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Cuno, J. B. (Ed.). (1996). Harvard’s art museums: 100 years of collecting. Harvard University Museums.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Danchilla, B. (2012). Beginning WebGL for HTML5. Apress. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Derry, L., Kruguer, J., Muelle, M., & Schnapp, J. (2022). Designing a choreographic interface during covid-19. Movement and Computing Conference. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Diagne, C., Barradeau, N., & Doury, S. (2018). T-SNE Map. Experiments with Google. [URL]
– DiMaggio, P., & Hargittai, E. (2001). From the “digital divide” to “digital inequality”: Studying internet use
as penetration increases. Center for Arts and Cultural Policy Studies, Princeton University.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Drucker, J. (2013). Performative materiality and theoretical approaches to
interface. Digital Humanities Quarterly,
7
(1). [URL]
– Duhaime, D. (2021). PixPlot. Yale Digital Humanities Lab. [URL] (Original work published 2017)
– Foster, H. (2011). The art-architecture complex. Verso.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Geismar, H. (2018). Museum object lessons for the digital age. UCL Press. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Harvard Art Museums. (2012). International Image Interoperability Framework at Harvard
University. [URL]
– Impett, L., & Moretti, F. (2017). Totentanz. Operationalizing Aby Warburg’s
Pathosformeln. New Left Review,
107
1, 68–97.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Jacomy, M., Venturini, T., Heymann, S., & Bastian, M. (2014). ForceAtlas2, a continuous graph layout algorithm for handy
network visualization designed for the Gephi software. PLoS ONE,
9
(6), e98679. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Jodidio, P. (2014). Piano: Renzo Piano building workshop, complete works,
1966-today. Taschen.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Karjus, A., Solà, M. C., Ohm, T., Ahnert, S. E., & Schich, M. (2022). Compression ensembles quantify aesthetic complexity and the evolution of
visual art. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Kenderdine, S., Mason, I., & Hibberd, L. (2021). Computational archives for experimental museology, 3–18. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Klinke, H. (2021, October 28). V-lab workshop: Visual analysis in cultural data – image plots and t-SNE
maps made easy. History of Art Department. [URL]
– Kräutli, F. (2016). Visualising cultural data: Exploring digital collections through
timeline visualisations. Royal College of Art.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Latour, B. (1988). The pasteurization of France (A. Sheridan & J. Law, Trans.; English edition). Harvard University Press.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Lima, M. (2011). Visual complexity: Mapping patterns of information. Princeton Architectural Press.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Maaten, L. van der, & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research,
9
(86), 2579–2605. [URL]
– Maizels, M., & Qiu, C. (Eds.). (2020). Curatorial A(i)gents. metaLAB (at) Harvard. [URL]
– Manovich, L. (2008). Data visualization as new abstraction and
anti-sublime. In B. Hawk, D. M. Rieder, & O. O. Oviedo (Eds.), Small tech: The culture of digital tools. University of Minnesota Press.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Manovich, L. (2020). Cultural analytics. MIT Press. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Manzini, E. (2016). Design culture and dialogic design. Design Issues,
32
(1), 52–59. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– McCully, E. A. (2019). Dreaming in code: Ada Byron Lovelace, computer pioneer. Candlewick Press.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– McInnes, L., Healy, J., & Melville, J. (2018). UMAP: Uniform Manifold Approximation and Projection for dimension
reduction. ArXiv.Org, stat.ML. [URL]
– metaLAB. (2022). MetaLAB (at) Harvard & FU Berlin. [URL]
– Moon, C. Y. E., & Rodighiero, D. (2020). Mapping as a contemporary instrument for orientation in
conferences. Atti Del IX Convegno Annuale AIUCD. La Svolta Inevitabile: Sfide e
Prospettive per l’informatica Umanistica. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– O’Shea, K., & Nash, R. (2015). An introduction to convolutional neural networks. ArXiv:1511.08458 [Cs]. [URL]
– Pietsch, C. (2022). VIKUS viewer [JavaScript]. [URL] (Original work published 2018)
– Rodighiero, D. (2021a). Mapping affinities: Democratizing data visualization (Open-access English edition). Métis Presses. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Rodighiero, D. (2021b, August 18). Ars memorativa as the genesis of information design: A
conversation with Manuel Lima. Nightingale. [URL]. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
– Rodighiero, D., Wandl-Vogt, E., & Carsenat, E. (2021). Making visible the invisible work of scientists during the
Covid-19 pandemic. Visual Culture Studies,
2
1. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Rodighiero, D., Wandl-Vogt, E., & Carsenat, E. (2022). A visual translation of the pandemic. Leonardo,
55
(3). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., & Fei-Fei, L. (2015). Imagenet large scale visual recognition challenge. ArXiv:1409.0575 [Cs]. [URL]. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
– Seguin, B. (2018). The Replica project: Building a visual search engine for art
historians. XRDS: Crossroads, The ACM Magazine for Students,
24
(3), 24–29. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Steward, J. (2021). API documentation. Harvard Art Museums. [URL] (Original work published 2015)
– Turing, A. M. (1950). Computing machinery and intelligence. Mind,
LIX
(236), 433–460. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Vane, O. (2019). Timeline design for visualising cultural heritage data. Royal College of Art.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
– Weaver, W. (1948). Science and complexity. American Scientist,
36
(4), 536–544. JSTOR. [URL]
Cited by (2)
Cited by two other publications
Dhaenens, Laurens & Frederik Truyen
2024.
Art in circulation: Creating content and context for digital reproduction of artworks.
Curator: The Museum Journal ![DOI logo](//benjamins.com/logos/doi-logo.svg)
Yang, Yuchen & Linyida Zhang
2023.
Proceedings of the 5th Workshop on analySis, Understanding and proMotion of heritAge Contents,
► pp. 5 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 4 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.