Content Display User Research Methodology

Research Question

The task of the Content Display Functional Requirements group, within the context of the DLP Discovery Phase, was “to provide requirements and user needs related to the display of content in the repository.”

The following use cases* were established to frame the research question in more granular detail:

1. As a repository end-user, I want to discover content stored in the repository by searching or browsing, so that I can learn about materials stored there which may support my research.
2. As a repository end user, I want to preview the details of a repository resource to determine its appropriateness for my research, so that I know whether or not to download/view the material in greater detail.
3. As a repository end user, I want to view/play/search the actual contents of the material that I discover in the repository, in order to support my research.
4. As a repository end user, I want to understand download options that are available for the contents of the material, so that I can determine if I can download the material for my own use.
5. As a repository end user, I want to be able to cite the material that I find in the repository, so that I provide attribution in my research or share the materials with others.

*Based upon FRBR use cases (find, identify, select, obtain) for bibliographic data systems; MODS metadata levels of adoption use cases (five general categories of user functionality)

Background

A prior study, entitled “The DLP User Research Report,” was completed in summer 2015. This study recruited users and gathered impressions about current state digital library applications. This documentation provided background on Emory Library customer attitudes toward current digital library content and services, as well as experience of recruiting users and conducting user interview sessions.

Working Group Membership

The working group was composed of a cross-departmental team targeted at the various facets of this research question. The position titles of the members and their associated divisions are as follows:

1. Digital Projects Librarian, Library Technology & Digital Strategies (Convener)
2. Anthropology Librarian, Services
3. Senior Graphic Designer, Exhibits
4. Head of Research Services, Rose Library
5. Digital Photography Coordinator, Content
6. Senior Scholarly Repository Specialist, Scholarly Communications
7. Lead Web Developer, Library Technology & Digital Strategies*
8. UX Librarian, Library Technology & Digital Strategies*
9. Project Manager, Digital Library Program*

*represents consulting member

Protocol Design

Designing a study protocol to satisfy the research question was posed a challenge, as there is a surfeit of published literature related to this topic. At present, Emory Libraries does not offer a comprehensive access point to its unique digital collections. The DLP UX study provided some background in assessment for the current state applications in this area.

Given available resources and our problem set, we opted to perform open-ended, interface-driven interviews with our recruited users, following a predefined script but allowing for serendipitous interface exploration. This served as a practical solution for which solely front-end interaction was required. Furthermore, this provided simple access to real, diverse digital library content.

In order to target the research objective, we created six protocols, the full text of which are available on the DLP Project Wiki.

Our six protocols revolved around a similar base of demographic questions, but we themed each around a different discovery goal or content type interaction. This variability also allowed us to expose users to different interfaces. Each interview involved at least two separate digital libraries, and comparative impressions were sought in each session.

Recruitment

The group began recruitment with a discussion of the stakeholders belonging to each member’s division. In the course of this conversation, we identified several user groups from which we could recruit interview participants.

In order to extend the reach of our interviewee pool, we queried the Research and Instruction Librarians group, which has exposure to a large section of library customers. We also leveraged in-person meetings of the DLP Stakeholders Group in order to reach a larger group of Emory patrons. The resulting 20 interviewees fell into the following demographic segments and affiliated campus areas:

1. Art History
2. CFDE
3. ECDS
4. Film Studies
5. Goizueta Business Library
6. School of Law
7. Candler School of Theology
8. Political Science
9. Rose Library
10. Laney Graduate School
11. Rollins School of Public Health
12. Teaching and Learning Technologies
13. Woodruff Research Engagement Services

Interface Selection

In the interest of exposing users to the widely variable and highly customizable Samvera framework, the group identified several live implementations at various institutions as the objects of study. Each deployed a version of Blacklight, part of the Samvera stack, as the main search and discovery interface. We further refined our focus to interfaces that provided a federated discovery environment (meaning that various types of content were searchable in one access point). The five objects of study included:

1. Digital Commonwealth (digitalcommonwealth.org)
2. UCSD Digital Collections (library.ucsd.edu/dc)
3. Tufts Digital Library (dl.tufts.edu)
4. UCSB Alexandria Digital Library (alexandria.ucsb.edu)
5. Hull (hydra.hull.ac.uk)

Interviews

Interviews took place in the spring of 2017. For each session, the working group convener acted as facilitator. In some cases, another working group member contributed as note taker. All sessions recorded audio and screen activity. The 20 interviews yielded approximately 12 hours of footage.

Analysis

The group shared the task of analyzing session footage for relevant findings. Each member was assigned a number of sessions and provided a worksheet to aid in capturing impressions. The worksheet was aligned to four main target areas: search and indexing needs; filter sort, facet, browse, and navigation needs, user interaction with content, and metadata/record display.

After the conclusion of initial analysis, each group member’s grouped impressions were consolidated into a spreadsheet. This data is available on the project wiki, with user impressions linked to session numbers and user segments.

Summary of User Research Findings

The broad group of users interviewed for the study had a consistently positive response to the benchmark interfaces, and to the prospect of having a similar resource to explore unique digital material at Emory. The study provided a strong foundation of insight into how users interact with the Samvera stack, primarily the Blacklight discovery interface. Overall, this search and discovery functionality was well-received, and seemed to suit user needs quite well. However, implementation and configuration is highly flexible for Blacklight, and some user confusion seemed to result from how various institutions tailored it to their local environment. DLP will work in close consultation with users to ensure that applications are suitably customized for usability.

Our findings and deliverables were also prepared in collaboration with the DLP Metadata Implementation Working Group. The Content Display group reviewed the user-facing elements of our Descriptive and Technical Metadata specifications with the needs of our respective user bases in mind. The Metadata group also reviewed the raw findings data from the Content Display group and found that metadata-oriented findings align with the metadata specifications DLP will provision for the repository.

While generalized findings are presented below, the full range of study data is available in the form of recorded user sessions, and the spreadsheet analysis of raw data.

Findings were associated with the following high-level use cases:

1.     As a repository end-user, I want to discover content stored in the repository by searching or browsing, so that I can learn about materials stored there which may support my research.

Given the strong emphasis on search and browse activity in user research protocols, research yielded the greatest number of insights in this area. Users found the variety of Blacklight implementations they saw to be largely sufficient. Across the variety of user types interviewed, the need for robust searching, both simple and advanced, was widely expressed.

Intuitiveness of search and browse experiences was a general theme throughout. Within search result lists, users often questioned how default “relevance” rankings were determined, and wanted simple ways to reorder results according to standard sort parameters. Users also desired clear UI indicators of how combinations of facet limitations affected displayed results. They also expressed the need to interpret when a new entry into the search bar begins a new search, or one with the present parameters already applied. Finally, users appreciated numerical indicators for the number of items categorized within specific display facets.

Specific fields and metadata elements were also of demonstrated importance. Users wanted to be able to search or refine by format. Creator, language, and date of creation were also widely expressed to be essential discovery fields. Certain field labels also caused confusion, and should be addressed with further testing. These included “topic” and its relationship to “subject,” “genre,” and “contributor.” Additionally, interactions with users revealed that their own terminology at times conflicts with librarian terminology, so future state products would benefit from guidance and documentation.

In a larger context, users also desired clear pathways between the Digital Library and the main library website. Previous research has shown current state Emory digital library applications to be difficult to locate via current website navigation.

2.    As a repository end user, I want to preview the details of a repository resource to determine its appropriateness for my research, so that I know whether or not to download/view the material in greater detail.

The repeated search and browse activity emphasized in the protocol also exposed users to search result lists numerous times. In Blacklight, the metadata fields and other display characteristics of item previews are highly customizable. In most cases, users were satisfied with the information presented in these previews. One consistent requirement was an indication for item type, which is a logical request for repositories containing heterogeneous content.

Several user impressions merit further consideration or testing. In some cases, users were confused by general or abstract item titles. A larger scope question may consider what user expectations surround this field, and how explanatory they expect it to be. Repositories also employed a number of different default thumbnail sizes for A/V material. In some cases, users found specific examples too small. In some cases, they were too large. Determining an acceptable size, or an easily customizable interface, is a potential endeavor for testing in future project phases.

3.     As a repository end user, I want to view/play/search the actual contents of the material that I discover in the repository, in order to support my research.

The Samvera stack does not designate a particular playback or book reader solution, and the selected benchmark interfaces exhibited a number of different solutions. For both book readers and A/V viewing, users seemed satisfied with the solutions offered. Most of the players incorporated the minimal requirements users seemed to expect. For images, zoom and rotate features were desired. For A/V players, scrubbing and timestamps were. When transcripts of A/V material were available in full on the item detail page, users reacted positively. Some critique was leveled at players whose UI elements were small or hard to discern, so that is an issue to keep in mind. Responsiveness and speed was also something users valued, especially for classroom presentation contexts.

For book readers, several of the users that consistently read texts online remarked that the ones in the study worked well in comparison to the ones that they were familiar with. Indications of page status, total pages, and easy access to a specific page were basic desirable features, as was consistency between page numbers in the text and in the book reader software. Keyword searching and embedded tables of contents were more elaborate features desirable when they could be implemented.

Overall, the choice of viewers and players for the end-user interface will depend on a number of internal factors, as well as any potential developments in the larger Samvera community. We will continue to monitor emerging requirements and solutions in future phases and test them with users in order to arrive at a final selection.

4.     As a repository end user, I want to understand download options that are available for the contents of the material, so that I can determine if I can download the material for my own use.

Within the interview scenario, users did not express many strong preferences around the download of material. Whereas in previous research relating to Emory’s preservation repository, users expressed questions about how to download material, this sentiment was not often expressed in the Content Display sessions, unless prompted by the facilitator. Each interface presented a fairly straightforward download capability, some with multiple resolution or size options. When questioned, users expressed satisfaction with this functionality and did not express preferences for any specific download features, or item specifications.

5.     As a repository end user, I want to be able to cite the material that I find in the repository, so that I provide attribution in my research or share the materials with others.

Users were also directly asked about citing material from the repository for reuse in academic contexts. They generally found that benchmark repositories provided sufficient metadata to cite material. When presented with prepared citations, users almost unanimously reacted positively, so this is a feature that should be strongly considered for incorporation. Rights and citations are broad areas that merit further study, and future phases should create targeted test scenarios on Emory-developed interfaces for more detailed study.

Future Efforts

The deliverables of the Content Display group represent a baseline exploration of user-centered design practices. Our findings will form the background for iterative, participatory design work during the implementation phase of the DLP Project, as well as kick-starting initial user interface prototypes for implementation. A major outcome of this working group is the general impression that users are interested in the unique digital research collections of Emory Libraries, and that they are willing participants in the design and development of a new access point.