Skip to content

Summers-Knoll.org Usability Testing

November 1, 2010

In preparation for usability testing on Summers-Knoll School’s digital student portfolios and the web-based version of the Work Sampling System, I conducted brief usability tests on Summers-Knoll.org. This exercise was designed more to develop my skills as a test facilitator than with generating especially valuable data for SK, particularly since SK’s web site is soon to be restructured to allow for user login capabilities including parental access to online student portfolios.
______________________________________________________________________

A Closer Look at Summers-Knoll.org: A Study in Usability and Perception

Abstract: Assessing the Need for Usability Testing

Summers-Knoll School (SK), located in Ann Arbor, Michigan, is a kindergarten through fifth grade school committed to cross-curricular, experiential, and inquiry-based learning. As the founders of SK were key players in the field of emerging design technologies of the 1990s, the school was established with a secondary focus of multidisciplinary technology integration. Today, SK remains on the cutting edge of instruction, including the use of technology. Despite this, the school still struggles with how to make its web site more compelling and interactive, how to change it from an information depository to a relevant and dynamic destination site.

With the aid of graduate students from the School of Information at the University of Michigan, SK is currently addressing ways to design a user portal at the school’s site. An educational technology team is also designing the blueprint for eventual construction of digital, multimedia student performance portfolios to be accessed by students and parents at Summers-Knoll.org. The usability test I created for this report addresses the usefulness and look and feel of Summers-Knoll.org in its current iteration.

Introduction & Background

Authentic Integration of Technology at Summers-Knoll School

Summers-Knoll (SK) is an independent, kindergarten through fifth grade, experiential learning-based school located in Ann Arbor, Michigan. While the school’s primary focus is on cross-curricular, balanced liberal arts education, the integration or mainstreaming of technology into teacher instruction, student learning, and communication is an integral ingredient of the school community’s collective ethos. Originally founded by a small team of educators and parents including Tom Knoll, lead developer of Adobe Photoshop, and his wife Ruth Knoll, Summers-Knoll has been well equipped with current hardware and software since its inception in 1996. The school’s commitment to integrated technology instruction, as well as building and maintaining a dynamic and ever-evolving infrastructure, make SK a singular institution among the myriad of K-5 choices in this university town.

SK’s unique approach to technology lies in the general philosophical understanding of all community members (parents, faculty, staff, etc.) that technological tools — whether a hammer and nail or a laptop computer — do not exist in isolation. Rather, the tools SK community members use in solving problems are context dependent. Approaching technology from this position allows teachers at SK to seamlessly integrate a variety of contemporary technologies into multidisciplinary lessons and activities. Technology is not a discipline. Instead, it provides a palette of tools for real-world problem solving rooted in traditional learning objectives.

The Problem: Is Summers-Knoll.org Useful?

Despite Summers-Knoll’s progressive philosophical and methodological approaches to technology integration, the school’s web site has faced challenges in adequately representing SK’s ethos as well as establishing itself as a destination site for members of the community. Though Summers-Knoll.org has long been a valuable resource for prospective families and teachers, it has lacked usefulness or relevancy in further development of already-established relationships: once people become members of the community, they historically have had little reason to return to the school’s homepage. And although the homepage provides links to each teacher’s homeroom or special subject blog, the blogs are primarily accessed by parents and students directly rather than through the SK portal. Additionally, the bulk of parent/teacher, parent/staff, and teacher/staff communication is conducted via email. In the past, teachers have used a variety of web-based applications, both free and subscription-based, to manage community communications, but, though many of these tools proved adequate, none were directly linked to the SK site.

Summers-Knoll.org has existed in numerous iterations within four primary wireframe designs since its original launch in the mid ‘90s. Its current design is probably its most contemporary, aesthetically appealing, and accurate in its visual and textual reflection of the school’s philosophy and ethos. Despite its evolutionary improvements, however, its lack of authentic usefulness to the dynamic community renders it an eye-pleasing though somewhat superfluous knickknack. Below is a clickable screen shot of Summers-Knoll.org’s homepage.

Design augmentation of Summers-Knoll.org is currently in the planning stages. Accepted by the graduate program at the University of Michigan School of Information to be used as a project study, Summers-Knoll.org is expanding to include member login capabilities and access to user data including student assessment. Student digital portfolio wireframes are concurrently under construction and will be accessible by the SK portal beginning in fall 2011. By allowing users to create unique accounts and access and manage their community-related information, Summers-Knoll.org will position itself as a destination site and a necessary and useful tool for communication/information management.

Usability Testing of Summers-Knoll.org

Background

Concurrent with Summers-Knoll.org’s expansion planning, I conducted a small-scale usability test of the site in its present iteration. Though I have been a tester in other usability scenarios, this was my first experience as a facilitator, and my plan was based on Steve Krug’s do-it-yourself book, Rocket Surgery Made Easy, lecture and discussion from Dr. Steve Benninghhoff’s English 524 – Advanced Technical Writing and Research, and a dash of common sense and invention.

Test Design and Results

I chose remote testing for this project for two reasons. First, the bulk of my professional and educational work is done remotely in virtual contexts. Second, remote testing allowed me to sample a greater cross section of users from a wider socioeconomic spectrum with a greater array of racial, ethnic, political, and sexual/gender identifications. In short, remote testing allowed for the collection of a wider range of data. Further, testing remotely directly lent itself to easier recruiting, smoother scheduling, and elimination of travel concerns.

I began with an in-depth review of Summers-Knoll.org. I checked all links to ensure they were active and up-to-date, maneuvered across sections and pages to determine general ease of use, and used contact links/email addresses to communicate with teachers and staff. Further, I focused on the look and feel of Summers-Knoll.org in an attempt to quantify if the site projects an overall ethos aligned with SK community members and the school’s physical site.

Next, I scripted ten questions/problem scenarios based around Summers-Knoll.org. Some questions were more qualitative in nature, such as “What are your initial thoughts when viewing the homepage?” It’s through these types of questions I hoped to generate reflexive responses from users in regards to their perceptions of the school’s philosophy. Other questions, like “What curricular program does Summers-Knoll use for math instruction? Where did you find this information?” were designed with practical needs in mind. Potential members of the SK community are often deeply interested in nuts and bolts information, such as standards and curricula, so making this type of content easy to find is paramount for usefulness.

After designing a mix of questions/problems, I uploaded them to Surveymonkey.com. Surveymonkey allowed me to create a questionnaire that would record users’ written responses. I next recruited testers by telephone, most of whom had at least a peripheral interest in education and/or parenting. After successful recruitment of six participants ranging in age from 20 – 76, I followed up with these email instructions:

Over a five-day period, I met virtually with each of my six testers. I met four using the telephone, and two using Skype 5.0. Skype allowed me to interact visually with my testers and may have enhanced my note taking by providing me with real-time body language feedback from users. I would have preferred using Skype with all six users, though using the service with some and not others allowed for a comparison of the two different testing scenarios. Additionally, meeting with testers virtually allowed for the use of a variety of computers, operating systems, and web browsers. This provided me with the ability to compare performance/usefulness across a variety of platforms.

Each test, both the four relying on telephone communication and the two using Skype, followed a similar structure. I explained in detail to my testers that my objective was to analyze the web site and not them or their abilities. Each of my testers then opened two windows on his/her computer. One window was directed to Summers-Knoll.org and the other to Surveymonkey.com. Beginning at Surveymonkey.com, I read aloud the first question to each user and asked if they required additional clarification. Next, each user responded to the question based on his/her reactions at the SK homepage. The responses were recorded textually on Surveymonkey.com and also in accompanying notes I kept for each test user. Below is a clickable screen shot of one of my questions on Surveymonkey including the responses from my six users.

Lessons Learned

As evidenced by the responses to this and other more evaluative questions in the survey, the users represent a variety of backgrounds and influences. In an earlier survey question, participant 3 viewed the school as “multi-ethnic” while participant 4 suggested it’s “possibly non-diverse.” I believe both of these responses are influenced by each user’s socioeconomic status (one is a small business owner, the other unemployed) and geographic location (one is a member of a heterogeneous Southern community, the other resides in a diverse, urban city in the North). I was pleasantly surprised by the variety of responses I received based on user’s experiences and perceptions, or what Kenneth Burke calls “terministic screens.” Prior to the testing, I had not considered how such screens could affect the responses from the users, and perhaps most revealing to me was my own narrow scope of understanding in regards to the way people approach seemingly benign text and imagery. Further, I fully understand and agree with all responses, even oppositional ones. Each user’s response to each prompt allowed me as facilitator to understand each experience from a firsthand perspective. Overall, though the testing provided me with some potentially useful data on the current state of Summers-Knoll.org, I believe the greater benefit was how the exercise allowed me to experience multiple perceptions simultaneously. The experiment evolved into a lesson for me on how to communicate with users, how to ultimately relinquish control to them, and how to submit to their leadership within the boundaries of my loosely constructed framework. It was as much a social science experiment as a test of a particular web site’s usefulness and usability.

Praxis: Implications for Future Testing

As stated previously, this was my fist attempt at facilitating web site usability testing. It proved valuable in helping me organize the test, recruit test subjects, and navigate the numerous variables involved. Though I closely followed the suggestions of Steve Krug, I also developed my own set of best practices based on interactions with my six users. Overall, this was a positive introductory experience for me, and it will provide invaluable help in designing and implementing authentic, larger-scale usability tests of Summers-Knoll.org after the completion of the current design augmentations. This testing experience also prepares me for my planned winter 2011 testing of SK’s digital student portfolios. As one component of my Written Communication MA graduate project, my digital portfolio testing with both parents and teachers will now be informed by both theory and practice.

References

“History.” Summers-Knoll School. (2010, October). Web. Retrieved from http://www.summers-knoll.org

Krug, S. (2010). Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. Berkeley, CA: New Riders

Advertisements
2 Comments leave one →
  1. Anne permalink
    November 2, 2010 4:02 am

    JD,
    I am a parent from SK who was sent the link to your website so I could become familiar with your research on SK. I enjoyed reading through the SK.org Usability Testing portion of your blog. I too am interested in better understanding the perceptions of people who use the SK website.

    *I do have a question though, could you briefly explain “remote testing” to me?
    Thanks! Anne

  2. November 2, 2010 3:18 pm

    Hi Anne:

    A remote test means the participant and facilitator are separated by space (as opposed to being face-to-face and sitting in front of a single monitor). There are a lot of free screen sharing apps out there that enhance remote testing and other collaborative work. I’ve used DimDim and Jing.

    Thanks for your comment,
    John

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: