Skip to content

Grad Project Proposal Draft

December 5, 2010

Introduction: Capturing Authentic Assessment of K-5 Student Performance

Measuring student performance in K-12 education has been at the forefront of public debate and policy since Puritan settlers established the first public school in Boston in 1635 (Seybolt, 1935). The concern over K-12 academic achievement has reached heightened levels of attention in recent years due to a variety of factors including declining test scores in core content areas such as math, reading, and science; increased socioeconomic disparities among members of the working and middle classes; and an international sociocultural shift to a hyper-competitive, information-based economy. One popular and far-reaching response to the new challenges faced by K-12 educators, education policy makers, and other education community members has been increased quantitative student assessments, especially in the form of standardized tests. Results from these high stakes measurement tools have been used to categorize students into rigid learning classifications; drive curricula design and instructional methodology choices; and promote and demote teachers, schools, and even entire school districts. While there are possible benefits of summative student assessments — they may promote abstract problem solving, for example — the data they capture ultimately illustrate a single performance in a single, specific context without regard to a host of potential variables including, but not limited to:

  • English language proficiency
  • Overall physical health of student
  • Mental health of student
  • Testing site conditions
  • Learning style and preference
  • Test taking preparedness
  • Testing of knowledge instead of ability
  • Gender
  • Socioeconomic status

An additional drawback of relying on standardized testing to reward or punish students and teachers and drive curricula development and implementation is that high stakes tests rank students against each other instead of themselves. Such tests capture knowledge-based scores, but offer no illustration of student growth and achievement over time. I posit that application of a balanced assessment program, including narrative-based evaluations and multimedia portfolios of student work, offer both quantitative and harder-to-measure qualitative data reflective of learner knowledge and performance. Further, the use of multimedia, or digital, portfolios positions students in the driver’s seat of their own education. Through reflective selection of works demonstrating peak performance as well as improvement over time, students become active, engaged, and empowered participants in their own learning and growth. Additionally, I believe that qualitative performance data captured in narrative assessment reports can be reframed or reinterpreted to meet the more quantitative parameters of traditional educational benchmarks.

Background: Alternative Practices in K-5 Curricula, Instruction, and Assessment at Summers-Knoll School

Prior to entering my current position as an editor of training recommendation reports for the US Navy, I spent six years as a homeroom teacher at Summers-Knoll School (SK) in Ann Arbor, Michigan. SK is an independent, kindergarten through fifth grade school that focuses on traditional liberal arts content delivered in a cross-curricular, experiential format. Through embedded learning and developmental objectives and qualitative performance assessment, students’ energies are channeled into how to learn as opposed to what to learn. Students and families are involved in curricular decision making, and they also play active roles in assessing student performance through the use of the school’s two primary assessment tools: Work Sampling System (WSS), a narrative-based assessment tool developed by the University of Michigan, and portfolios. I taught grades one through seven (the middle school program has since disbanded) at SK from 2002 to 2008, and I remain committed to the school’s educational philosophy and mission.

SK practices progressive instruction and student evaluation. Parents of students at SK become familiar with the school’s philosophy and practices during the in-depth application process. Despite the parent buy-in, or understanding, of SK’s teaching methodologies, a minority of parents expresses concern that skills developed at SK may not be transferable to more conventional learning environments. This concern appears to heighten as students rise through the grades and middle school looms. Though the school offers annual, optional, standardized testing to students in grades 4-5, some parents would prefer a balanced assessment approach that incorporates norms-based measures. While this is contrary to the philosophy of SK, I believe that quantitative performance data can be generated if SK switches to digital portfolios and the web-based WSS assessment tools. I posit that these data can be — and already are — generated via the current assessment tools. I believe digital portfolios and web-based WSS assessments will offer reinterpretations of student performance data that can be captured in quantifiable terms. By moving student assessments onto digital platforms, SK can remain faithful to the mission of its founders, incorporate technology into instruction and assessment in meaningful ways, and meet the needs and wants of parents with seemingly conflicting agendas.

The Problem and the Plan: Moving Summers-Knoll School’s Assessment Tools to Digital Platforms

SK is currently working with graduate students from the University of Michigan’s School of Information to develop a digital student portfolio wireframe and a working prototype. The head of school at SK, Joanna Hastings, has asked me to participate in the project by working on a team comprised of educators and designers to develop the communication component of this venture, facilitate iterative usability beta testing on the digital portfolios, and conduct secondary research into the implementation of digital portfolios in K-5 settings and their ability to capture quantitative student performance data via qualitative means. Further, I will facilitate usability testing on the school’s own website, to which the digital portfolios will be linked via a secure parent / teacher / student user portal, and evaluate WSS’s web-based version (teachers currently produce WSS evaluations in hardcopy) through additional usability testing and teacher and parent interviews. Upon approval from EMU’s University Human Subjects Review Committee, my data collection will begin in winter 2011, and I will wrap up my study by early summer 2011. Little research exists on digital portfolio implementation and assessment in elementary school settings, and I hope to contribute meaningfully to this burgeoning field of inquiry.

Methodologies

My project includes pre-study planning meetings with SK’s head of school, Joanna Hastings; University of Michigan School of Information graduate student wireframe and prototype developers Charity Ben-Daniels, Erin Dietrich, Thomas Piggott, Megan Schwarz, and Kathryn Totz; SK’s contracted web designer, Anjanette Bunce; and members of SK’s technology committee, an amalgam of K-12 and post-secondary educators from Summers-Knoll School, Eastern Michigan University, and University of Michigan. The design of my project will be informed by postmodern critical theory. Rather than taking a detached, descriptive, explanatory approach to my research, I will develop a co-learner partnership with my human subjects. The collaborative framework within which I will carry out my research will attempt to establish an egalitarian distribution of influence and power. Taking a critical perspective will allow me to work “with and not on a group” (McLaren, 154).

To collect relevant data on digital assessment portfolios I will conduct iterative usability testing on the UM SI-designed portfolio prototype. Hypothetical student data will be loaded into the digital portfolio prototype and usability testing will be conducted with a small team of SK teachers and parents. The results of my tests will be conveyed to web-designer Anjanette Bunce who is taking over primary portfolio design responsibility after the prototype has been delivered in December 2010. Ongoing usability testing and interviewing with teachers and parents will directly inform Anjanette’s design modifications to ensure development of a product that best meets the needs of the three primary user groups: students, parents, and teachers.

To collect relevant data on the web-based version of the University of Michigan’s WSS narrative assessment tool, I will conduct interviews with teachers and parents to determine the strengths and weaknesses of the current paper and pencil version used by SK. My interviews will also probe into respondents’ philosophies on K-5 education, experiential and other non-traditional learning methods, and qualitative and quantitative assessment. I will then create a virtual classroom of eight to ten multicultural personas and load the web-based version of WSS with hypothetical performance data of this group of students. Data will reflect a range of experiences, learning styles and preferences, socioeconomic backgrounds, nationalities and language proficiencies, and academic strengths and challenges. These hypothetical, though authentically reflective, data will be presented to teachers and parents via usability evaluations. I believe that the web-based WSS will allow the qualitative-leaning data to be framed or positioned in such a way that better satisfies the wants and needs of parents interested in quantitative analysis of student performance.

Data Analysis

Data will be collected through audio taped interviews of parents of K-5 students at SK, three K-5 homeroom teachers, and four specials teachers (music, art, physical education, Latin/French). All human subjects will be volunteers and each will be provided with a typed transcript of the interview after completion.

Further data will be collected through audio taped usability testing on at least one and possibly two beta versions of digital portfolios. Participants will be the same teachers and parents as in the interviews. Data will be evaluated via a combination of content analysis and narrative analysis methods.

Literature Review

A prelude to my project design included conducting a literature review of scholarly research on portfolio assessment. Though my project focuses specifically on digital, or electronic, portfolios, I expanded my reading to include the use of traditional portfolios in K-12 and post-secondary education as well. Foremost in my review was searching for a clear explanation of what a portfolio is. Ocak and Ulu offer a concise definition of the primary-level portfolio calling it “a carefully selected collection of student work that provides clear evidence to the student, parent, and…educators of the student’s knowledge, skills, strategies, grasp of concepts, attitudes, and achievement in a given area over a specific time period” (2009, p. 28). Peacock, Gordon, Murray, Morss, and Dunlop provide a digital-specific definition in their 2010 study: “An ePortfolio is an electronic system that facilitates the development, collection, and management of digital resources which may be drawn from a range of learner experiences over a period of time and could include those from formal and non-formal learning opportunities” (p. 828). This interpretation may be especially applicable to experiential and cross-curricular learning by suggesting that digital, or ePortfolios, are capable of capturing development and achievement that occurs outside of conventional scenarios, including, as Wall, Higgins, Miller, and Packard point out, art, physical education, design, and technology (2006). Perhaps the most succinct definition comes from Meyer, Abrami, Wade, Aslan, and Deault who call the electronic portfolio “a digital container capable of storing visual and auditory content including text, images, video, and sound” (2010, p. 84). By including the digital portfolio’s multimedia capabilities, the authors highlight another key benefit of electronic performance showcasing.

In searching for clear-cut descriptions of digital portfolios that would help me develop my own informed understanding, I was struck by the lack of consistency in terminology across the literature. Alternately referred to by Chang as web-based learning portfolios (WBLP) (2001), Meyer et al. as electronic portfolios (EP) (2010), and Vermilion as Efolios (2008), the naming or development of a lexicon appears secondary to charting and tracking the primary functions and purposes of digital portfolios.

The documented benefits of digital portfolio assessment are numerous. Among the most interesting to me and relevant to my proposed study with SK is Chang’s assertion that portfolio assessment identifies student skills rather than shortcomings, identifies the unique needs of each student, increases student motivation, and encourages students to develop decision-making skills (2008). This is in stark contrast to more traditional summative assessment tools prevalent in lecture/test models that focus on memorization and context-specific performance. Chang’s research also indicates that digital portfolio assessment may be beneficial to students with low motivation and/or self-esteem, underperforming students due to at-risk-classification, and special needs students including twice exceptional and gifted learners (2008). Wall et al. further suggest that digital portfolios are an appropriate method for recording achievement of pupils who find writing difficult (2006). In other words, portfolio assessment may meet the unique needs of learners whose talents sometimes fall outside the confines of the bell curve.

Lam and Lee’s research study focused specifically on how digital portfolio assessments can facilitate quantitative achievement in English as a second language (ESL) and the language arts. After implementation of portfolio assessment in the authors’ ESL classroom, the data indicate that students’ writings were more grammatically and structurally accurate and that students generated more and better ideas in their assignments (2010). Vermillion reports anecdotal evidence that digital portfolios improve skills in technology, public speaking, and leadership (2008). And Hewitt’s seminal 1995 research study on portfolio assessment, as reported by Chang, uncovered the following seven benefits:

  1. Portfolios demonstrate students’ growth
  2. Portfolios encourage students to set up learning goals
  3. Portfolios provide evidence of students’ efforts
  4. Portfolios demonstrate students’ performance
  5. Portfolios help teachers review students’ performance
  6. Portfolios stimulate students’ introspective thinking and enhance self-assessment
  7. Portfolios encourage students’ learning interests, improve their communication skills, and build self confidence (2001, p.436)

While all of the above provide obvious advantages to learners and teachers alike, the stimulation of introspective thinking is especially interesting to me. One of the precepts of contemporary education studies is that reflective learning results in increased retention and achievement; therefore one can draw a direct correlation between Hewitt’s list of benefits and authentic, measurable learning. This is further reinforced by Meyer et al. who posit that metacognitive activities, such as reflecting while building and maintaining digital portfolios, teach students to learn how to learn and overcome deficiencies in core competencies (2010). And such egalitarian learning, write Ocak and Ulu (2009) and echoed by Seitz and Bartholomew (2008), results in increased student empowerment in classrooms where teachers are facilitators and students are constructors of knowledge.

The quantitative and anecdotal data surveyed in this review, in addition to subsequent readings in electronic learning and web site testing and development, have provided my proposed study with a solid background and framework. My somewhat amorphous collection of prior knowledge about digital portfolio construction and assessment has been brought into focus and solidified by the efforts of the researchers included in this review. Though all of the studies concentrate specifically on portfolios in K-12 and post-secondary education, each offers a unique perspective, including potential cultural influences (i.e. language proficiency, nationality, socioeconomic status, etc.) on interface design and interaction. These readings have helped situate my understanding of digital portfolios within the frameworks of sociocultural historical theory, postmodern identity theory, and critical perspective theories, including radical pedagogical and poststructuralist.

Though there is scant research specific to digital portfolios in K-5 settings, I find this revelation rather exciting as it offers Summers-Knoll School and me, as a representative of EMU, the opportunity to contribute meaningfully to this burgeoning field. I will rely on the knowledge and best practices gleaned and developed by the scholars included in this review and apply them specifically to the needs of SK’s diverse K-5 population. It is my belief that doing so will not only propel Summers-Knoll School into the vanguard of digital, formative assessment, but, more important, contribute to increased student learning and achievement and a more egalitarian education model that positions children, parents, and teachers as cooperative co-learners.

2010 – 2011 Project Timeline

Though my EMU-sponsored research concludes with completion of my graduate project in spring 2011 and my subsequent graduation from the Written Communication MA program, my work with SK will continue into the summer and beyond. I will continue iterative testing of the digital portfolios and assist in their implementation as well as teacher and student training in fall 2011.

__________________________________________________________________________

Works Cited

Akpinar, Y., Simsek, H. (2007). Pre-Service Teachers’ Learning Object Development: A Case Study in K-12 Setting. Interdisciplinary Journal of Knowledge and Learning Objects, 3, 197-217. Retrieved from http://ijklo.org/all.html

Baron, C. L. (2010). Designing a Digital Portfolio (Second Edition). Berkeley, CA: New Riders

Berrill, D.P., & Whalen, C. (2007). “Where Are the Children?” Personal Integrity and Reflective Teaching Portfolios. Teaching and Teacher Education, 23.6, 868-884. doi:10.1016/j.tate.2007.02.002

Chang, C. (2001). A Study on the Evaluation and Effectiveness Analysis of Web-Based Learning Portfolio (WBLP). British Journal of Educational Technology, 32.4, 435-458. doi:10.1111/1467-8535.00212

Chang, C. (2008). Enhancing Self-Perceived Effects Using Web-Based Portfolio Assessment. Computers in Human Behavior, 24, 1753-1771. doi:10.1016/j.chb.2007.07.005

Chang, C., Tseng, K.H. (2009). Use and Performances of Web-Based Portfolio Assessment. British Journal of Educational Technology, 40.2, 358-370. doi:10.1111/j.1467-8535.2008.00885.x

Lam, R., Lee, I. (2010). Balancing the Dual Functions of Portfolio Assessment. ELT Journal, 64.1, 54-64. doi:10.1093/elt/ccp024

McLaren, P. L. (1991). “Field Relations and the Discourse of the Other: Collaboration in Our Own Ruin.” Experiencing Fieldwork: An Inside View of Qualitative Research. Shaffir, W. B., & Stebbins, R. A., Eds. Newbury Park, CA: Sage

Meyer, E., Abrami, P.C., Wade, C.A., Aslan, O., & Deault, L. (2010). Improving Literacy and Metacognition with Electronic Portfolios: Teaching and Learning with ePearl. Computers & Education, 55, 84-91. doi:10.1016/j.compedu.2009.12.005

Ocak, G., Ulu, M. (2009). The Views of Students, Teachers, and Parents and the Use of Portfolio at the Primary Level. Procedia Social and Behavioral Sciences, 1, 28-36. doi:10.1016/j.sbspro.2009.01.009

Peacock, S., Gordon, L., Murray, S., Morss, K., & Dunlop, G. (2010). Tutor Response to Implementing an ePortfolio to Support Learning and Personal Development in Further and Higher Education Institutions in Scotland. British Journal of Educational Technology, 41.5, 827-851. doi:10.1111/j.1467-8535.2009.00986.x

Schneider, C.G. (2008). The Proof Is in the Portfolio. Liberal Education, 95.1. Retrieved from http://www.aacu.org/liberaleducation/le-wi09/le-wi09_president.cfm

Seitz, H, Bartholomew, C. (2008). Powerful Portfolios for Young Children. Early Childhood Education Journal, 36, 63-68. doi:10.1007/s10643-008-0242-7

Seybolt, R.F. (1935). The Public Schools of Colonial Boston, 1635-1775. Boston, MA: Harvard University Press

Vermilion, E.R. (2008). The EdVantage of Efolios. Distance Learning, 5.4, 67-72. Retrieved from http://www.infoagepub.com/distance-learning.html

Wall, K., Higgins, S., Miller, J., & Packard, N. (2006). Developing Digital Portfolios: Investigating How Digital Portfolios Can Facilitate Pupil Talk About Learning. Technology, Pedagogy and Education, 15.3, 261-273. doi:10.1080/14759390600923535

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: