Skip to content

From Portafoglio to ePortfolio: A Brief History of the Portfolio (r)Evolution

March 4, 2011

My earliest memory of the word “portfolio” dates back to the mid-1980s when I became aware of fine artists using a case or sturdy folder, often in black leather, for transporting and easily sharing samples of their work. These portfolios housed both original pieces, such as watercolors and pencil sketches, as well as photographic reproductions of larger or ephemeral works, like pottery, mixed media sculpture, and performance. Models, too, carried portfolios, usually much smaller and often in more of a traditional binder format, though they commonly referred to them, in the patois of their tribe, as “books.” I found both of these uses exceedingly romantic and was attracted to the idea of having samples of one’s efforts always at-hand. For years I believed portfolios were reserved exclusively for the arts.

Some years later, I became aware of another common use of the term in finance, specifically portfolio as a collection of debt and equity securities. “Portfolio theory is a formal analysis of the relationship between the rates of return on a portfolio of risky securities and the rates of return on the securities contained in that portfolio” (2007, p. 167) writes James Bradfield in Introduction to the Economics of Financial Markets. In other words, careful understanding and application of portfolio theory allows the investor to tailor a financial profile that best meets his/her specific needs. One needn’t be versed in portfolio theory, however, to understand and manage his/her own economic portfolio. Those of us more entrenched in humanities than economics may simply refer to our generic amalgam of investments — home mortgage, retirement fund, mutual funds, etc. — as our “portfolio,” and this is a popular understanding of the term.

Since the mid-1970s, a new type of portfolio has entered the education lexicon and consciousness. This new portfolio is designed for instruction and formative assessment. The earliest reference to educational portfolios in ERIC, an international database of journal and non-journal education literature, is a 1976 report by Joanna Sweet titled “Experience Portfolio: An Approach to Student Writing.” Though Sweet’s portfolio was simply a one-page checklist, she did suggest that it could be referred to throughout the year to assist both teacher and student, and this is a hallmark characteristic of how we’ve come to know and use portfolios in education since. Shortly after Sweet’s apparent inaugural repurposing of the term, portfolio use in progressive education enjoyed a boost in implementation as well as an increase in the breadth of the tool’s possibilities. According to Susan M. Brookhart, Coordinator of Assessment and Evaluation for the School of Education at Duquesne University,

“In the late 1970s and 1980s, portfolios were discovered and became so popular that they were used for all sorts of things, whether they were the most appropriate assessment for the purpose or not. Portfolios arose out of a search for assessments that would encourage lifelong learning skills and student responsibility. They arose during a time of school reform when one of the objectives was to encourage complex thinking and student ownership and agency regarding their work” (2008, p. 445).

Brookhart’s description of portfolios as teaching and assessment tools, perhaps more broadly understood as communication tools, is aligned with my experiences with them in elementary school (K-5) contexts. From 2002 to 2008 I taught grades one through eight (the middle school has since disbanded) at Summers-Knoll School (SK) in Ann Arbor, Michigan. SK’s pedagogical foundation is in experiential, cross-curricular, project-based learning, a model not well served by the constraints of traditional summative assessment tools. To measure student achievement and growth over time, SK uses a two-pronged assessment package developed by the University of Michigan called Work Sampling System (WSS). The first component of WSS is the teacher-produced narrative reports of student development and achievement in eight domains, such as music, foreign language, physical education, fine art, and mathematics. Portfolios complete the package. The portfolios are maintained by students and teachers throughout the year, shared with parents and guardians at regular intervals, and provide visual representation of student progress in the same domains used in the narrative reports. Portfolios offer tangible evidence of sometimes difficult to quantify objectives, such as arts performances and community service activities, as well as opportunities for student reflection, which promotes content retention, awareness of self, and expression of identity, all key objectives of “new,” “authentic,” and “performance” assessment movements rooted in sociocultural historical theory and postmodern identity theory (Berrill & Whalen, 2007). According to Berrill and Whalen portfolios offer visual, textual, and sometimes aural demonstrations of epistemological (valued knowledge and skills), ontological (the nature of knowledge and skills in historical, social, and cultural contexts), and axiological (values and beliefs associated with knowledge and skills) learning (2007).

Before moving on to the portfolio’s newest incarnation, the digital or e-portfolio, the subject of my current research including my capstone MA project at EMU, let’s backtrack just a bit to get a better understanding of the etymology of the word along with its traditional and contemporary definitions. According to the Oxford English Dictionary, the Italian word “portafoglio,” meaning a stiff folder or case for holding papers or prints, first appeared in an isolated attestation in 1556 (OED Online). Known alternately as “portafogli” by 1804, the term entered into popular use in Western Europe and the United Kingdom as well as North America during the nineteenth century. From the Latin “porta,” the imperative of “portare,” or carry, plus “foglio” (from the classical Latin “folium”), or leaf, the device became referred to as “portefeuille,” “porto-folio,” “portefolio,” “port-folio,” and finally “portfolio” by the mid-1830s. (OED Online). A separate meaning for the term, an “office or area of responsibility of a government minister”, used primarily in France and Canada, can be traced back to 1797 (OED Online). The use of a physical portfolio employed by an artist, writer, model, etc., as a conveyance of his/her achievements in a particular field first appeared in an 1883 Times (London) article (OED Online). The portfolio as a range of financial investments debuted in an 1848 Times (London) piece (OED Online), and the Chicago Daily Tribune first coined the term to mean a generic (i.e. not strictly financial) “range of products, services, assets, or qualities offered or possessed” in 1933 (OED Online).

While most of these definitions of portfolios are commonplace today, their popular use in non-financial professional and educational contexts is what interests me most. From the portfolios assembled and maintained by recent college graduates, education professionals seeking employment, and K-12 students mapping their development and achievements, what they all have in common is a specific agenda. Mathers, Challis, Howe, and Field simply define the portfolio as “a collection of evidence maintained and presented for a specific purpose” (1999). Cole expands on this idea by stating, “The term ‘portfolio’ applies to different types of material, collected for different purposes, in different contexts. A short list of components that should be considered in portfolio models includes the platform for recording the collection of evidence, mentoring, and the process of introduction and support for the portfolio” (2005). And the benefits of K-12 and post-secondary portfolios are nicely summed up by Chang in his reference to Hewitt’s seminal 1995 research study:

  1. Portfolios demonstrate students’ growth
  2. Portfolios encourage students to set up learning goals
  3. Portfolios provide evidence of students’ efforts
  4. Portfolios demonstrate students’ performance
  5. Portfolios help teachers review students’ performance
  6. Portfolios stimulate students’ introspective thinking and enhance self-assessment
  7. Portfolios encourage students’ learning interests, improve their communication skills, and build self confidence (2001)

I agree with all of the above statements and have seen evidence of each in both building and maintaining my professional portfolio as well as the numerous collaborative portfolios I’ve designed and managed with K-5 students over the last decade.

The portfolio, in keeping with the natural evolution and dynamism of language, is currently in the midst of a yet another fundamental change, and it is this metamorphosis that is the focus of my personal interests and graduate research at EMU. Pedagogical emphases on digital literacy, perhaps especially evident in secondary and post-secondary settings, have given birth to the digital or e-portfolio, and this platform transition requires more than just transference of hardcopy items onto digital spaces.

Since 2001, Chi-Cheng Chang and his colleagues at National Taipei University of Technology have been on the vanguard of digital portfolio research and design. In a 2001 study, Chang offered specific recommendations for student e-portfolio wireframes (2001), and in later studies, such as 2008’s “Enhancing Self-Perceived Effects Using Web-Based Portfolio Assessment,” which recounts the researcher’s tests of e-portfolios against traditional hardcopy portfolios on two junior high school computer classes of 30 students each, Chang uncovered concrete advantages of the new portfolios that are not simply limited to student evaluation (2008). Specifically, Chang’s findings indicated e-portfolios may be beneficial to learners who often fall through the cracks, including students with low motivation and/or self-esteem, underperforming students due to at-risk classification, and special needs students including twice exceptional and gifted learners (2008). This finding is echoed by Meyer, Abrami, Wade, Aslan, and Deault who conducted a similar study in Montreal and found that e-portfolios have the ability to stimulate metacognitive processes including planning, monitoring, and regulating, and that they facilitate learning how to learn (2010).

Of the preliminary secondary research I’ve thus far conducted on digital portfolio design and implementation, the bulk, aside from Chang’s single middle-school study, involve the use of e-portfolios is secondary and post-secondary teaching, assessment, and career preparation. While traditional hardcopy portfolios are popular in a variety of K-5 contexts worldwide, minimal research has been conducted on e-portfolio implementation in the elementary grades. It’s the belief of the faculty and staff at Summers-Knoll School, as well as my personal belief, that the benefits of e-portfolios charted at the higher levels of education can be extrapolated to include the youngest of learners. We will attempt to confirm this through qualitative and quantitative methods, and, hopefully, contribute meaningfully to the continuing evolution of the term “portfolio.”

__________________________________________________________________________

References

Berrill, D.P., & Whalen, C. (2007). “Where Are the Children?” Personal Integrity and Reflective Teaching Portfolios. Teaching and Teacher Education, 23.6, 868-884. doi:10.1016/j.tate.2007.02.002

Bradfield, J. (2007). Introduction to the Economics of Financial Markets. New York, NY: Oxford University Press

Brookhart, S. M. (2008). Portfolio Assessment. In Good, T. L. (Ed.), 21st Century Education: A Reference Handbook Vol. 2 (pp. 443-453). Thousand Oaks, CA: SAGE Publications

Chang, C. (2001). A Study on the Evaluation and Effectiveness Analysis of Web-Based Learning Portfolio (WBLP). British Journal of Educational Technology, 32.4, 435-458. doi:10.1111/1467-8535.00212

Chang, C. (2008). Enhancing Self-Perceived Effects Using Web-Based Portfolio Assessment. Computers in Human Behavior, 24, 1753-1771. doi:10.1016/j.chb.2007.07.005

Cole, G. (2005). The definition of ‘portfolio.’ Medical Education, 39(11), 1141. Doi:10.1037/0278-6133.24.2.225

Mathers, N.J., Challis, M.C., Howe, A.C., & Field, N.J. (1999). Portfolios in continuing medical education – effective and efficient? Medical Education, 33, 521-530

Meyer, E., Abrami, P.C., Wade, C.A., Aslan, O., & Deault, L. (2010). Improving Literacy and Metacognition with Electronic Portfolios: Teaching and Learning with ePearl. Computers & Education, 55, 84-91. doi:10.1016/j.compedu.2009.12.005

Oxford English Dictionary Online. (November 2010). Oxford University Press. Retrieved from http://www.oed.com/view/Entry/148175?redirectedFrom=portfolio

Sweet, J. (1976). Experience Portfolio: An Approach to Student Writing. The English Journal, 65(6), 50-51. Retrieved from http://www.jstor.org/stable/814557

Grad Project Proposal Draft

December 5, 2010

Introduction: Capturing Authentic Assessment of K-5 Student Performance

Measuring student performance in K-12 education has been at the forefront of public debate and policy since Puritan settlers established the first public school in Boston in 1635 (Seybolt, 1935). The concern over K-12 academic achievement has reached heightened levels of attention in recent years due to a variety of factors including declining test scores in core content areas such as math, reading, and science; increased socioeconomic disparities among members of the working and middle classes; and an international sociocultural shift to a hyper-competitive, information-based economy. One popular and far-reaching response to the new challenges faced by K-12 educators, education policy makers, and other education community members has been increased quantitative student assessments, especially in the form of standardized tests. Results from these high stakes measurement tools have been used to categorize students into rigid learning classifications; drive curricula design and instructional methodology choices; and promote and demote teachers, schools, and even entire school districts. While there are possible benefits of summative student assessments — they may promote abstract problem solving, for example — the data they capture ultimately illustrate a single performance in a single, specific context without regard to a host of potential variables including, but not limited to:

  • English language proficiency
  • Overall physical health of student
  • Mental health of student
  • Testing site conditions
  • Learning style and preference
  • Test taking preparedness
  • Testing of knowledge instead of ability
  • Gender
  • Socioeconomic status

An additional drawback of relying on standardized testing to reward or punish students and teachers and drive curricula development and implementation is that high stakes tests rank students against each other instead of themselves. Such tests capture knowledge-based scores, but offer no illustration of student growth and achievement over time. I posit that application of a balanced assessment program, including narrative-based evaluations and multimedia portfolios of student work, offer both quantitative and harder-to-measure qualitative data reflective of learner knowledge and performance. Further, the use of multimedia, or digital, portfolios positions students in the driver’s seat of their own education. Through reflective selection of works demonstrating peak performance as well as improvement over time, students become active, engaged, and empowered participants in their own learning and growth. Additionally, I believe that qualitative performance data captured in narrative assessment reports can be reframed or reinterpreted to meet the more quantitative parameters of traditional educational benchmarks.

Background: Alternative Practices in K-5 Curricula, Instruction, and Assessment at Summers-Knoll School

Prior to entering my current position as an editor of training recommendation reports for the US Navy, I spent six years as a homeroom teacher at Summers-Knoll School (SK) in Ann Arbor, Michigan. SK is an independent, kindergarten through fifth grade school that focuses on traditional liberal arts content delivered in a cross-curricular, experiential format. Through embedded learning and developmental objectives and qualitative performance assessment, students’ energies are channeled into how to learn as opposed to what to learn. Students and families are involved in curricular decision making, and they also play active roles in assessing student performance through the use of the school’s two primary assessment tools: Work Sampling System (WSS), a narrative-based assessment tool developed by the University of Michigan, and portfolios. I taught grades one through seven (the middle school program has since disbanded) at SK from 2002 to 2008, and I remain committed to the school’s educational philosophy and mission.

SK practices progressive instruction and student evaluation. Parents of students at SK become familiar with the school’s philosophy and practices during the in-depth application process. Despite the parent buy-in, or understanding, of SK’s teaching methodologies, a minority of parents expresses concern that skills developed at SK may not be transferable to more conventional learning environments. This concern appears to heighten as students rise through the grades and middle school looms. Though the school offers annual, optional, standardized testing to students in grades 4-5, some parents would prefer a balanced assessment approach that incorporates norms-based measures. While this is contrary to the philosophy of SK, I believe that quantitative performance data can be generated if SK switches to digital portfolios and the web-based WSS assessment tools. I posit that these data can be — and already are — generated via the current assessment tools. I believe digital portfolios and web-based WSS assessments will offer reinterpretations of student performance data that can be captured in quantifiable terms. By moving student assessments onto digital platforms, SK can remain faithful to the mission of its founders, incorporate technology into instruction and assessment in meaningful ways, and meet the needs and wants of parents with seemingly conflicting agendas.

The Problem and the Plan: Moving Summers-Knoll School’s Assessment Tools to Digital Platforms

SK is currently working with graduate students from the University of Michigan’s School of Information to develop a digital student portfolio wireframe and a working prototype. The head of school at SK, Joanna Hastings, has asked me to participate in the project by working on a team comprised of educators and designers to develop the communication component of this venture, facilitate iterative usability beta testing on the digital portfolios, and conduct secondary research into the implementation of digital portfolios in K-5 settings and their ability to capture quantitative student performance data via qualitative means. Further, I will facilitate usability testing on the school’s own website, to which the digital portfolios will be linked via a secure parent / teacher / student user portal, and evaluate WSS’s web-based version (teachers currently produce WSS evaluations in hardcopy) through additional usability testing and teacher and parent interviews. Upon approval from EMU’s University Human Subjects Review Committee, my data collection will begin in winter 2011, and I will wrap up my study by early summer 2011. Little research exists on digital portfolio implementation and assessment in elementary school settings, and I hope to contribute meaningfully to this burgeoning field of inquiry.

Methodologies

My project includes pre-study planning meetings with SK’s head of school, Joanna Hastings; University of Michigan School of Information graduate student wireframe and prototype developers Charity Ben-Daniels, Erin Dietrich, Thomas Piggott, Megan Schwarz, and Kathryn Totz; SK’s contracted web designer, Anjanette Bunce; and members of SK’s technology committee, an amalgam of K-12 and post-secondary educators from Summers-Knoll School, Eastern Michigan University, and University of Michigan. The design of my project will be informed by postmodern critical theory. Rather than taking a detached, descriptive, explanatory approach to my research, I will develop a co-learner partnership with my human subjects. The collaborative framework within which I will carry out my research will attempt to establish an egalitarian distribution of influence and power. Taking a critical perspective will allow me to work “with and not on a group” (McLaren, 154).

To collect relevant data on digital assessment portfolios I will conduct iterative usability testing on the UM SI-designed portfolio prototype. Hypothetical student data will be loaded into the digital portfolio prototype and usability testing will be conducted with a small team of SK teachers and parents. The results of my tests will be conveyed to web-designer Anjanette Bunce who is taking over primary portfolio design responsibility after the prototype has been delivered in December 2010. Ongoing usability testing and interviewing with teachers and parents will directly inform Anjanette’s design modifications to ensure development of a product that best meets the needs of the three primary user groups: students, parents, and teachers.

To collect relevant data on the web-based version of the University of Michigan’s WSS narrative assessment tool, I will conduct interviews with teachers and parents to determine the strengths and weaknesses of the current paper and pencil version used by SK. My interviews will also probe into respondents’ philosophies on K-5 education, experiential and other non-traditional learning methods, and qualitative and quantitative assessment. I will then create a virtual classroom of eight to ten multicultural personas and load the web-based version of WSS with hypothetical performance data of this group of students. Data will reflect a range of experiences, learning styles and preferences, socioeconomic backgrounds, nationalities and language proficiencies, and academic strengths and challenges. These hypothetical, though authentically reflective, data will be presented to teachers and parents via usability evaluations. I believe that the web-based WSS will allow the qualitative-leaning data to be framed or positioned in such a way that better satisfies the wants and needs of parents interested in quantitative analysis of student performance.

Data Analysis

Data will be collected through audio taped interviews of parents of K-5 students at SK, three K-5 homeroom teachers, and four specials teachers (music, art, physical education, Latin/French). All human subjects will be volunteers and each will be provided with a typed transcript of the interview after completion.

Further data will be collected through audio taped usability testing on at least one and possibly two beta versions of digital portfolios. Participants will be the same teachers and parents as in the interviews. Data will be evaluated via a combination of content analysis and narrative analysis methods.

Literature Review

A prelude to my project design included conducting a literature review of scholarly research on portfolio assessment. Though my project focuses specifically on digital, or electronic, portfolios, I expanded my reading to include the use of traditional portfolios in K-12 and post-secondary education as well. Foremost in my review was searching for a clear explanation of what a portfolio is. Ocak and Ulu offer a concise definition of the primary-level portfolio calling it “a carefully selected collection of student work that provides clear evidence to the student, parent, and…educators of the student’s knowledge, skills, strategies, grasp of concepts, attitudes, and achievement in a given area over a specific time period” (2009, p. 28). Peacock, Gordon, Murray, Morss, and Dunlop provide a digital-specific definition in their 2010 study: “An ePortfolio is an electronic system that facilitates the development, collection, and management of digital resources which may be drawn from a range of learner experiences over a period of time and could include those from formal and non-formal learning opportunities” (p. 828). This interpretation may be especially applicable to experiential and cross-curricular learning by suggesting that digital, or ePortfolios, are capable of capturing development and achievement that occurs outside of conventional scenarios, including, as Wall, Higgins, Miller, and Packard point out, art, physical education, design, and technology (2006). Perhaps the most succinct definition comes from Meyer, Abrami, Wade, Aslan, and Deault who call the electronic portfolio “a digital container capable of storing visual and auditory content including text, images, video, and sound” (2010, p. 84). By including the digital portfolio’s multimedia capabilities, the authors highlight another key benefit of electronic performance showcasing.

In searching for clear-cut descriptions of digital portfolios that would help me develop my own informed understanding, I was struck by the lack of consistency in terminology across the literature. Alternately referred to by Chang as web-based learning portfolios (WBLP) (2001), Meyer et al. as electronic portfolios (EP) (2010), and Vermilion as Efolios (2008), the naming or development of a lexicon appears secondary to charting and tracking the primary functions and purposes of digital portfolios.

The documented benefits of digital portfolio assessment are numerous. Among the most interesting to me and relevant to my proposed study with SK is Chang’s assertion that portfolio assessment identifies student skills rather than shortcomings, identifies the unique needs of each student, increases student motivation, and encourages students to develop decision-making skills (2008). This is in stark contrast to more traditional summative assessment tools prevalent in lecture/test models that focus on memorization and context-specific performance. Chang’s research also indicates that digital portfolio assessment may be beneficial to students with low motivation and/or self-esteem, underperforming students due to at-risk-classification, and special needs students including twice exceptional and gifted learners (2008). Wall et al. further suggest that digital portfolios are an appropriate method for recording achievement of pupils who find writing difficult (2006). In other words, portfolio assessment may meet the unique needs of learners whose talents sometimes fall outside the confines of the bell curve.

Lam and Lee’s research study focused specifically on how digital portfolio assessments can facilitate quantitative achievement in English as a second language (ESL) and the language arts. After implementation of portfolio assessment in the authors’ ESL classroom, the data indicate that students’ writings were more grammatically and structurally accurate and that students generated more and better ideas in their assignments (2010). Vermillion reports anecdotal evidence that digital portfolios improve skills in technology, public speaking, and leadership (2008). And Hewitt’s seminal 1995 research study on portfolio assessment, as reported by Chang, uncovered the following seven benefits:

  1. Portfolios demonstrate students’ growth
  2. Portfolios encourage students to set up learning goals
  3. Portfolios provide evidence of students’ efforts
  4. Portfolios demonstrate students’ performance
  5. Portfolios help teachers review students’ performance
  6. Portfolios stimulate students’ introspective thinking and enhance self-assessment
  7. Portfolios encourage students’ learning interests, improve their communication skills, and build self confidence (2001, p.436)

While all of the above provide obvious advantages to learners and teachers alike, the stimulation of introspective thinking is especially interesting to me. One of the precepts of contemporary education studies is that reflective learning results in increased retention and achievement; therefore one can draw a direct correlation between Hewitt’s list of benefits and authentic, measurable learning. This is further reinforced by Meyer et al. who posit that metacognitive activities, such as reflecting while building and maintaining digital portfolios, teach students to learn how to learn and overcome deficiencies in core competencies (2010). And such egalitarian learning, write Ocak and Ulu (2009) and echoed by Seitz and Bartholomew (2008), results in increased student empowerment in classrooms where teachers are facilitators and students are constructors of knowledge.

The quantitative and anecdotal data surveyed in this review, in addition to subsequent readings in electronic learning and web site testing and development, have provided my proposed study with a solid background and framework. My somewhat amorphous collection of prior knowledge about digital portfolio construction and assessment has been brought into focus and solidified by the efforts of the researchers included in this review. Though all of the studies concentrate specifically on portfolios in K-12 and post-secondary education, each offers a unique perspective, including potential cultural influences (i.e. language proficiency, nationality, socioeconomic status, etc.) on interface design and interaction. These readings have helped situate my understanding of digital portfolios within the frameworks of sociocultural historical theory, postmodern identity theory, and critical perspective theories, including radical pedagogical and poststructuralist.

Though there is scant research specific to digital portfolios in K-5 settings, I find this revelation rather exciting as it offers Summers-Knoll School and me, as a representative of EMU, the opportunity to contribute meaningfully to this burgeoning field. I will rely on the knowledge and best practices gleaned and developed by the scholars included in this review and apply them specifically to the needs of SK’s diverse K-5 population. It is my belief that doing so will not only propel Summers-Knoll School into the vanguard of digital, formative assessment, but, more important, contribute to increased student learning and achievement and a more egalitarian education model that positions children, parents, and teachers as cooperative co-learners.

2010 – 2011 Project Timeline

Though my EMU-sponsored research concludes with completion of my graduate project in spring 2011 and my subsequent graduation from the Written Communication MA program, my work with SK will continue into the summer and beyond. I will continue iterative testing of the digital portfolios and assist in their implementation as well as teacher and student training in fall 2011.

__________________________________________________________________________

Works Cited

Akpinar, Y., Simsek, H. (2007). Pre-Service Teachers’ Learning Object Development: A Case Study in K-12 Setting. Interdisciplinary Journal of Knowledge and Learning Objects, 3, 197-217. Retrieved from http://ijklo.org/all.html

Baron, C. L. (2010). Designing a Digital Portfolio (Second Edition). Berkeley, CA: New Riders

Berrill, D.P., & Whalen, C. (2007). “Where Are the Children?” Personal Integrity and Reflective Teaching Portfolios. Teaching and Teacher Education, 23.6, 868-884. doi:10.1016/j.tate.2007.02.002

Chang, C. (2001). A Study on the Evaluation and Effectiveness Analysis of Web-Based Learning Portfolio (WBLP). British Journal of Educational Technology, 32.4, 435-458. doi:10.1111/1467-8535.00212

Chang, C. (2008). Enhancing Self-Perceived Effects Using Web-Based Portfolio Assessment. Computers in Human Behavior, 24, 1753-1771. doi:10.1016/j.chb.2007.07.005

Chang, C., Tseng, K.H. (2009). Use and Performances of Web-Based Portfolio Assessment. British Journal of Educational Technology, 40.2, 358-370. doi:10.1111/j.1467-8535.2008.00885.x

Lam, R., Lee, I. (2010). Balancing the Dual Functions of Portfolio Assessment. ELT Journal, 64.1, 54-64. doi:10.1093/elt/ccp024

McLaren, P. L. (1991). “Field Relations and the Discourse of the Other: Collaboration in Our Own Ruin.” Experiencing Fieldwork: An Inside View of Qualitative Research. Shaffir, W. B., & Stebbins, R. A., Eds. Newbury Park, CA: Sage

Meyer, E., Abrami, P.C., Wade, C.A., Aslan, O., & Deault, L. (2010). Improving Literacy and Metacognition with Electronic Portfolios: Teaching and Learning with ePearl. Computers & Education, 55, 84-91. doi:10.1016/j.compedu.2009.12.005

Ocak, G., Ulu, M. (2009). The Views of Students, Teachers, and Parents and the Use of Portfolio at the Primary Level. Procedia Social and Behavioral Sciences, 1, 28-36. doi:10.1016/j.sbspro.2009.01.009

Peacock, S., Gordon, L., Murray, S., Morss, K., & Dunlop, G. (2010). Tutor Response to Implementing an ePortfolio to Support Learning and Personal Development in Further and Higher Education Institutions in Scotland. British Journal of Educational Technology, 41.5, 827-851. doi:10.1111/j.1467-8535.2009.00986.x

Schneider, C.G. (2008). The Proof Is in the Portfolio. Liberal Education, 95.1. Retrieved from http://www.aacu.org/liberaleducation/le-wi09/le-wi09_president.cfm

Seitz, H, Bartholomew, C. (2008). Powerful Portfolios for Young Children. Early Childhood Education Journal, 36, 63-68. doi:10.1007/s10643-008-0242-7

Seybolt, R.F. (1935). The Public Schools of Colonial Boston, 1635-1775. Boston, MA: Harvard University Press

Vermilion, E.R. (2008). The EdVantage of Efolios. Distance Learning, 5.4, 67-72. Retrieved from http://www.infoagepub.com/distance-learning.html

Wall, K., Higgins, S., Miller, J., & Packard, N. (2006). Developing Digital Portfolios: Investigating How Digital Portfolios Can Facilitate Pupil Talk About Learning. Technology, Pedagogy and Education, 15.3, 261-273. doi:10.1080/14759390600923535

Literature Review: Digital Portfolio Assessment

December 2, 2010

My culminating Written Communication MA graduate project focuses on formative academic assessment in K-5 education. As a certified elementary school teacher, continuing education student, and professional writer, I am especially interested in exploring, identifying, and creating methods for quantifying academic performance in alternative or non-mainstream settings. I am also interested in how to best communicate performance data to parents and guardians of students engaged in learning contexts that are not bound by conventional measurement methods. My study site is Summers-Knoll School (SK) in Ann Arbor, Michigan. Summers-Knoll is an independent, kindergarten through fifth grade school that focuses on traditional liberal arts content delivered in a cross-curricular, experiential format. Through embedded learning and developmental objectives and qualitative performance assessment, students’ energies are channeled into how to learn as opposed to what to learn. Students and families partner with teachers in curricular decision making, and they also play active roles in assessing student performance through the use of the school’s two primary assessment tools: Work Sampling System (WSS), a narrative-based assessment tool developed by the University of Michigan and portfolios. Teachers at SK currently use the paper and pencil version of WSS for narrative student assessments and hardcopy portfolios for showcasing student work. The purpose of my study is to determine the value of moving both of these assessment tools onto digital platforms and whether or not such a shift would benefit all members of the SK community including students, parents, faculty, and administrators.

A prelude to my project design included conducting a literature review of scholarly research on portfolio assessment. Though my project focuses specifically on digital, or electronic, portfolios, I expanded my reading to include the use of traditional portfolios in K-12 and post-secondary education as well. Foremost in my review was searching for a clear explanation of what a portfolio is. Ocak and Ulu offer a concise definition of the primary-level portfolio calling it “a carefully selected collection of student work that provides clear evidence to the student, parent, and…educators of the student’s knowledge, skills, strategies, grasp of concepts, attitudes, and achievement in a given area over a specific time period” (2009, p. 28). Peacock, Gordon, Murray, Morss, and Dunlop provide a digital-specific definition in their 2010 study: “An ePortfolio is an electronic system that facilitates the development, collection, and management of digital resources which may be drawn from a range of learner experiences over a period of time and could include those from formal and non-formal learning opportunities” (p. 828). This interpretation may be especially applicable to experiential and cross-curricular learning by suggesting that digital, or ePortfolios, are capable of capturing development and achievement that occurs outside of conventional scenarios, including, as Wall, Higgins, Miller, and Packard point out, art, physical education, design, and technology (2006). Perhaps the most succinct definition comes from Meyer, Abrami, Wade, Aslan, and Deault who call the electronic portfolio “a digital container capable of storing visual and auditory content including text, images, video, and sound” (2010, p. 84). By including the digital portfolio’s multimedia capabilities, the authors highlight another key benefit of electronic performance showcasing.

In searching for clear-cut descriptions of digital portfolios that would help me develop my own informed understanding, I was struck by the lack of consistency in terminology across the literature. Alternately referred to by Chang as web-based learning portfolios (WBLP) (2001), Meyer et al. as electronic portfolios (EP) (2010), and Vermilion as Efolios (2008), the naming or development of a lexicon appears secondary to charting and tracking the primary functions and purposes of digital portfolios.

The documented benefits of digital portfolio assessment are numerous. Among the most interesting to me and relevant to my study with SK is Chang’s assertion that portfolio assessment identifies student skills rather than shortcomings, identifies the unique needs of each student, increases student motivation, and encourages students to develop decision-making skills (2008). This is in stark contrast to more traditional summative assessment tools prevalent in lecture/test models that focus on memorization and context-specific performance. Chang’s research also indicates that digital portfolio assessment may be beneficial to students with low motivation and/or self-esteem, underperforming students due to at-risk-classification, and special needs students including twice exceptional and gifted learners (2008). Wall et al. further suggest that digital portfolios are an appropriate method for recording achievement of pupils who find writing difficult (2006). In other words, portfolio assessment may meet the unique needs of learners whose talents sometimes fall outside the confines of the bell curve.

Lam and Lee’s research study focused specifically on how digital portfolio assessments can facilitate quantitative achievement in English as a second language (ESL) and the language arts. After implementation of portfolio assessment in the authors’ ESL classroom, the data indicate that students’ writings were more grammatically and structurally accurate and that students generated more and better ideas in their assignments (2010). Vermillion reports anecdotal evidence that digital portfolios improve skills in technology, public speaking, and leadership (2008). And Hewitt’s seminal 1995 research study on portfolio assessment, as reported by Chang, uncovered the following seven benefits:

  1. Portfolios demonstrate students’ growth
  2. Portfolios encourage students to set up learning goals
  3. Portfolios provide evidence of students’ efforts
  4. Portfolios demonstrate students’ performance
  5. Portfolios help teachers review students’ performance
  6. Portfolios stimulate students’ introspective thinking and enhance self-assessment
  7. Portfolios encourage students’ learning interests, improve their communication skills, and build self confidence (2001, p.436)

While all of the above provide obvious advantages to learners and teachers alike, the stimulation of introspective thinking is especially interesting to me. One of the precepts of contemporary educational studies is that reflective learning results in increased retention and achievement; therefore one can draw a direct correlation between Hewitt’s list of benefits and authentic, measurable learning. This is further reinforced by Meyer et al. who posit that metacognitive activities, such as reflecting while building and maintaining digital portfolios, teach students to learn how to learn and overcome deficiencies in core competencies (2010). And such egalitarian learning, write Ocak and Ulu (2009) and echoed by Seitz and Bartholomew (2008), results in increased student empowerment in classrooms where teachers are facilitators and students are constructors of knowledge.

The quantitative and anecdotal data surveyed in this review, in addition to subsequent readings in electronic learning and web site testing and development, have provided my study with a solid background and framework. My somewhat amorphous collection of prior knowledge about digital portfolio construction and assessment has been brought into focus and solidified by the efforts of the researchers included in this review. Though all of the studies concentrate specifically on portfolios in K-12 and post-secondary education, each offers a unique perspective, including potential cultural influences (i.e. language proficiency, nationality, socioeconomic status, etc.) on interface design and interaction. These readings have helped situate my understanding of digital portfolios within the frameworks of sociocultural historical theory, postmodern identity theory, and critical perspective theories, including radical pedagogical and poststructuralist.

Though there is scant research specific to digital portfolios in K-5 settings, I find this revelation rather exciting as it offers Summers-Knoll School and me the opportunity to contribute meaningfully to this burgeoning field. I will rely on the knowledge and best practices gleaned and developed by the scholars included in this review and apply them specifically to the needs of SK’s diverse K-5 population. It is my belief that doing so will not only propel Summers-Knoll School into the vanguard of digital, formative assessment, but, more important, contribute to increased student learning and achievement and a more egalitarian education model that positions children, parents, and teachers as cooperative co-learners.

__________________________________________________________________________

Works Cited

Akpinar, Y., Simsek, H. (2007). Pre-Service Teachers’ Learning Object Development: A Case Study in K-12 Setting. Interdisciplinary Journal of Knowledge and Learning Objects, 3, 197-217. Retrieved from http://ijklo.org/all.html

Baron, C. L. (2010). Designing a Digital Portfolio (Second Edition). Berkeley, CA: New Riders

Berrill, D.P., & Whalen, C. (2007). “Where Are the Children?” Personal Integrity and Reflective Teaching Portfolios. Teaching and Teacher Education, 23.6, 868-884. doi:10.1016/j.tate.2007.02.002

Chang, C. (2001). A Study on the Evaluation and Effectiveness Analysis of Web-Based Learning Portfolio (WBLP). British Journal of Educational Technology, 32.4, 435-458. doi:10.1111/1467-8535.00212

Chang, C. (2008). Enhancing Self-Perceived Effects Using Web-Based Portfolio Assessment. Computers in Human Behavior, 24, 1753-1771. doi:10.1016/j.chb.2007.07.005

Chang, C., Tseng, K.H. (2009). Use and Performances of Web-Based Portfolio Assessment. British Journal of Educational Technology, 40.2, 358-370. doi:10.1111/j.1467-8535.2008.00885.x

Lam, R., Lee, I. (2010). Balancing the Dual Functions of Portfolio Assessment. ELT Journal, 64.1, 54-64. doi:10.1093/elt/ccp024

Meyer, E., Abrami, P.C., Wade, C.A., Aslan, O., & Deault, L. (2010). Improving Literacy and Metacognition with Electronic Portfolios: Teaching and Learning with ePearl. Computers & Education, 55, 84-91. doi:10.1016/j.compedu.2009.12.005

Ocak, G., Ulu, M. (2009). The Views of Students, Teachers, and Parents and the Use of Portfolio at the Primary Level. Procedia Social and Behavioral Sciences, 1, 28-36. doi:10.1016/j.sbspro.2009.01.009

Peacock, S., Gordon, L., Murray, S., Morss, K., & Dunlop, G. (2010). Tutor Response to Implementing an ePortfolio to Support Learning and Personal Development in Further and Higher Education Institutions in Scotland. British Journal of Educational Technology, 41.5, 827-851. doi:10.1111/j.1467-8535.2009.00986.x

Schneider, C.G. (2008). The Proof Is in the Portfolio. Liberal Education, 95.1. Retrieved from http://www.aacu.org/liberaleducation/le-wi09/le-wi09_president.cfm

Seitz, H, Bartholomew, C. (2008). Powerful Portfolios for Young Children. Early Childhood Education Journal, 36, 63-68. doi:10.1007/s10643-008-0242-7

Vermilion, E.R. (2008). The EdVantage of Efolios. Distance Learning, 5.4, 67-72. Retrieved from http://www.infoagepub.com/distance-learning.html

Wall, K., Higgins, S., Miller, J., & Packard, N. (2006). Developing Digital Portfolios: Investigating How Digital Portfolios Can Facilitate Pupil Talk About Learning. Technology, Pedagogy and Education, 15.3, 261-273. doi:10.1080/14759390600923535

Summers-Knoll.org Usability Testing

November 1, 2010

In preparation for usability testing on Summers-Knoll School’s digital student portfolios and the web-based version of the Work Sampling System, I conducted brief usability tests on Summers-Knoll.org. This exercise was designed more to develop my skills as a test facilitator than with generating especially valuable data for SK, particularly since SK’s web site is soon to be restructured to allow for user login capabilities including parental access to online student portfolios.
______________________________________________________________________

A Closer Look at Summers-Knoll.org: A Study in Usability and Perception

Abstract: Assessing the Need for Usability Testing

Summers-Knoll School (SK), located in Ann Arbor, Michigan, is a kindergarten through fifth grade school committed to cross-curricular, experiential, and inquiry-based learning. As the founders of SK were key players in the field of emerging design technologies of the 1990s, the school was established with a secondary focus of multidisciplinary technology integration. Today, SK remains on the cutting edge of instruction, including the use of technology. Despite this, the school still struggles with how to make its web site more compelling and interactive, how to change it from an information depository to a relevant and dynamic destination site.

With the aid of graduate students from the School of Information at the University of Michigan, SK is currently addressing ways to design a user portal at the school’s site. An educational technology team is also designing the blueprint for eventual construction of digital, multimedia student performance portfolios to be accessed by students and parents at Summers-Knoll.org. The usability test I created for this report addresses the usefulness and look and feel of Summers-Knoll.org in its current iteration.

Introduction & Background

Authentic Integration of Technology at Summers-Knoll School

Summers-Knoll (SK) is an independent, kindergarten through fifth grade, experiential learning-based school located in Ann Arbor, Michigan. While the school’s primary focus is on cross-curricular, balanced liberal arts education, the integration or mainstreaming of technology into teacher instruction, student learning, and communication is an integral ingredient of the school community’s collective ethos. Originally founded by a small team of educators and parents including Tom Knoll, lead developer of Adobe Photoshop, and his wife Ruth Knoll, Summers-Knoll has been well equipped with current hardware and software since its inception in 1996. The school’s commitment to integrated technology instruction, as well as building and maintaining a dynamic and ever-evolving infrastructure, make SK a singular institution among the myriad of K-5 choices in this university town.

SK’s unique approach to technology lies in the general philosophical understanding of all community members (parents, faculty, staff, etc.) that technological tools — whether a hammer and nail or a laptop computer — do not exist in isolation. Rather, the tools SK community members use in solving problems are context dependent. Approaching technology from this position allows teachers at SK to seamlessly integrate a variety of contemporary technologies into multidisciplinary lessons and activities. Technology is not a discipline. Instead, it provides a palette of tools for real-world problem solving rooted in traditional learning objectives.

The Problem: Is Summers-Knoll.org Useful?

Despite Summers-Knoll’s progressive philosophical and methodological approaches to technology integration, the school’s web site has faced challenges in adequately representing SK’s ethos as well as establishing itself as a destination site for members of the community. Though Summers-Knoll.org has long been a valuable resource for prospective families and teachers, it has lacked usefulness or relevancy in further development of already-established relationships: once people become members of the community, they historically have had little reason to return to the school’s homepage. And although the homepage provides links to each teacher’s homeroom or special subject blog, the blogs are primarily accessed by parents and students directly rather than through the SK portal. Additionally, the bulk of parent/teacher, parent/staff, and teacher/staff communication is conducted via email. In the past, teachers have used a variety of web-based applications, both free and subscription-based, to manage community communications, but, though many of these tools proved adequate, none were directly linked to the SK site.

Summers-Knoll.org has existed in numerous iterations within four primary wireframe designs since its original launch in the mid ‘90s. Its current design is probably its most contemporary, aesthetically appealing, and accurate in its visual and textual reflection of the school’s philosophy and ethos. Despite its evolutionary improvements, however, its lack of authentic usefulness to the dynamic community renders it an eye-pleasing though somewhat superfluous knickknack. Below is a clickable screen shot of Summers-Knoll.org’s homepage.

Design augmentation of Summers-Knoll.org is currently in the planning stages. Accepted by the graduate program at the University of Michigan School of Information to be used as a project study, Summers-Knoll.org is expanding to include member login capabilities and access to user data including student assessment. Student digital portfolio wireframes are concurrently under construction and will be accessible by the SK portal beginning in fall 2011. By allowing users to create unique accounts and access and manage their community-related information, Summers-Knoll.org will position itself as a destination site and a necessary and useful tool for communication/information management.

Usability Testing of Summers-Knoll.org

Background

Concurrent with Summers-Knoll.org’s expansion planning, I conducted a small-scale usability test of the site in its present iteration. Though I have been a tester in other usability scenarios, this was my first experience as a facilitator, and my plan was based on Steve Krug’s do-it-yourself book, Rocket Surgery Made Easy, lecture and discussion from Dr. Steve Benninghhoff’s English 524 – Advanced Technical Writing and Research, and a dash of common sense and invention.

Test Design and Results

I chose remote testing for this project for two reasons. First, the bulk of my professional and educational work is done remotely in virtual contexts. Second, remote testing allowed me to sample a greater cross section of users from a wider socioeconomic spectrum with a greater array of racial, ethnic, political, and sexual/gender identifications. In short, remote testing allowed for the collection of a wider range of data. Further, testing remotely directly lent itself to easier recruiting, smoother scheduling, and elimination of travel concerns.

I began with an in-depth review of Summers-Knoll.org. I checked all links to ensure they were active and up-to-date, maneuvered across sections and pages to determine general ease of use, and used contact links/email addresses to communicate with teachers and staff. Further, I focused on the look and feel of Summers-Knoll.org in an attempt to quantify if the site projects an overall ethos aligned with SK community members and the school’s physical site.

Next, I scripted ten questions/problem scenarios based around Summers-Knoll.org. Some questions were more qualitative in nature, such as “What are your initial thoughts when viewing the homepage?” It’s through these types of questions I hoped to generate reflexive responses from users in regards to their perceptions of the school’s philosophy. Other questions, like “What curricular program does Summers-Knoll use for math instruction? Where did you find this information?” were designed with practical needs in mind. Potential members of the SK community are often deeply interested in nuts and bolts information, such as standards and curricula, so making this type of content easy to find is paramount for usefulness.

After designing a mix of questions/problems, I uploaded them to Surveymonkey.com. Surveymonkey allowed me to create a questionnaire that would record users’ written responses. I next recruited testers by telephone, most of whom had at least a peripheral interest in education and/or parenting. After successful recruitment of six participants ranging in age from 20 – 76, I followed up with these email instructions:

Over a five-day period, I met virtually with each of my six testers. I met four using the telephone, and two using Skype 5.0. Skype allowed me to interact visually with my testers and may have enhanced my note taking by providing me with real-time body language feedback from users. I would have preferred using Skype with all six users, though using the service with some and not others allowed for a comparison of the two different testing scenarios. Additionally, meeting with testers virtually allowed for the use of a variety of computers, operating systems, and web browsers. This provided me with the ability to compare performance/usefulness across a variety of platforms.

Each test, both the four relying on telephone communication and the two using Skype, followed a similar structure. I explained in detail to my testers that my objective was to analyze the web site and not them or their abilities. Each of my testers then opened two windows on his/her computer. One window was directed to Summers-Knoll.org and the other to Surveymonkey.com. Beginning at Surveymonkey.com, I read aloud the first question to each user and asked if they required additional clarification. Next, each user responded to the question based on his/her reactions at the SK homepage. The responses were recorded textually on Surveymonkey.com and also in accompanying notes I kept for each test user. Below is a clickable screen shot of one of my questions on Surveymonkey including the responses from my six users.

Lessons Learned

As evidenced by the responses to this and other more evaluative questions in the survey, the users represent a variety of backgrounds and influences. In an earlier survey question, participant 3 viewed the school as “multi-ethnic” while participant 4 suggested it’s “possibly non-diverse.” I believe both of these responses are influenced by each user’s socioeconomic status (one is a small business owner, the other unemployed) and geographic location (one is a member of a heterogeneous Southern community, the other resides in a diverse, urban city in the North). I was pleasantly surprised by the variety of responses I received based on user’s experiences and perceptions, or what Kenneth Burke calls “terministic screens.” Prior to the testing, I had not considered how such screens could affect the responses from the users, and perhaps most revealing to me was my own narrow scope of understanding in regards to the way people approach seemingly benign text and imagery. Further, I fully understand and agree with all responses, even oppositional ones. Each user’s response to each prompt allowed me as facilitator to understand each experience from a firsthand perspective. Overall, though the testing provided me with some potentially useful data on the current state of Summers-Knoll.org, I believe the greater benefit was how the exercise allowed me to experience multiple perceptions simultaneously. The experiment evolved into a lesson for me on how to communicate with users, how to ultimately relinquish control to them, and how to submit to their leadership within the boundaries of my loosely constructed framework. It was as much a social science experiment as a test of a particular web site’s usefulness and usability.

Praxis: Implications for Future Testing

As stated previously, this was my fist attempt at facilitating web site usability testing. It proved valuable in helping me organize the test, recruit test subjects, and navigate the numerous variables involved. Though I closely followed the suggestions of Steve Krug, I also developed my own set of best practices based on interactions with my six users. Overall, this was a positive introductory experience for me, and it will provide invaluable help in designing and implementing authentic, larger-scale usability tests of Summers-Knoll.org after the completion of the current design augmentations. This testing experience also prepares me for my planned winter 2011 testing of SK’s digital student portfolios. As one component of my Written Communication MA graduate project, my digital portfolio testing with both parents and teachers will now be informed by both theory and practice.

References

“History.” Summers-Knoll School. (2010, October). Web. Retrieved from http://www.summers-knoll.org

Krug, S. (2010). Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. Berkeley, CA: New Riders

Further Evolution of Research Question and Problem Statement

October 24, 2010

Graduate Project Problem Statement


Graduate Project Background

The culminating graduate project for my Written Communication, Professional Writing MA at EMU involves working with teachers and parents of students at Summers-Knoll (SK) School in Ann Arbor, Michigan. Summers-Knoll is an independent, kindergarten through fifth grade school that focuses on traditional liberal arts content delivered in a cross-curricular, experiential format. Through embedded learning and developmental objectives and qualitative performance assessment, students’ energies are channeled into how to learn as opposed to what to learn. Students and families are involved in curricular decision making, and they also play active roles in assessing student performance through the use of the school’s two primary assessment tools: Work Sampling System (WSS), a narrative-based assessment tool developed by the University of Michigan, and portfolios. I taught grades one through seven (the middle school program has since disbanded) at SK from 2002 to 2008. I remain committed to the school’s educational philosophy and mission, and I am excited to be working with the SK community again, albeit in a new function.

SK is currently working with graduate students from the University of Michigan’s School of Information to develop digital student portfolio wireframes. The head of school at SK has asked me to participate in the project by working on a team to develop the communication component of this venture, facilitate usability testing on the completed wireframes, and conduct secondary research into the implementation of digital portfolios in K-12 settings and their ability to capture quantitative student performance data via qualitative means. Further, I will facilitate usability testing on the school’s own website and evaluate WSS’s web-based version (teachers currently produce WSS evaluations in hardcopy) through additional usability testing, teacher and parent interviews, and teacher and parent questionnaires. My data collection will begin in winter 2011, and I will wrap up my study by late summer 2011. Little research exists on digital portfolio implementation and assessment in elementary school settings, and I hope to contribute meaningfully to this burgeoning field of inquiry.

 

The Problem

SK practices progressive instruction and student evaluation. Parents of students at SK become familiar with the school’s philosophy and practices during the in-depth application process. Despite the parent buy-in, or understanding, of SK’s teaching methodologies, a minority of parents expresses concern that skills developed at SK may not be transferable to more conventional learning environments. This concern appears to heighten as students rise through the grades and middle school looms. Though the school offers annual, optional, standardized testing to students in grades 4-5, some parents would prefer a balanced assessment approach that incorporates norms-based measures. While this is contrary to the philosophy of SK, I believe that quantitative performance data can be generated if SK switches to digital portfolios and the web-based WSS assessment tools. I posit that these data can be — and already are — generated via the current assessment tools. What digital portfolios and web-based WSS assessments offer are reinterpretations of student performance data that can be captured in quantifiable terms. By moving student assessment onto digital platforms, SK can remain faithful to the mission of its founders, incorporate technology into instruction and assessment in meaningful ways, and meet the needs and wants of parents with seemingly conflicting agendas.

 

My Research Questions

Should Summers-Knoll School move from hardcopy to digital portfolios? Should Summers-Knoll School move from the pen and paper version of the University of Michigan’s Work Sampling System to the new web-based version?

Lit Review: Getting Started

October 24, 2010

The following is a summary of three research articles for Dr. Allen’s ENGL 621 and also serves as the beginning of the literature review for my graduate project.
_________________________________________________________________________

Summary of Three Research Articles

My graduate research project involves working with parents and teachers at Summers-Knoll (SK) School in Ann Arbor, Michigan, an independent kindergarten through fifth grade learning center, to determine whether or not the school should incorporate digital student portfolios into their assessment program. The project entails a variety of research methods including usability testing of University of Michigan School of Information-designed electronic portfolio wireframes, interviewing of parents, teachers, and staff at SK, and extensive secondary research into the use of multimedia, digital assessment of elementary-aged students. The following is a brief review of three sources I will use as part of my literature review for this project.

 

Powerful Portfolios for Young Children, Hilary Seitz and Carol Bartholomew

Published in 2008 in the Early Childhood Education Journal, “Powerful Portfolios for Young Children” is a good starting point for my research as it provides an introductory overview of elementary portfolio assessment. While this essay is not specifically about digital portfolios — there is one reference to “electronic portfolios” — the arguments the authors posit transcend the platforms of hard and softcopy. The focus of any portfolio assessment, write Seitz and Bartholomew, “is to support a child (learner) with their understanding of concepts, ideas, and emotional self” (p. 63). Though portfolios historically provide more qualitative assessment of student performance, Seitz and Bartholomew argue that they additionally “provide[s] evidence of accountability of meeting standards, which may be of interest to administrators and even parents” (p. 63). This statement is particularly relevant to my research as one of the concerns of some parents of students at Summers-Knoll School is that current assessment practices provide little or no quantitative data to measure student achievement. While teaching methods at SK are specifically designed to lean toward qualitative results — i.e., growth over time, personal best achievement, etc. — providing parents with a balanced assessment program that includes traditional, norms-based measures (without compromising school philosophy) is the school’s best bet for satisfying the needs and wants of its diverse parent pool. Providing parents with documentation of a portfolio’s ability to adequately reflect content mastery within the framework of a more progressive learning environment is key for parent/teacher communication at SK. Seitz and Bartholomew offer foundational evidence that will help me construct a comprehensive argument to support this position.

 

Balancing the Dual Functions of Portfolio Assessment, Ricky Lam and Icy Lee

Lam and Lee’s essay, published in 2009 in Oxford University’s ELT Journal, a quarterly publication dedicated to scholarly research on English as a foreign or second language, is interesting as it focuses specifically on using portfolio assessment to explore formative English writing development of non-native speakers. Using 31 Hong Kong-based, non-English majors whose first language is Cantonese, Lam and Lee studied the effects of portfolio assessment on student improvement over time and levels of student satisfaction in participating in formative evaluation measures.

Lam and Lee’s study begins with an overview of portfolio assessment (PA): “While PA traditionally has a prominent summative function of providing a summary of students’ writing achievements in the writing process, it also enables teachers to provide ongoing feedback that informs both teaching and learning” (p. 54). They then set up their central question of how PA can be used for formative evaluation, stimulate meaningful student reflection, and align teaching with assessment. Through the use of student questionnaires, student interviews, and instructor interviews, the authors illustrate the measurable benefits of writing and language proficiency when approached from a formative perspective. After the implementation of PA, the data indicate students’ writings were more grammatically and structurally accurate and that students generated more and better ideas in their assignments.

Most interesting to me about this article is the culture-based resistance to PA assessment from students and teachers involved in the research. Despite the measurable improvement in the students’ writing, most students and teachers involved in the experiment remained unconvinced of the study’s value. The authors suggest this is because the participants were “brought up in a norm-referenced assessment culture which accustomed them to comparing themselves with each other rather than with their own previous learning” (p. 61). Despite this prevailing sentiment, some instructors were able to identify shortcomings of norm-based performance measurement, including Instructor 1 who wrote:

“A big problem of traditional assessment now is that students do not learn, they are only concerned about the marks and they do not know how to improve. Portfolio assessment is different, because students receive ongoing feedback which is purely formative, and they have to improve based upon they comments they received” (p. 60).

Overall, I enjoyed this essay very much. The data are valuable in supporting my own research, but more important is the potential cultural identification may have on understanding the benefits of portfolio assessment. This had not occurred to me before, and it may be relevant in addressing the needs and wants of Summers-Knoll School’s multicultural student/parent/teacher population.

 

Improving Literacy and Metacognition with Electronic Portfolios: Teaching and Learning with ePEARL, Elizabeth Meyer, Philip C. Abrami, C. Anne Wade, Ofra Aslan, Louise Deault

This essay is the result of a small group study of scholars from Concordia University and McGill University, both in Montreal, Quebec. Published in 2010 in the journal Computers & Education, Meyer et al.’s report focuses exclusively on the application of the electronic portfolio as a meaningful assessment tool in K-12 education. Straightaway the authors offer a concise definition of electronic portfolios: “An electronic portfolio (EP) is a digital container capable of storing visual and auditory content including text, images, video and sound” (p.84). They then offer this more comparative definition: “EPs are the Information Age’s version of the artist’s portfolio for students in the sense that they not only summarize a student’s creative achievements but also illustrate the process of reaching those achievements” (p. 85).

The authors provide extensive detail about the tri-level purposes of EP — process, showcase, and assessment — and their ability to stimulate metacognitive processes including planning, monitoring, and regulating. They posit that by engaging K-12 learners in metacognitive activities, such as building and maintaining digital portfolios, the students will “learn how to learn” and, perhaps most important, “their deficiencies in core competencies may be overcome” (p. 85). This final point is especially important to my research as it suggests quantitative measures may be achieved as a byproduct of qualitative assessment.

The participants in this study were 32 teachers from grades 4-6 and their 388 students from nine urban and rural English school boards in Quebec, Manitoba, and Alberta. The study took place during the 2007-2008 school year. Teachers received training in a specific digital portfolio tool called ePEARL, and teachers and students used ePEARL, at varying levels, throughout the year. Through the use of student pre- and post-tests, evaluation of ePEARL data, and several student and teacher questionnaires, the researchers found that “students who used ePEARL in medium or high implementation classrooms demonstrated learning gains on a standardized literacy measure and reported positive changes in key SRL [self-regulated learning] skills” (p. 89). Further, “teachers in these classrooms also reported how the use of ePEARL had a positive impact on their SRL teaching strategies” (p. 89). These findings are particularly interesting to me because they specifically address the use of digital portfolios (as opposed to hardcopy) in elementary assessment. Much has been written about digital portfolios in support of teacher preparation and professional development, but little research has been conducted on the benefits of their application among elementary-aged students. I hope to contribute to this burgeoning field of inquiry, and this study provides me with additional data to support my hypothesis. Further, it points me in the direction of a free, web-based portfolio tool, ePEARL, which I will investigate and perhaps incorporate into my study.

 

References

Lam, R., & Lee, I. (2010). Balancing the Dual Functions of Portfolio Assessment. ELT Journal, 64.1, 54-64. doi:10.1093/elt/ccp0

Meyer, E., Abrami, P.C., Wade, C.A., Aslan, O., & Deault, L. (2010). Improving Literacy and Metacognition with Electronic Portfolios: Teaching and Learning with ePEARL. Computers & Education, 55, 84-91. doi:10.1016/j.compedu.2009.12.005

Seitz, H, & Bartholomew, C. (2008). Powerful Portfolios for Young Children. Early Childhood Education Journal, 36, 63-68. doi:10.1007/s10643-008-0

Evolution of Research Question

October 11, 2010

Research Question: Where I Started
After brainstorming ideas for a couple of weeks – with the help of classmates and professors – I settled on the following question to guide my research project:

Do experiential learning and narrative-based assessments adequately prepare K-5 students for future success in more conventional learning environments?

After receiving feedback from peers, the question morphed into:

Do experiential learning and qualitative assessments adequately prepare K-5 students for transference of knowledge, skills, and experiences to other learning environments?

Background
Though I am in the Professional Writing track of the Written Communication program, my background is in elementary education, and many of my educational/professional passions are in this area. I am especially interested in curricula design, experiential learning, collaborative learning, integrated curricula, and balanced liberal arts with a healthy emphasis on fine art, music, drama, and sport. I taught elementary and middle school in Ann Arbor for six years, substitute taught for an additional four years, and I’ve been providing one-on-one English, ESL, and math instruction to learners at a variety of levels for about ten years.

Beyond issues of learning and instruction, I’m very interested in parent/teacher communication, teacher/teacher communication, and qualitative, narrative-style assessment of student academic performance. I list these interests separately, because, though still education-focused, they each revolve around specific professional written practices. Often these practices are overlooked or misidentified when scholars consider K-5 education, and I prefer taking a more holistic approach that includes all potential players and communities. Ignoring teacher/teacher communication practices in favor of focusing exclusively on instruction/curricula delivery, for example, ignores an integral component of K-5 culture. Same goes for relegating parent/teacher contact to report card checklists and semi-annual face-to-face meetings. I believe equal focus on all of these K-5 situations, contexts, and relationships is necessary for student performance and growth, teacher empowerment and satisfaction, and parent buy-in and support. A popular K-5 motto is, “It’s all about the children.” I disagree. It’s all about the community, and the wants and needs of all members are valid and important and must be considered for the success of everyone.

Summers-Knoll School

From 2002-2008 I taught grades 1-7 at Summers-Knoll (SK) School in Ann Arbor, Michigan. I left SK two years ago to take a position as a technical editor with Cape Henry Associates, Inc., a government subcontractor providing training, education, and manpower analyses and recommendations for a variety of government entities including the US Navy. I really enjoy my new job, and the people I work with and for are top notch. Though I now spend my days in front of a computer instead of in front of a classroom, I’m still able to interact with students each week via one-on-one tutoring. I’ve tutored a variety of subjects in the past, but am currently focusing solely on English.

I also remain in touch with my former colleagues at Summers-Knoll. When I began brainstorming for grad project ideas, I contacted Joanna Hastings, head of school at SK. Joanna had several ideas for ways I could get involved with SK that would allow me to give back to the school that’s given me so much and also provide me with experience, data, and deliverables to satisfy requirements for my MA program – a win-win situation. The two ideas of Joanna’s that I latched onto are:

1. Grad students from UM’s School of Information are currently building digital portfolio wireframes for SK. These portfolios will be used to document and present student work and move beyond current hardcopy portfolios by allowing easier inclusion of video and sound. Joanna suggested I might provide the communications component of this venture. This potential project also leads to several other related activities including usability testing of the school’s web site and participation in the development of a parent portal at Summers-Knoll.org, both of which I’m interested in pursuing.

2. Summers-Knoll’s focus is on experiential, project-based learning within a larger liberal arts framework. The school’s primary academic assessment tool is Work Sampling System (WSS), a narrative and portfolio-based qualitative assessment application developed by the University of Michigan. It is the best assessment tool I’ve ever worked with. A concern, however, of a minority of SK parents is the tool’s inability to provide quantitative evaluations of kids’ work. Some parents fear SK’s curricula may not prepare children for future success in more conventional learning environments, and WSS is unable to capture meaningful data that demonstrate the ability of SK learners to transfer learning to new contexts. Joanna asked if I could figure out a way to demonstrate what we both know theoretically and anecdotally. I agreed, and think the answer may be as simple as an in-depth literature review and teacher testing of WSS’s new web-based application. (My hunch is WSS online will make it easier for teachers to run more quantitative reports on student achievement and growth over time.)

Research Question: Where I Am Now

After a great face-to-face with Dr. Krause on 8 October, I decided to limit my project to just the digital portfolios and web-based WSS. My plan is to upload sample digital portfolios and sample classroom assessments on WSS. I will conduct teacher usability studies on both and perhaps parent usability studies or roundtable discussions as well. I will work with SK on developing a parent portal at their school web site, and have already begun usability testing of this site to meet requirements for my current Advanced Technical Writing class with Dr. Benninhoff. (Work on Summers-Knoll.org may or may not be included in my final grad project.)

Currently, my research question looks something like this:

Should Summers-Knoll School adopt the web-based Work Sampling System and UM-designed digital portfolios as part of their permanent student assessment program?

My literature review will focus on the use of digital portfolios in elementary education, Work Sampling System, and general assessment of experiential learning. I’ve got another meeting with Joanna Hastings this week. If she approves my current plan (which I’ll flesh out even further beforehand), then I’ll meet with SK’s homeroom teachers next week to get them on board. I plan to position my project to them as a study of their communication practices. I think they’ll be more receptive if they feel they’re in the driver’s seat.


Follow

Get every new post delivered to your Inbox.