Annotated Bibliography: Digital Portfolios and Portfolio Assesssment

Berrill, D.P., & Whalen, C. (2007). “Where Are the Children?” Personal Integrity and Reflective Teaching Portfolios. Teaching and Teacher Education, 23.6, 868-884. doi:10.1016/j.tate.2007.02.002

Berrill and Whalen’s study chronicles a two-year, school wide, reflective teaching portfolio initiative in a British elementary school. The authors posit that personal professional portfolios (PPPs) offer visual, textual, and sometimes aural demonstrations of epistemological (valued knowledge and skills), ontological (the nature of knowledge and skills in historical, social, and cultural contexts), and axiological (values and beliefs associated with knowledge and skills) learning. The authors originally introduced the reflective PPPs as a method for teacher demonstration of government mandate compliance. Over time, though, teacher participants took ownership of the portfolios and used them as visual demonstrations of student learning and development. The researchers found that teacher autonomy was key in generating both perceived and actual value of the portfolios. Further, each teacher’s felt importance was necessary, both via writing explicit belief statements and through implicit design choices, for recognizing and articulating the unique focus of each classroom and the contributions of each child. This awareness and expression of individual and group identity through reflection is aligned with sociocultural historical theory and postmodern identity theory.

Though the focus of my research is on portfolio evaluation of students, especially through the use of digital means, this study introduced me to ways in which teachers might use PPPs to assess large groups of children within the context of a specific teacher’s tutelage. PPPs are an additional tool I had not previously considered, and they might prove valuable for establishing teacher/classroom identity to be used both for teacher reflection and school marketing purposes.

Chang, C. (2001). A Study on the Evaluation and Effectiveness Analysis of Web-Based Learning Portfolio (WBLP). British Journal of Educational Technology, 32.4, 435-458. doi:10.1111/1467-8535.00212

Chang’s study focuses exclusively on digital portfolios for use in student assessment and reflection in higher education. The report begins with a brief overview of portfolio research including the following benefits per Hewitt’s 1995 research study:

1. Portfolios demonstrate students’ growth
2. Portfolios encourage students to set up learning goals
3. Portfolios provide evidence of students’ efforts
4. Portfolios demonstrate students’ performance
5. Portfolios help teachers review students’ performance
6. Portfolios stimulate students’ introspective thinking and enhance self-assessment
7. Portfolios encourage students’ learning interests, improve their communication skills, and build self confidence (p.436)

Chang and colleagues at National Taipei University of Technology designed web-based learning portfolios (WBLP) and evaluated them using student usability participant questionnaires, user interviews, and digital portfolio expert interviews. They identified limitations in their research and evaluated the findings of their research offering a heuristic for planning and designing WBLPs for student use.

Though this study focuses specifically on higher education, much of the research is directly applicable to my interests in K-5 WBLPs. Of special interest to my research is Chang’s recommendation of WBLP contents including specific wireframe recommendations. This study also offers an excellent example of obtaining and triangulating both quantitative and qualitative data.

Chang, C. (2008). Enhancing Self-Perceived Effects Using Web-Based Portfolio Assessment. Computers in Human Behavior, 24, 1753-1771. doi:10.1016/j.chb.2007.07.005

Chang’s study investigates how implementing a web-based portfolio assessment tool influences both real and perceived academic achievement among students. His extensive literature review highlights research-based benefits of digital portfolio assessment including identifying student skills rather than shortcomings, identifying unique needs of each student, increasing student motivation, and encouraging students to develop decision-making skills (p. 1756). Perhaps most important is that digital portfolios cultivate student assessment and development abilities not limited to student evaluation.

For his study, Chang used two junior high school computer classes of 30 students each. One class used and tested digital portfolios and the other used traditional hardcopy assessment methods. Data collection was done through the use of student questionnaires, tests, and expert-designed portfolio assessment forms. Chang found that web-based portfolio assessment does not have significant influence on student achievement, though it does have significant and positive effect on self-perceived learning performance. Further, his research determined that digital portfolio assessment is especially beneficial for poorly motivated students in terms of increasing motivation and self-perceived learning performance.

This study is interesting because its findings are in contrast to the bulk of the research that indicates portfolio assessment can improve learning performance. Nonetheless, Chang’s report has relevance to my study due to his findings on motivation and self-perception. This may indicate that digital portfolio assessment may be beneficial to students with low motivation and/or self-esteem, underperforming students due to at-risk classification, and special needs students including twice exceptional and gifted learners.

Chang, C., Tseng, K.H. Use and Performances of Web-Based Portfolio Assessment. British Journal of Educational Technology, 40.2, 358-370. doi:10.1111/j.1467-8535.2008.00885.x

This is the third in a series of studies by Chang, this one conducted with colleague Tseng. The stated objective of the authors’ research was to explore the influence of web-based portfolio assessment on middle school students’ performance. The data from this study appear to be the same as the last, though they have been reinterpreted by being filtered through alternative definitions of “performance.” Chang and Tseng “exclude the interference of academic achievement and computer achievement” (p. 367) from their data evaluation and instead focus on a variety of performance indicators such as goal setting, reflection, peer assessment, peer interaction, data gathering continuous improvement, and problem solving. Performance in each of these measures was determined by the use of questionnaires, tests, and expert review of final products. Based on the new performance criteria, or a reinterpretation of already-existing data, the authors find “that the use of the web-based portfolio assessment system significantly affects the students’ performance” (p. 367).

This report is puzzling to me as it appears to be merely a repositioning of the data from Chang’s previous study in which he determined web-based portfolio assessment does not have significant influence on student performance. While he and Tseng do not necessarily argue otherwise here, they eliminate academic performance concerns entirely and focus instead on other positive byproducts of digital portfolio implementation. I was able to glean numerous benefits from the previously published study, and I found it much more balanced and detailed. This report seems rather superfluous to me.

Lam, R., & Lee, I. (2010). Balancing the Dual Functions of Portfolio Assessment. ELT Journal, 64.1, 54-64. doi:10.1093/elt/ccp024

Lam and Lee’s essay, published in 2009 in Oxford University’s ELT Journal, a quarterly publication dedicated to scholarly research on English as a foreign or second language, focuses specifically on using portfolio assessment to explore formative English writing development of non-native speakers. Using 31 Hong Kong-based, non-English majors whose first language is Cantonese, Lam and Lee studied the effects of portfolio assessment on student improvement over time and levels of student satisfaction in participating in formative evaluation measures.

Lam and Lee’s study begins with an overview of portfolio assessment (PA). They then set up their central question of how PA can be used for formative evaluation, stimulate meaningful student reflection, and align teaching with assessment. Through the use of student questionnaires, student interviews, and instructor interviews, the authors illustrate the measurable benefits of writing and language proficiency when approached from a formative perspective. After the implementation of PA, the data indicate students’ writings were more grammatically and structurally accurate and that students generated more and better ideas in their assignments. This finding is aligned with the majority of research on PA and performance.

The data from Lam and Lee’s essay are valuable in supporting my own research, but more important is the potential cultural identification may have on understanding the benefits of portfolio assessment. This had not occurred to me before, and it may be relevant in addressing the needs and wants of the multicultural student/parent/teacher populations of my own study.

Meyer, E., Abrami, P.C., Wade, C.A., Aslan, O., & Deault, L. (2010). Improving Literacy and Metacognition with Electronic Portfolios: Teaching and Learning with ePearl. Computers & Education, 55, 84-91. doi:10.1016/j.compedu.2009.12.005

This essay is the result of a small group study of scholars from Concordia University and McGill University, both in Montreal, Quebec. Published in 2010 in the journal Computers & Education, Meyer et al.’s report focuses exclusively on the application of the electronic portfolio as a meaningful assessment tool in K-12 education. Straightaway the authors offer a concise definition of electronic portfolios: “An electronic portfolio (EP) is a digital container capable of storing visual and auditory content including text, images, video, and sound” (p.84). They then offer this more comparative definition: “EPs are the Information Age’s version of the artist’s portfolio for students in the sense that they not only summarize a student’s creative achievements but also illustrate the process of reaching those achievements” (p. 85).

The authors provide extensive detail about the tri-level purposes of EP — process, showcase, and assessment — and their ability to stimulate metacognitive processes including planning, monitoring, and regulating. They posit that by engaging K-12 learners in metacognitive activities, such as building and maintaining digital portfolios, the students will “learn how to learn” and, perhaps most important, “their deficiencies in core competencies may be overcome” (p. 85). This final point is especially important to my research as it suggests quantitative measures may be achieved as a byproduct of qualitative assessment.

Ocak, G., & Ulu, M. (2009). The Views of Students, Teachers, and Parents and the Use of Portfolio at the Primary Level. Procedia Social and Behavioral Sciences, 1, 28-36. doi:10.1016/j.sbspro.2009.01.009

The objective of Ocak and Ulu’s study was to gauge the opinions of students, teachers, and parents on the use of portfolio assessments in learning. The authors begin with a succinct definition of the academic portfolio: “A student portfolio is a carefully selected collection of student work that provides clear evidence to the student, parent, and…educators of the student’s knowledge, skills, strategies, grasp of concepts, attitudes, and achievement in a given area over a specific time period” (p. 28). They provide solid arguments in favor of alternative assessment including:

“Grades or percentages emphasize…a particular skill and show what students do not know.”

 

“Students’ evaluations should be based on the quality of their work and not on their ability to recall information.”

 

“Traditional tests do not assess the child as a whole. They do not reflect the current theories in learning, do not give a clear picture of how the child performs in the classroom, and do not provide usefulness for the future.” (p. 29)

Participants in the authors’ study were 37 elementary school teachers, 92 parents, 194 fifth grade students, and 178 eighth grade students. Students designed and constructed portfolios and presented them to peers, teachers, and parents. All participants were queried via open-ended survey questions and extensive Likert-type questionnaires. The resulting data indicate that portfolios foster communication among students, teachers, and parents; play prominent roles in student learning; assist students in understanding their strengths and limitations; and improve students’ critical thinking and evaluative skills.

This is an excellent study with data that mirrors much that I’ve hypothesized, and the authors demonstrate a deep understanding of the unique needs of contemporary learners: “Teachers are becoming facilitator[s] in the classrooms and students [are] become[ing] the constructor[s] of their knowledge” (28).

Peacock, S., Gordon, L., Murray, S., Morss, K., & Dunlop, G. (2010). Tutor Response to Implementing an ePortfolio to Support Learning and Personal Development in Further and Higher Education Institutions in Scotland. British Journal of Educational Technology, 41.5, 827-851. doi:10.1111/j.1467-8535.2009.00986.x

Peacock et al.’s study of the use of digital portfolios in higher education offers a unique definition of the ePortfolio: “An ePortfolio is an electronic system that facilitates the development, collection, and management of digital resources which may be drawn from a range of learner experiences over a period of time and could include those from formal and non-formal learning opportunities” (828). They further highlight that portfolios, as opposed to traditional assessment methods, illustrate both process and product providing formative and summative assessment of learners’ achievements.

The authors conducted 23 semi-structured interviews with higher and continuing education tutors at ten learning institutions across Scotland. Each of the tutors was already using an ePortfolio system in his/her teaching. The authors found that 100 percent of the participants valued the ePortfolio and wanted to continue to refine and develop its use in support of learning. Perhaps most important and relevant to my study of ePorfolios in K-5 education is that “the majority of tutors felt that ePortfolios provide an ideal opportunity to integrate reflection into the curriculum” (p. 839). One of the precepts of contemporary educational studies is that reflective learning results in increased retention and achievement. Therefore Peacock et al.’s study draws a direct correlation between digital portfolios and student performance that can be extrapolated to transcend the tertiary education context of this study.

Seitz, H, & Bartholomew, C. (2008). Powerful Portfolios for Young Children. Early Childhood Education Journal, 36, 63-68. doi:10.1007/s10643-008-02427

With a focus on portfolio assessment in early childhood education, Seitz and Bartholomew’s study zeroes in on the immediate benefits of portfolios for young learners and the pedagogical byproduct of demonstrating accountability of meeting standards and benchmarks: “Not only are we showing that children are meeting and understanding standards, but that teachers are teaching that which has been set out for us to do by federal, state, and local governments” (p. 64) They provide further arguments for the benefits of portfolio construction and assessment. Perhaps most important is that portfolios empower children by compelling them to take responsibility for their learning and allowing them to become partners with teachers in the classroom. The authors also explore the reality that diverse populations employ a variety of learning methods, many of which are nontraditional and cannot be assessed via paper and pencil. A passing reference is made to the possibility “electronic portfolios or even web pages” (p. 66).

This essay provides some valuable, research-based arguments that will inform my own study. The glaring omission, however, is the role that technological tools can and should play in the development of K-5 student portfolios.

Vermilion, E.R. (2008). The EdVantage of Efolios. Distance Learning, 5.4, 67-72. Retrieved from http://www.infoagepub.com/distance-learning.html

Vermilion’s brief essay describes an electronic portfolio initiative in Manatee County School District in Florida. The school district began by establishing four primary objectives for all elementary students:

1. Continually demonstrate enthusiasm for the self-directed pursuit of knowledge
2. Articulate personal goals, create plans to achieve those goals, and exhibit progress toward their attainment
3. Continually participate in democratic processes
4. Actively engage in global outreach (p. 68)

Next, during a pilot study, volunteer teachers were trained in the use of digital portfolio software followed by implementation of eFolios in their classrooms. Teachers then trained their pupils in the use and management of the eFolios including categorizing each eFolio entry under one of the four district-wide objectives.

Vermilion reports anecdotal evidence of teachers and students in the eFolio classrooms as having improved their skills in technology, public speaking, and leadership. She further sites evidence of additional benefits of eFolio use, most interesting to me is that they allow cross-referencing of student work.

Wall, K., Higgins, S., Miller, J., & Packard, N. (2006). Developing Digital Portfolios: Investigating How Digital Portfolios Can Facilitate Pupil Talk About Learning. Technology, Pedagogy and Education, 15.3, 261-273. doi:10.1080/14759390600923535

Wall et al.’s study was borne out of the need for increased research in metacognition, formative assessment, and information and communications technology (ITC) in the primary grades. Wall and her colleagues conducted 14 case studies in 12 elementary schools across four local education authorities in northeast England. All primary age groups were represented in the study as well as one special education unit for children with communication difficulties. All of the schools were using digital portfolios for assessment, reflection, learning, and document/data storage. Citing the 1989 Article 12 of the United Nations Convention on the Rights of the Child to have a right to be involved in decisions affecting him/her, Wall et al. designed their study to focus specifically on the children. Open-ended interview questions were designed using bubble dialogue and response templates. Interviews were conducted by teachers in their classrooms. Data were evaluated by Wall and her colleagues. A key theme that emerged from the data is the pupil recognition that the process of creating a digital portfolio and the final end product are equally important as part of the learning process. Further findings include ICT can effectively support learning and enhance student autonomy; digital portfolios can capture learning in areas that are sometimes difficult to quantify, such as art, physical education, and design and technology; and ICT is an appropriate method for recording achievement of pupils who find writing difficult.