Rebecca K. Miller
AECT Domain: Evaluation
Definition. Evaluation focuses on determining the "adequacy" of the learning materials designed, developed, and utilized (Seels and Richey, 1994, p. 54). True evaluation occurs throughout the design, development, and utilization processes, and assists designers and instructors in deciding how to adjust the materials, and whether or not to continue using them in certain situations. Specifically, evaluation contains four subdomains: problem analysis, criterion-referenced measurement, formative and summative evaluation, and long-range planning.
Reflection. One of the concepts that was new to me as I learned about evaluation through the IDT coursework was that evaluation occurs throughout the instructional design process. When I heard the term "evaluation," I always interpreted that as meaning something more like "assessment" at the end of a project. However, I learned that evaluation occurs at every point throughout the process, starting with front-end evaluation. Much like I mentioned in my reflection on the design process, while I may not always have time to focus on evaluation in real life the way that I have been able to through my IDT coursework, I am convinced that it is important to always do a little bit.
Problem Analysis
Needs Assessment for Food, Fun, and Good Nutrition Module (web link)
Collaboratively written for coursework completed in EDIT 5164: Principles of Instructional Design, this needs assessment outlines the nature and parameter of the problem--rising childhood obesity rates--and identifies a specific gap in knowledge that the instruction that will be designed and developed endeavors to bridge.
Criterion-Referenced Measurement
Practice and Assessment Items for APA Citation Tutorial (PDF here, web links below):
The PDF and links above describe the criterion referenced practice and assessment items used for the APA Citation Tutorial developed in EDIT 5534: Applied ID Theories. The PDF links to the alignment chart, which describes the objectives, practice items, and test items. A rubric and answer key are also included in the PDF. The links below the PDF are Google Docs, representing what the learners actually see when they are accessing the practice and assessment items. Learners' performance on these items help determine the adequacy and efficacy of the instructional materials.
Formative and Summative Evaluation
Long-Range Planning
Formative Evaluation Plan for Food, Fun, and Good Nutrition Tutorial (web link)
Developed as part of EDIT 5164: Principles of Instructional Design, this formative evaluation plan identifies subject matter experts and target learners as two specific groups of people to work with when completing formative evaluations in order to further develop the instructional material. The field testing phase is also described as a way to identify areas that may be inadequate or ripe for improvement within the instructional material.
Organizational Learning and Innovation in the Academic Libray (PDF)
This literature review and analysis is also linked the Research area of this portfolio. Written as part of the coursework within EDIT 5234: Intro to the Learning Sciences, this paper grapples with ideas drawn from the learning sciences; however, much of the paper is focused on how academic libraries can utilize organizational learning in order to plan strategically and ensure organizational success. While it does not offer a detailed plan for a particular organization, this research paper does touch on a number of case studies and summarize a number of ideas that contribute to long-range planning in academic libraries.
Seels, B. B., & Richey, R. C. (1994). Instructional technology: The definition and domains of the field. Washington, DC: Association for Educational Communications and Technology.
Needs Assessment for APA Citation Tutorial (PDF)
Similar to the needs assessment linked above, this needs assessment was written for coursework completed in EDIT 5534: Applied ID Theories. It highlights the importance of gathering specific evidence about the problem that has been identified in order to safely decide that instruction is, indeed, the solution to that problem.
GRAD 5124 Full Evaluation Report (PDF)
This document represents the full evaluation report for the GRAD 5124 program. The evaluation was conducted as part of the coursework for EDIT 5584: Program and Product Evaluation. It is both summative and formative in nature, as it attempts to evaluate the adequacy of the course, as it currently exists, and identify areas in the course that should be discontinued. The findings from this report are being used in an on-going modification of the GRAD 5124 course.