By Tiffany A. Koszalka, Ph.D. | IDD&E Professor
The spring 2021 semester was a challenge as the university continued to focus on the health and safety of students, faculty, and staff and we started back into the classroom. One IDD&E doctoral seminar was offered as a residency, or campus-based course. We, of course, had to be ready to revert to online if necessary. IDE 800 Field Observational Research in Online and Blended Environments II faced many logistical challenges as students chose to participate in the classroom or at a distance each time we met. We ended up meeting six times, mostly through HyFlex—in class and at a distance, simultaneously. In these sessions, students in the classroom were masked. This format allowed us to more easily record each session, hear each other, engage with each other, share progress on coursework, and collaborate during discussions and showcases of work.
Based on request, we reverted to fully online mode later in the semester. As a seminar, a majority of the course work consisted of outside-of-class activities which focused on students working on their own research topic. For instance, practicing observation techniques with student-created instruments and protocols through video-based scenarios or conducting actual research through live Zoom observations. To prepare for in-class time, which was dedicated to sharing and discussing observation fieldwork, students also paired in teams to review each other’s work and practice techniques together via Zoom.
These live sessions were dynamic and allowed each student to share work, discuss progress and challenges, and contribute ideas to each other. Their topics for research were widely different in context, allowing each to share experiences and educate others on both a phenomena of study and observational field research experiences. Take a look at the profiles of each student’s research area.
Investigating students’ cognitive presence and content knowledge levels in asynchronous online discussions
Students construct knowledge during asynchronous online discussions (AODs). Discussion patterns can show students’ cognitive presence (CP) (i.e., engagement) and level of understanding (i.e., learning). The purpose of this study was to investigate CP patterns and content knowledge levels demonstrated in AODs transcripts from a cohort of 12 graduate students across two courses over two semesters. Preliminary analysis results revealed that students’ CP patterns mostly concentrated on the “exploration” category and most of the discussions demonstrated deeper levels of content learning. The study further unearthed evidence of how students probed their conversations in various ways under the “exploration” category. The longitudinal patterns and relationship between CP and demonstrated knowledge level are continuing to be examined. —Zhijuan Niu (Jenny; China), IDD&E first-year doctoral student
Standardized patient educators’ use of evaluation in the instructional design of human simulation training procedures
Standardized/Simulated Patients (SPs) are the live people who portray human roles in simulations for the education and assessment of healthcare learners and practitioners. SP educators (SPEs) are the professionals who design, develop, and implement SP simulations; manage SP programs; recruit and train SPs; and assess SPs & evaluate SP training procedures. The aim of the study was to explore how SPEs’ SP training procedures are evaluated. Preliminary results suggested that annual SP performance assessment was mandated after SP Program accreditation was sought. SP performance evaluations are perceived to be robust and consistent. Data is infrequently collected about SP training procedures, though the SPE desired to obtain this information if not for a lack of time and heavy workload. The SPE reported to constantly reflect on their own job performance, though this was usually an internal (non-written) process. Further research will help close knowledge gaps about how SP training procedures are evaluated. —Amber A. Walton (U.S.), IDD&E second-year doctoral student
A study of establishing validity of a learning resources rubric
Instructional designers (IDs) create or select learning resources and platforms for university faculty to support student learning. However, with the advent of the digital era and increasing access to learning resources, choosing and integrating learning resources into instruction to prompt deep learning in an effective and efficient way presents a challenge for IDs and educators alike—especially novice IDs. To aid ID, the study described the process of creating and testing a learning resources rubric based on principles from three established learning theories of deep learning: generative learning, cognitive flexibility, and reflection theories. The purpose of this study was to establish the content validity and usability of the proposed learning resources rubric (LRR). The LRR was originally comprised of 10 items. The validity study consisted of a group of novice instructional designers (NIDs), sought from existing social media groups that focus on instructional design practices, who were prompted to review and use the LLR. Quantitative and qualitative data were collected during observations of the NIDs using the LLR both to critique sample learning resources and design a learning resource during a 3-phase field observation study. The preliminary data analysis suggested that minor revisions were necessary to clarify and perhaps combine items on the LLR. Additional analyses are being planned to further examine the validity of the LLR and prepare it for factor analysis studies. —Lei Wang (China), IDD&E second-year doctoral student
Developing and validating an observational checklist to support comparison studies among face-to-face and online courses
The question about equivalency between face-to-face and online learning is one that has recently skewed toward online learning. Many studies now suggest that online instruction has the same results as, or better than, residency instruction. Yet these studies often fail to describe the design of the courses being compared, which raises the question about the equivalence of course designs, potentially suggesting apples-to-oranges kind of comparisons. This study was focused on developing and validating observation instruments (i.e., design inventory and interaction observation protocols) to help effectively compare face-to-face and online learning formats using observable measures, namely interaction constructs. The instrument adopted the transactional theory’s three types of interactions to observe instructional and learning activities in both platforms—online vs. classroom. The three types of interactions include learners-to-learners, learners-to-instructor, and learners-to-content. The instrument’s goal, as an observation protocol, was to identify critical factors that provide a foundation of “sameness” to compare different learning platforms. Conceptual validity (based on the transactional theory and representation research studies) has been achieved based on rigorous literature reviews. This current study was to gain both expert and user validity of the protocol: (a) are the items aligned with the theory?, (b) do they represent observable measures of interaction?, and (c) is the instrument useable? Experts have suggested that items represent transactional theory well. We have created observer training to prepare users to test the protocol instrument and are planning user validity tests. —D. Garmondyu Whorway (Liberia), IDD&E second-year doctoral student
A longitudinal study exploring levels and patterns of social presence in asynchronous online discussions
A majority of the social presence research have sought to evaluate and measure students’ own perceptions of social presence by solely employing self-report surveys and sometimes supporting these data with interviews. These types of data overlook the importance of assessing and observing student’s actual social presence behaviors through specific indicators. In addition, many of the previous studies were conducted as short-term studies in the same context with different participants. The purpose of this study was to explore social presence in asynchronous online discussions (AODs) beyond self-report surveying. Observation techniques, document analysis, and social network mapping were employed to provide another point of analysis including social participation and relationships. The research questions addressed what patterns and levels of social presence in AODs were present across two asynchronous online courses over two semesters. This study used the community-of-inquiry framework as a coding scheme to identify constructs of social presence in the AODs through qualitative content analysis. A social networking mapping tool was used to create visuals of the student interactions (relationships) showing frequency of postings to other students. The data analysis is currently being conducted. —Qi Wu (China), IDD&E first-year doctoral student
Role of instructor’s theory in instructional decision making
Faculty are responsible for making central decisions about designing the learning experiences and calibrating their designs to meet program, school, institutional, and accreditation standards. This study highlighted the central role of a faculty’s personal approach to instructional design called Instructor’s Personal Instructional Theories (IPIT) in shaping their course decisions. Understanding IPIT is central to providing adequate support (professional development or instructional design consulting) to faculty to improve their decision making. Therefore, this study aimed to understand how IPIT influences the design of their courses and to identify the subcomponents that shape IPIT through a grounded theory approach. The study focused on understanding the design decision making of teaching-award winning faculty, i.e., experts. The research employed observational methods of document analysis and talk-aloud observations/interviews to understand how instructors identified and described factors affecting their decision making in planning, executing, and modifying (live or just-in-time) their course activities. The initial analysis of these observations and documents led to a better understanding of the types of preferences, experiences, and other factors that influence award-wining instructors’ decisions—factors that are generally not in design models or professional development events. —Zeenar Salim (Pakistan), IDD&E third-year doctoral student
Summary
These doctoral students have been exploring observational techniques to investigate phenomena of their own interests. Through readings, course practices, collaborative discussions, and showcase critique sessions they learned with each other often in a connected, distant format. Several are using this early research practice to begin forming their ongoing research topic. Others are a little farther along in their doctoral preparation and using this course as a kick-off to their required research apprenticeships projects on the way to dissertation. Two have already used preliminary data results in conference presentation proposals and at least three will be preparing papers for journal submissions by the end of Summer 2021. Although these doctoral students may end up using other research designs in their dissertations, this course has provided them with a foundation and practice in observational techniques, data analysis, and publication. Given the sometimes trying technical and distant environment, all made much progress over the year—even in a pandemic we were successful in building competence and confidence in scholarly work.
Word Count: 1,661
No Comments