Abstracts Track 2021

Area 1 - Artificial Intelligence in Education

Nr:	8
Title:	Using Automated Measures to Predict Learner Engagement on an Online Language Learning Platform
Authors:	Xiaoqiu Xu and Deborah Dugdale
Abstract:	Engagement is essential to learning. Research has demonstrated that engaging learners in the learning process increases their attention and persistence in learning, motivates them to practice higher-level critical thinking skills, promotes meaningful learning experiences, and improves learning outcomes (Fredericks et al., 2004, Carini et al., 2006; Elyas & Al-Bogami, 2018). Various methods have been employed to measure learner engagement in K-12 educational settings, including participant surveys, class observations, and automated measures (Henrie, C.R. et al., 2015; Whitehill et al., 2014). In this study, we sought to investigate whether and how well artificial intelligence (AI)- powered automated measures can predict learner engagement on an online language learning platform. 159 learners enrolled in VIPKid Major Course (MC), an online English as a foreign language (EFL) tutoring program, participated in the study. MC, the main curriculum offered at VIPKid, conducts one-on-one EFL online classes facilitated remotely by English teachers based in the United States or Canada. Participants were studying at different levels of MC, and their ages ranged from 7 to 14 years old. After a 25-minute online class, each learner completed a learner engagement survey investigating their emotional and behavior engagement. Automated measures were also generated based on that class, including variables such as frequency of teacher and learner positive facial expressions, which were generated using facial recognition; speed and amount of teacher talk and student talk using speech recognition; frequency of learner front facial recognitions, indicating learner attention; and frequency of teacher gestures, etc. The model also includes numerous other variables that have been proven relevant and useful, including learner demographic information, learner proficiency level, and teacher information. Results demonstrated that young learners were highly engaged during the one-to-one online learning setting both emotionally (mean = 4.64, out of 5) and behaviorally (mean = 4.48, out of 5). The study also correlated the learners’ self-reported engagement with the engagement variables extracted by the AI and found that learners' front facial recognitions (indicating their attentiveness during class), time stamp of teachers’ last image (indicating how much longer the teacher stayed after class), and students’ MC levels are significant and positive predictors of learner engagement. Practical implications are discussed on how to further improve the automated measures and how to better use them to guide learning and teaching on the online learning platform.

Nr:	15
Title:	How, If at All, Do the Developers' Discourses Influence the Way They Design AI?
Authors:	Jason S. Fuller
Abstract:	Students, parents, teachers, building and district administrators continuously are bombarded with educational technology software options that promise to engage students and raise achievement. The industry is growing, with the education technology industry projected to triple from $107 billion in 2015 to $350 billion by 2025 (Gilchrist, 2020). This proposed presentation focuses on an often-understudied aspect of educational technology – the intended and unintended impact of developers’ Discourses (being-doing-valuing combinations, (Gee, 2001) on the design, intent, and goals of an Intelligent Tutoring System (ITS). More specifically, the software these designers developed uses Artificial Intelligence (AI) to make individualized decisions and predictions based entirely on data collected from kindergarten through 2nd grade students. Hearing from designers about their intentions of the software helps identify how their personal and joined beliefs, values, and experiences in education impact the overall student experience. Snowball sampling (Goodman, 1961) was conducted with developers and programmers in a single educational software company. Semi-structured interviews were coded and interpreted through Discourse analysis (Gee, 2014). A list of 21 questions were used to guide the interview. Developers were encouraged to ask clarifying questions or take their responses in the direction they deemed appropriate. I would often ask follow-up questions on either defining software specific terminology, how developers viewed themselves as readers and memories reading from childhood, or their personal goals for the software. Interviews were recorded via Zoom and the live transcript feature was used to create a script of the conversation. The first round of coding will be to identify general themes including “values”, “past experiences”, “impact”, “pedagogy”, “social-emotional development”, “intentional outcome”, and “unintentional outcome”. The following coding phase will seek out Discourse within the interview including “student”, “educator”, “expert”, and others manifested. During the first two coding phases, individual interviews will have different themes and Discourse identified specific to the individual participant. On the next phase of coding the transcripts, the interviews will be compared and calibrated for commonalities in themes and Discourse. Through this study, I first sought to understand how and why the particular software exists as it does. Hearing from the designers will help me know ways that the software is doing what it is intended to do and/or areas that I can offer suggestions to meet unanticipated, latent, and overt goals. This current research will ultimately impact my ability to design future targeted and impactful studies for the current ITS. Sharing the process and impact of starting with analyzing the Discourses of developers, prior to designing student focused studies, can influence the methods and research design of future studies in the field of AI and ITS. Explored further in this paper, is the opportunity for educational technology software developers to be agents of pedological and cultural change in education through the design, delivery, intended purpose, values, and impact of their software.

Area 2 - Information Technologies Supporting Learning

Nr:	12
Title:	Self Efficacy in e-Writing Portfolio of L2 Learning
Authors:	Hye A. Ryoo
Abstract:	The purpose of this study is to evaluate the pedagogical effectiveness of e-portfolios and process-oriented writing in second language acquisition. To that end, the study examined students’ motivation, perception, sense of anxiety and other affective factors which contribute to their confidence in utilising the second language. There were at least four affective factors at play: motivation, anxiety related to negative feedback from peers, discomfort with grade, and anxiety with direct interaction. For this study, fifty-nine university students enrolled in Korean intermediate classes were tasked with developing an e-portfolio over the course of a semester. Their e-portfolios were open to all, and the students were also directed to evaluate each other’s works. The web based exchange of writing drafts and process of peer-evaluation was broadly well-received. Qualitative analysis of survey responses revealed that students perceived this tool as generally beneficial and educationally meaningful, though a minority expressed some anxiety about possible negative feedback as well as some doubts as to their ability to provide effective feedback. A follow-up interview suggested that the general positive response was attributable to pre-writing interactions during the planning stage, the provision of an online peer-evaluation system during the writing stage, and the instructors’ comments during the feedback stage.

Nr:	13
Title:	"Simo-syl": A Computer-based Programme to Enhance Child's Emergent Literacy Skills
Authors:	Tamara Bastianello, Valentina Persici, Rachele Ferrari, Beatrice Bertelli and Marinella Majorano
Abstract:	Introduction. Reading acquisition requires the integration of several skills from different cognitive domains. Many studies have shown that the adoption of specific supporting programmes during preschool enhances early language and meta-phonological skills (Dickinson et al., 2010; Goldstein et al., 2017; Walker & Carta, 2020). For example, some studies have reported that a meta-phonological intervention based on the syllabic approach, which is aimed at reinforcing the child’s syllabic awareness skills to learn reading and writing, supports preschool and school-age children in the automatisation of reading skills (Bertelli et al., 2013; Padovani et al., 2018). Furthermore, computer-based or multimedia intervention programmes in preschool and primary school are useful not only to maximally engage children in the learning process, but also to support teachers in their didactic activities (Cojacariu & Boghian, 2014; Neumann & Neumann, 2017). Aims. The objective of the present study is to explore the efficacy of "Simo-syl", a 12-week multimedia intervention programme for enhancing Italian preschool children's early literacy skills. “Simo-syl” consists in a series of meta-phonological games led by an invented character, the little fish "Simo”. We expected the children participating in the Simo-syl programme to reach higher meta-phonological scores as measured through a computer-based assessment tool as compared to the children not participating in the programme (control group). Method. Sixty-three Italian pre-schoolers aged between 4;10 and 5;9 years were randomly assigned to one of two groups: the Simo-syl group participating in the intervention programme (n = 46) and the control group (n = 17). Children’s lexical, morpho-syntactical, meta-phonological, phonological, and phono-articulatory skills were assessed at the beginning of the last year of preschool using a standardised paper-based multidimensional assessment. Moreover, on the same day children’s meta-phonological skills (i.e., the ability to recognise syllables and vowels) and word reading skills were tested using a computer-based assessment. The intervention lasted 12 weeks and included three activities per week. Activities focused on syllable blending and spelling (week 1), on syllable recognition (weeks 2-11), and on vowel recognition (week 12). A few children in the Simo-syl group were tested again one week after the end of the intervention programme (Post-test) using the same computer-based assessment. Results. Preliminary analysis of the children's scores before intervention showed that the children in the Simo-syl and in the control group displayed similar scores (all ps >.05). To test the efficacy of Simo-syl in enhancing the children’s meta-phonological skills, four 2X2 repeated measures analyses of variance (ANOVAs) were conducted using time (before and after intervention) as within-subject factor and group (Simo-syl and control group) as between-subject factor. An interaction effect between time and group for the word reading task emerged (F(1, 28) = 6.83, p = .014, ηp2= .196). Post-hoc tests with the Bonferroni correction revealed that the Simo-syl group showed a significant improvement in word reading after the intervention (t(28) = -6.811, p <.001). Discussion and Conclusion. The Simo-syl intervention is shown to enhance the children's word reading skills in 12 weeks. Further studies with a larger number of participants are needed to confirm these results.

Nr:	14
Title:	How Are Computational Concepts Learned and Taught?: A Thematic Analysis Study Informing the Design of an Augmented Reality Coding Platform
Authors:	Thomas L. Sharkey, Robert Twomey, Amy Eguchi and Ying Choon Wu
Abstract:	How could social, collaborative, immersive coding experiences anchored in 3D physical space transform the teaching and learning of computational concepts? The concept of implementing a 3D block or node-based coding platform in Augmented Reality (AR) in support of collaborative and body-based engagement is motivated by foundational work on the benefits of pair programming (Hanks et al, 2011) and collaborative learning (Johnson & Johnson, 1987). It is also motivated by findings that science, math, and even computational concepts are fundamentally rooted in sensorimotor experience (Lindgren & Johnson-Glenberg, 2013; Lakoff & Núñez, 2000; Black et al, 2012) or body syntonicity (Papert, 1993). Further, extensive evidence suggests that body movements in general support brain function and cognition important for learning, memory, creativity, and problem-solving (Basso & Suzuki, 2017; Oppezzo & Schwartz, 2014; Statton et al, 2015; Thomas & Lleras, 2009). A first step in designing such a platform is understanding fundamental challenges faced by students learning to code. To address this, we conducted Zoom interviews with eight high school and middle school computer science (CS) educators in San Diego county. We asked open-ended questions, prompting them to speak at length and give unprompted evaluations of tools, platforms, and approaches. We present the thematic analysis of these interviews in this abstract. Most participants had used block-based programming tools (namely, Scratch) and universally found them a powerful tool for promoting algorithmic thinking by abstracting syntax into an “intuitive” interface. However, two major drawbacks were raised. 1) As projects get large, the interface becomes cumbersome and collapsing chunks of code exacerbates rather than mitigates this problem. 2) High school students often perceive block-based programming as “childish” and “unprofessional,” causing them to avoid it. For some participants, rather than formally teaching core programming concepts (e.g. loops, conditionals), they would have students first search online forums and videos and then focus on projects that allow them to apply what they have learned. The rationale for this approach prioritized engagement through project-based learning and mastery of skills and tools necessary to solve coding challenges. In contrast, other teachers stated that project-intensive approaches can prevent students from developing a solid foundation in programming fundamentals. Surprisingly, none of our participants described another common programming metaphor: flow-based programming where nodes in a flow-diagram are connected by curves to show how the information/algorithm flows through time/space. These environments can often be found in professional tools (Unreal, Unity, Rhino/Grasshopper, Blender) for coding complex event-based behavior. Despite never mentioning these tools, a number of participants did use visual diagrams and flow charts as a valuable external means for understanding program flow. Using the relationship between these themes, we can identify the properties of a tool needed to aid CS education for students. The physical metaphors that come from block-based coding needs to pair with the capacity for complexity exhibited by text-based coding. Additional research needs to be conducted to evaluate the affordances of flow-based coding as a third modality. This needs to be packaged in an IDE enabling self-motivated, creative endeavors.

Area 3 - Learning/Teaching Methodologies and Assessment

Nr:	3
Title:	Do Motivation and Participation Predict Student Performance in Blended Learning?
Authors:	Margault Sacré, Marie-Christine Toczek, Florence Policard, Guillaume Serres, Catherine Paulet, Sabine Petit, Emmanuelle Neuville and Dominique Lafontaine
Abstract:	The COVID-19 pandemic has disrupted higher education. It is likely that blended and distance education systems will be used on a massive scale in the post-pandemic period. However, the governing bodies of universities and colleges are all asking the same question: Will this transition benefit students? In this study, we aim to assess the effect of motivation and participation on student success in a blended course. The program offered to the students is organized according to an alternation between distance learning and face-to-face sessions. The totality of the learning content is proposed in video clips accessible on the Moodle platform. Motivation is an essential variable because it has a strong influence on different academic variables such as persistence, commitment and success (Darnon & Butera, 2005; Dweck, 1986). In some blended courses, student engagement may be weakened compared to a traditional teaching context (Tuckman, 2007, cited by (Sun & Rueda, 2012)). However, student engagement is seen as a fundamental variable (Sun & Rueda, 2012). In blended or distance education, an easily observable aspect of student engagement can be measured through student participation, which is one of the indicator of student engagement (Baragash & Al-Samarraie, 2018; Cacciamani et al., 2012; Hrastinski, 2008). We measured motivation using two scales: academic self-concept (Dierendonck, 2008) and achievement goals (Darnon & Butera, 2005). The indicator of online participation corresponds to the number of logs on the platform. A multiple regression model including many control variables (age, gender, social status, educational and professional background, access to ICT, attitudes towards ICT, previous knowledge) was designed. The backward method was chosen in order to keep only the variables that contribute significantly to R2. The results show that two achievement goals and participation predict student performance. More specifically, we see a positive effect of performance-approach goals and mastery-avoidance goals. There was no effect of academic self-concept. The performance-approach goals correspond to the desire to be competent in relation to others and therefore refer to normative comparisons. This result would be a reflection of the higher education context in which they appear since competence can be defined in terms of relative ability and normative comparisons (Barron & Harackiewicz, 2003). In addition, student persistence is positively influenced by the adoption of performance-approach goals (Ferla et al., 2010). Mastery avoidance goals are goals adopted to avoid not mastering the task or regressing. It is possible that despite the insecurity of students who adopt these goals, they may remain focused on their learning objectives. Finally, this study shows that participation in online activities is a key consideration when designing blended courses.

Nr:	6
Title:	The Influence of Self-efficacy and Self-regulation on Students’ Use of Learning Technologies
Authors:	Karen Sutherland, Ginna Brock, Margarietha J. de Villiers Scheepers, Prudence. M. Millear, Sherelle Norman, Tim Strohfeldt, Terri Downer, Nicole Masters and Alison. L. Black
Abstract:	Blended Learning as a pedagogical approach has increased significantly during the COVID-19 Pandemic, with blended and online learning environments becoming the new digital norm for Higher Educational institutions around the world. While discussions of Blended Learning have featured in the literature for thirty years, a common approach has been to categorise learners to support educators in better understanding students’ approaches to learning technologies. This method, largely unsupported by empirical evidence, has largely failed to address the challenges of integrating learning technologies to fit with students’ preferences, achieve learning outcomes and contribute to student success. Focusing on student preferences, our study presents results from a pre-COVID survey of undergraduate students across four campuses of an Australian regional university where students shared their preferred Blended Learning approaches and their academic self-efficacy and self-regulated learning. Results identify that students want consistency, relevance, and effectiveness with the use of blended learning tools, with a preference for lecture recordings and video resources to support their learning, while email and Facebook Messenger were preferred for communicating with peers and academic staff. Our research suggests a quality Blended Learning environment supports self-regulated learning using fit-for-purpose technological applications. Academic self-efficacy increases when students perceive the educational technologies supplied by their institution are sufficient for their learning needs.

Nr:	10
Title:	Principal’s Training Results Study of an Online based Educational Change Management Program
Authors:	Germán Fromm
Abstract:	Given the importance of principalship for the overall development of an educational system, this article depicts the experience with principals in an educational change management training program. The case of the Educational Leadership Center of the Concepción University in Chile took the challenge to train 62 students in 2020 through instructional designs grounded in realistic problem-solving in a virtual simulation context. The simulation was validated through previous research and reflect how participants can show educational change management skill, as if they would strategize and apply them in real schools. Data collection methods were questionnaires of profiling variables and ongoing assessment of the participants during the simulation use. The descriptive research is of the results of this training experience in regard to learning, skill development and participants profiles. They show an effectively designed program that is able to produce immediate consequences for the participant’s skills. Also, young age, overall intelligence, strategic thinking and 'gaming' are associated with better performance, while sex, work experience, and leadership skills are not. Moreover, this sets an example of how literatures recommendations for improved modern instructional and assessment techniques can and should be implemented for the training programs and principals for their key position in the educational system.