Wednesday, March 2, 2016

5/100 The effect of contextualized conversational feedback in a complex open-ended learning environment

Segedy, J. R., Kinnebrew, J. S., & Biswas, G. (2013). The effect of contextualized conversational feedback in a complex open-ended learning environment. Educational Technology Research and Development, 61(1), 71-89.

Notes:
1. What is the theoretical foundation of the intervention design (why it should success)

In this article, the authors suggests two design guidelines for Open-End Learning Environments, which they think will benefit students' learning. But they don't offer any explicit theoretical reasons for the guideline, it is more like based on experiences and some general learning theory. This makes me think about "warrant". In our field, what are the researchers' common sense, by which I mean some learning or instruction principles that we don't need to write it explicitly as everyone knows.

It mentioned meta-cognitive strategies in the introduction part of the article, but didn't discuss it explicitly in the discussion part.

2. Measures 

This article use HMM and DSM to analyze students' learning behaviors in this OELEs. One measure for HMM is relevance: in my understanding based on their writing, it is a measure of the relationship between two adjoining actions. A higher score means this action is more informed by the previous action. It indicates "more systematic or focused approach to the learning task" (sounds like metacognition)

But they also wrote that both two analysis approach cannot match students' behaviors to the feedbacks. By which I think it means the variation of (high or low) relevance scores can not be account for the feedback. Also it is the pattern of a group of students not one specific students. I think maybe some qualitative analysis can help.

-----
Two guidelines for the design of feedback in OELEs to promote affect learning (p.72): Feedback should be contextualized by the student's task goal, learning artifacts and recent activities; Feedback should be delivered in a mix-initiative conversational format.

PA feedback is characterized by two main attributes. First, it is organized into prompts: short statements delivered as one-way communication. After an agent speaks, the learner has no opportunity to respond to the agent. Second, PA feedback is action-oriented; when students take an action in the system, agents delivering PA-feedback suggest potentially useful behaviors and strategies that are linked to that action.


Friday, February 26, 2016

6/100 Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis

Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning a systematic review and meta-analysis. Review of educational research, 0034654315582065.

Very interesting

Some notes

One results is that the aggregate measure of contextualization in the study has been reported as having a significant negative relationship with learning gains...

It seems this finding is conflict situating learning in context. My interpretation is it depends on what are the learning outcomes, higher order thinking skills or simple fact, math operation. However, the authors listed four possible reasons:

  1. "...rich narratives and visual complexity distract students from the intended learning content or provide alternative goals within the game...."
  2. "Almost all the studies analyzed in this report involved immediate posttests focusing on lower order learning outcomes."
  3. "...our own coding rules may not have captured the critical relationships between narratives and learning in terms of time for telling about lower order learning objectives. Specifically, we coded the relevance of narratives in terms of relevance to the learning mechanic rather than assessment content."
  4. "We coded narrative in terms of relevance and thickness, but perhaps the critical features of narratives are whether they are engaging, high-quality, or accessible, regardless of thickness or relevance." 
Their coding system
  • General Study Characteristics 
    • Play duration
    • Additional instruction
    • Player configuration: 
  • Game Mechanics Characteristics
    • Sophistication of Mechanics: points/badges; more than points/badges
    • variety of player actions
      • small: tetris
      • medium: zoombinis
      • large: Quest Atlantis
    • Intrinsic Integration (learning mechanics/game mechanics)
      • Simplistically intrinsic
      • intrinsic
      • not fully intrinsic 
    • scaffolding
      • Low : success/failure/points
      • less low: display answers after an error
      • less high: beyond two above
      • high: teacher
  • Visual and Narrative Game Characteristics (Contextualization)
    • Visual realism
      • schematic
      • cartoon-2D
      • realistic-3D
    • Anthropomorphism
      • low 
      • medium
      • high
    • Perspective
      • Third person
      • over the shoulder/overhead tracking
      • First person
    • Story relevance
      • none
      • irrelevant
      • relevant 
    • Story depth 
      • thin: only setting, scenery, context
      • medium: some evolving story
      • thick: rich evolving story
  • Research Characteristics in Value-added and Media Comparisons
    • comparison condition quality
    • condition reporting (game)
    • method reporting (inappropriate statistical analysis/omissions in reporting)
    • assessment overalignment
    • assessment type
    • study design
This coding scheme can be used to guide the writing of the method part of my study.

The reason they create "Contextualization" is because they found variables under visual and narrative are correlated.