Wednesday, March 2, 2016

5/100 The effect of contextualized conversational feedback in a complex open-ended learning environment

Segedy, J. R., Kinnebrew, J. S., & Biswas, G. (2013). The effect of contextualized conversational feedback in a complex open-ended learning environment. Educational Technology Research and Development, 61(1), 71-89.

Notes:
1. What is the theoretical foundation of the intervention design (why it should success)

In this article, the authors suggests two design guidelines for Open-End Learning Environments, which they think will benefit students' learning. But they don't offer any explicit theoretical reasons for the guideline, it is more like based on experiences and some general learning theory. This makes me think about "warrant". In our field, what are the researchers' common sense, by which I mean some learning or instruction principles that we don't need to write it explicitly as everyone knows.

It mentioned meta-cognitive strategies in the introduction part of the article, but didn't discuss it explicitly in the discussion part.

2. Measures 

This article use HMM and DSM to analyze students' learning behaviors in this OELEs. One measure for HMM is relevance: in my understanding based on their writing, it is a measure of the relationship between two adjoining actions. A higher score means this action is more informed by the previous action. It indicates "more systematic or focused approach to the learning task" (sounds like metacognition)

But they also wrote that both two analysis approach cannot match students' behaviors to the feedbacks. By which I think it means the variation of (high or low) relevance scores can not be account for the feedback. Also it is the pattern of a group of students not one specific students. I think maybe some qualitative analysis can help.

-----
Two guidelines for the design of feedback in OELEs to promote affect learning (p.72): Feedback should be contextualized by the student's task goal, learning artifacts and recent activities; Feedback should be delivered in a mix-initiative conversational format.

PA feedback is characterized by two main attributes. First, it is organized into prompts: short statements delivered as one-way communication. After an agent speaks, the learner has no opportunity to respond to the agent. Second, PA feedback is action-oriented; when students take an action in the system, agents delivering PA-feedback suggest potentially useful behaviors and strategies that are linked to that action.


Friday, February 26, 2016

6/100 Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis

Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning a systematic review and meta-analysis. Review of educational research, 0034654315582065.

Very interesting

Some notes

One results is that the aggregate measure of contextualization in the study has been reported as having a significant negative relationship with learning gains...

It seems this finding is conflict situating learning in context. My interpretation is it depends on what are the learning outcomes, higher order thinking skills or simple fact, math operation. However, the authors listed four possible reasons:

  1. "...rich narratives and visual complexity distract students from the intended learning content or provide alternative goals within the game...."
  2. "Almost all the studies analyzed in this report involved immediate posttests focusing on lower order learning outcomes."
  3. "...our own coding rules may not have captured the critical relationships between narratives and learning in terms of time for telling about lower order learning objectives. Specifically, we coded the relevance of narratives in terms of relevance to the learning mechanic rather than assessment content."
  4. "We coded narrative in terms of relevance and thickness, but perhaps the critical features of narratives are whether they are engaging, high-quality, or accessible, regardless of thickness or relevance." 
Their coding system
  • General Study Characteristics 
    • Play duration
    • Additional instruction
    • Player configuration: 
  • Game Mechanics Characteristics
    • Sophistication of Mechanics: points/badges; more than points/badges
    • variety of player actions
      • small: tetris
      • medium: zoombinis
      • large: Quest Atlantis
    • Intrinsic Integration (learning mechanics/game mechanics)
      • Simplistically intrinsic
      • intrinsic
      • not fully intrinsic 
    • scaffolding
      • Low : success/failure/points
      • less low: display answers after an error
      • less high: beyond two above
      • high: teacher
  • Visual and Narrative Game Characteristics (Contextualization)
    • Visual realism
      • schematic
      • cartoon-2D
      • realistic-3D
    • Anthropomorphism
      • low 
      • medium
      • high
    • Perspective
      • Third person
      • over the shoulder/overhead tracking
      • First person
    • Story relevance
      • none
      • irrelevant
      • relevant 
    • Story depth 
      • thin: only setting, scenery, context
      • medium: some evolving story
      • thick: rich evolving story
  • Research Characteristics in Value-added and Media Comparisons
    • comparison condition quality
    • condition reporting (game)
    • method reporting (inappropriate statistical analysis/omissions in reporting)
    • assessment overalignment
    • assessment type
    • study design
This coding scheme can be used to guide the writing of the method part of my study.

The reason they create "Contextualization" is because they found variables under visual and narrative are correlated.

Wednesday, September 23, 2015

MOOC and Gamification

It seems to be a soft advertisment for Brain Chase, but I do see some interesting and meaningful Gamification elements there. But, just like what the comment said, it may be not fair to compare it with free course website.

The article

The website

Tuesday, April 7, 2015

Methodology

It comes from a podcast I listened to days before. The professor mentioned the term, "methodology of history". Actually, the questions come before that but the feeling of having the questions are not strong and clear. What is the methodology of education? Almost all of them come from social science. Are there a unique education research methodology?
This semester I took 641, one textbook is applied research design, the first chapter discussed the differences between basic research and applied research. Then I have the question, is any basic research within education research? Is there universal knowledge in education? I admit that knowledge has its own context like Newton's laws. There might not be universal knowledge in any field forever. However, it seems in education it more leaned to a solution to a specific question. It is possible that I have this feeling is because the field I am in -- instructional design. The two years training teaches me how to identify problems.
Chapter one also discussed differences in methods, this lead me to think about the question on methodology of education. It also brings me back to the discussion of whether education is science. I am taking a quantitative course in ESF now, everytime when I write the assignment it reminds me the differences between nature science and social science.
It is the first time that I feel I need to read some papers or books about education methodology so I can be more clear about my study.
It seems I am studying humans' behavior/performance, but I want to know their thoughts, "the interpsychological plane and the intrapsychological plane". Have no ideas how to do that. "Any method always goes with a theory", "Any method of research is a way to investigate some particular domain."
I can not state my problems clearly currently.      

Wednesday, January 14, 2015

Bias

During the winter break, I went to the San Francisco Exploratorium again. Last time I went there is tow years ago. Now it moves to Pier 15, larger than the one in Palace of Fine Arts.

I felt there are more games on social science now, and I spent most of my time for those social science games with Ms. Jin.

One amazing founding is my bias or my subjectivity. When I took 603, I thought I didn't have any and had a hard time to write to the reflection  journal about subjectivity. If I came to the Exploratorium first, I may have things to write. ;)

That game is really easy to set up. They have two sets of cards with some nouns on it. In one side of the table,the words "female" and "family" are in the same rectangle on the table, "male" and "career" are in the same rectangle. In the other side, "female" and "career" are together and "male" and "family" are together. Two players are needed to play the game. They choose a side to sit at the table. Shuffle the cards first then put each cards into the two rectangle according to the words on the card. When finished, change side.

The first round I sit at the female family side, I category the cards really quick. But when I chage the side with Ms. Jin, my speed is slow, I want to use the word "significantly" to describe the difference, even I didn't run a t-test.

I never know that before.

And today, one of my friend write she met a hot guy who drinks strawberry milk everyday. and I thought is that guy a gay? Then I found my subjectivity again.


Friday, November 22, 2013

Game as ground experience

Last two weeks, I introduced two researches on integrating games into classroom setting. Today, I will write how games benefit learning outside the classroom.

As we all know, games could offer gamers rich experience, no matter it is role-play games or puzzle games. Some researchers believe that combining gaming experience with formal learning experience it will provide depth in learning.

Playing Civilization simulation game then becoming an expert of this game doesn't increase one's history knowledge, however, research shows that playing and struggling with the game prepared students to learn more from reading a difficult college history text chapter.

In my opinion, when I play a game and attracted by its storyline, I will be more engaged in then reading a textbook, because the game offeres me immediate feedback after any operation I did. Sometimes, I will have the same feeling when I read a textbook. I raised a question from the text, and wanted to know the answer. If I can find the answer in the following part of this book, I would be satisfied and keep reading it. If not, I might stop to go somewhere else to find this answer. In the first situation, I feel I interacted with the book. In the game world, the feeling is stronger, as I can see how I effect the game world, at the same time, it force me to make decision, to think or guess.

According to some researchers, these game play experience will benefit one's future learning. Therefore, the best use of video games in learning might be in providing experience with the subject matter that is to be later acquired via more formal learning settings.



Friday, November 15, 2013

ASTRA EAGLE -- a drill and practice game

Today, I want to introduce a case study about a drill and practice game.

A drill and practice game is one type of puzzle game. As what its name tells us, the purpose of a drill and practice game is to support drill and practice. Different from simulation games I wrote about in last several weeks, drill and practice games are not thought as an effective tool to improving learning and skills by most researchers. The case study I introduce today takes a close look at drill and practice game and want to find out how a drill and practice works, what characteristics they embody, and what learners are doing as they play a game, thus informing whether and how drill and practice games would be a pedagogically sound learning environment.

ASTRA EAGLE[1] was been used in this study, which comprises a serious of web-based online game and developed by the Center for Advanced Technologies of a school district. These games are designed to reinforce academic standards for mathematics required by Pennsylvania System of School Assessment.

The research site is an elementary school in the same district. Participants are 15 4th-5th grade students who were enrolled in a summer program, where this study will be conducted. All the participants played eight ASTRA EAGLE math games during 10 two-hour sessions for five weeks.

The researcher found these drill and practice games enhance students' positive attitudes toward math learning significantly. However, what should be noted is that not every game would engage students' learning. The data shows some good learning game design principles, such as "situating learning activities within the game story and characters that players will take on, making games pleasantly challenging, and scaffolding reflections". Also, the off-computer activities are important, such as offline assistive learning tools, game-based collaborative activities, and the just-in-time guidance of an instructor.

Ke, F. (2008). A case study of computer gaming for math: Engaged learning from gameplay?. Computers & Education51(4), 1609-1620.

[1] You don't need an ID and password to log in, just click the login button in the login page, the website will assign you an account.