Game-based assessment

I’ve been thinking a lot lately about badges in learning as a result of my using Foursquare to chart some of the places I visit, and share what I do and don’t like about these places. I’m not as motivated by the badges on this particular platform as I’m really interested in the content. I can see, however, how using badges can help encourage users to check in more often and in that moment add more information to the service, increasing the service’s value to people like me. Win-win.

There are other apps that do this as well, while some apps (like SCVNGR) have challenges instead, but a similar idea of gamifying social interactions. In fact, universities are using apps like SCVNGR to add a gameplay element to their campus tours and orientation week walkabouts. In SCVNGR, users can do single challenges at one location, or as in the campus tour, they can be strung together in a “trek” that happens over a series of locations, possibly over an extended period of time (scvngr.com). The challenges also may be linked to real-world rewards like freebies or discounts.

Then there’s OpenStudy, which has gamified elements of academia, and users get badges for activities such as helping others with their homework (Young, 2012). Preetha Ram of Emory University started the company in an effort to give a better idea of what soft skills students are learning while studying. She says, “We all know that teaching someone is the best way to deepen your understanding of the concept” (as cited in Young, 2012). This begs the question, however, of how to document these achievements when they occur in the analog world.

The value of this gamification of learning seems to come in the incremental feedback and sense of achievement vs grading what hasn’t been mastered as in traditional grading (Gerstein, J., 2012). Khan Academy’s Sal Khan (2010) focuses in on this difference between positive and negative feedback in his editorial “YouTube U. Beats YouSnooze U.” in The Chronicle of Higher Education. He says:

Perhaps the worst artifact of this system is that most students end up mastering nothing. What is the 5 percent that even the A student, with a 95 percent, doesn’t know? The question becomes scarier when considering the B or C student. How can they even hope to understand 100 percent of a more advanced class? Is there any point in studying differential equations if you don’t have an intuitive understanding of basic calculus? Is there any point to taking biochemistry when you have less than perfect understanding of first-year biology and chemistry? (¶ 5).

In a pointed summary of his argument, Khan states, “The current system does not allow for addressing [comprehension] gaps, so both professor and students settle for superficial coverage of the material. Students don’t retain anything because they didn’t intuitively understand it to begin with” (¶ 7).

Education and games might have more in common than at first sight. James Gee relays his experiences of trying to learn how to play a game from the manual, saying that he couldn’t understand the game from the rulebook. But, after playing (poorly) a couple of time, and then coming back to the manual after attempting the game he had a better idea of what the contents, in particular the new vocabulary, were referring to. He compares textbooks to the game manual:

Now if you’d played the game, what you’d do with the manual is use it as a reference to look up stuff that you need to know to get better or to understand something in the game that you don’t think you fully understand. And that’s the same way a textbook ought to be used. You ought to be using your chemistry textbook when you’ve already understood there’s something you need to know about chemistry, and you go use it, then, proactively. (3:19-3:39)

He argues that games and education are actually quite similar, and how they’re different:

All a video game is is problem solving. It’s just a series- and if you think about it, in some weird way, a video game is just an assessment. All you do is get assessed every moment as you try to solve a problem, and if you don’t solve it, the game says, “You fail; try again.” and then you solve it, and then you have a boss, which is a test, and you pass the test. I mean, games essentially are a form of assessment. The thing that is probably the most painful, ludicrous part of schooling, but in a game it’s fun, right, because it’s handled in a very different way. One thing games don’t really do is separate learning and assessment. They don’t say, “Learn some stuff, and then later we’ll take a test.” They’re giving you feedback all the time about the learning curve that you’re on. (1:14-1:53)

 

Application to the Classroom

Judy Willis (2011) recognizes that the differentiation required to create “individualized instruction, assignments, and feedback, that allow students to consistently work at their individualized achievable challenge levels, are time-consuming processes not possible for teachers to consistently provide all students” (¶ 16), but this does not mean that the idea cannot be applied to the classroom. Rather, she argues, “keep achievable challenge and incremental progress feedback in mind when planning units of instruction” (¶ 16). To do this she recommends shifting the responsibility for monitoring progress to students themselves via independent progress recording. Using charts or graphs, students can tangibly see their own incremental progress toward a larger goal. Individual consultation and goal-setting can set the parameters for these increments.

Badges could potentially be used here, where students have set targets on their way to the larger goal. At each target, students receive a badge demonstrating achievement and giving a waypoint, much as how in a game when a player loses a “life” they don’t start back at the beginning of a level, but at the checkpoint closest to the most recent failure. This also creates a reference point for future teachers/educators working with the student to understand exactly what they do and do not know and to what degree. In combination with Problem-based learning, this could allow for more confidence in the legitimacy of the certification/badge the student has received.

When determining how to set up these increments and levels of achievement, educators can learn from the gaming world. Ingrid Lunden at Techcrunch reports on how app-makers will fail without proper planning and understanding of game psychology; mere gamification is insufficient. It all comes down to meaningful motivations and objectives (Lunden, 2012). Lunden also warns that the gamification of apps is reaching its peak in the Gartner hype cycle and interest in the style may soon wane (Lunden).

To make it work, game designer Tadgh Kelly (2012) offers some insights on what works:

  • Validation: the feedback from others that what the user has done was valuable, which gets construed as personal value.
  • Completion: progress bars and a clear set of requirements for completion help users reach their goals and feel good about it. Legitimate goals only, though, not backdoor ways to get the user (or student) to do your work for you.
  • Prizes: Extrinsic reinforcement. Caution: removal of prizes can backfire, so be committed or stay away.
When looking at how this may work, in coordination with PBL there is space for success. Validation could come from students in the feedback and presentation stages, Completion is realized in finding the answer to the problem. Prizes do no seem to be necessary as the model for motivation in PBL is intrinsic. Stages in the PBL process could be marked by badges, or another appropriate, meaningful objectives and outcomes.
To do:
Look at one or two outcomes from my courses and identify how these could be put into a PBL lesson.
  • How can the outcomes be badge-ified? 
  • What are the checkpoints at each stage? 
  • Why should these stages and outcomes be meaningful for my students? 
  • Run an experiment and see how it goes. Report back.
Ah, my own little project. Perfect.
Further reading:

 

References

Gee, J. (2010, April 9) James Paul Gee on Grading with Games. Edutopia.org. Youtube interview. Retrieved from http://www.youtube.com/watch?v=l-EbfteDBtg#!

Gerstein, J., (2012, May 12). Assessment as a means for developing a sense of acheivement. User Generated Education. Retrieved from http://usergeneratededucation.wordpress.com/2012/05/12/assessment-as-a-means-for-developing-a-sense-of-achievement/

Kelly, T. (2012, November 17). Everything you’ll need to know about gamification. Techcrunch.com Retrieved from http://techcrunch.com/2012/11/17/everything-youll-ever-need-to-know-about-gamification/

Lunden; I., (2012, November 27). Badges Beware: 80% Of Gamification Apps Will End Up Being Losers, Says Gartner. Techcrunch.com. Retreived from http://techcrunch.com/2012/11/27/badges-beware-80-of-gamification-apps-will-end-up-being-losers-says-gartner/

Willis, J., (2011, April 14). A Neurologist Makes the Case for the Video Game Model as a Learning Tool. Edutopia.org. Retrieved from http://www.edutopia.org/blog/video-games-learning-student-engagement-judy-willis

Young, J., (2012, January 8). ‘Badges’ Earned Online Pose Challenge to Traditional College Diplomas. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/Badges-Earned-Online-Pose/130241/ 

 

5 Essential Questions to Ask Before You Innovate in Your School | edSocialMedia

5 Essential Questions to Ask Before You Innovate in Your School | edSocialMedia.

Via Jackie Gerstein’s Twitter feed.

I often complain about how I see technology (and by that I mean largely computers and the Internet) being implemented in education. What waste. What effort. What….disappointing results.

It’s not all bad, however, and before implementing technology, or at any evaluation stage, you want to be asking these questions of your structures, motivations, and desired outcomes.

Still not enough time.

As I think about the semester, I am noticing that even though we are only doing half the book in the same amount of time, I’m feeling like we just aren’t covering as much content as we should. Truthfully, this tells me a couple things. First: the students are finding the content a challenge. Okay, that’s fine. But is he content a good match for them? Is it meeting them at the level they are at? How could I be scaffolding this better?

Today my grammar lesson fell flat. Bombed. Okay, part if the problem for the student who didn’t understand was not paying attention, but part of that is on me, too, to keep it interesting. I hate how grammar is taught in EFL texts. It’s taught to students as though they are teachers; a linguistics approach of form, function and irregularities rather than patterns with meaning. It’s too much to remember when it comes time to use it, let alone really understand it.

In addition, the listening activities are really quite good; not stilted an forced into obvious grammar formats as in books I’ve used before. I think I’m goin to try to spend more time working with these rather than hammering in grammar. The ideas I’ve picked up from Dogme might help here, but I’m going to need to review. I’m expecting dictagloss and other similar activities might help.

Do you teach EFL/ESL? How do you make the most of your listening activities? How do you teach grammar? What suggestions do you have?

What do we desire of our students?

One of my constant complaints as a teacher is that I feel I’m assigned to teach material that is irrelevant. It’s hard to make that interesting. In response to that, I suppose, I’ve been exploring the Dogme philosophy/pedagogy that has become rather en vogue in certain ESL circles. The idea behind Dogme, or unplugged learning, is to get away from a pre-determined syllabus, frequently inappropriate in level or content of interest to students, and instead draw on the students’ immediate needs, circumstances and interests to drive the content of the course.

In essence, the problem I face in my current situation boils down to this:

If there is one student attitude that most all faculty bemoan, it is instrumentalism. This is the view that you go to college to get a degree to get a job to make money to be happy. Similarly, you take this course to meet this requirement, and you do coursework and read the material to pass the course to graduate to get the degree. Everything is a means to an end. Nothing is an end in itself. There is no higher purpose.

When we tell students to study for the exam or, more to the point, to study so that they can do well on the exam, we powerfully reinforce that way of thinking. While faculty consistently complain about instrumentalism, our behavior and the entire system encourages and facilitates it. (Chronicle of Higher Education)

So I wonder what I can do. I’m in a situation where I’m required to cover a textbook, my students are part of a cohort that takes unified exams, and no one seems to be interested in anything other than going through the motions of education, although I feel very little is actually accomplished that has any value at all to the students.

Please leave any advice or comments you have below.

Breaking the Silence Barrier

The Problem

Any teacher working with reticent ESL students may be facing a culture barrier, or simple shyness in the face of others. The fear of making public mistakes is crippling for many (Cohen & Norst, 1989; Tsui, 1996), and even just being the first person to talk in a silent room can be awkward, regardless of whether one is in a language class or not (Newman, 1982).

A Solution

The Portable Apps homepage

One way I try to alleviate this anxiety is to play music in the background of the class. It serves two purposes: to eliminate that “silence” that is so hard to break, and to subliminally elevate and maintain student energy levels.

While you can spend hours developing playlists for each class to set a certain mood, it may be just as easy to use streaming Internet radio to set the pace. With a USB drive and an Internet connection it’s easy to accomplish, as long as you’re working on a Windows machine (Mac solution below).

The Method – Windows

Hop over to Portable Apps and download their portable OS for your USB drive. Once that’s installed, you can chose from a variety of portable applications, including Screamer Radio (direct download). From there you’ll have a set playlist of stations to choose from, or you can select your own from the many, many streaming radio stations around the world.

The Method – Mac

iTunes Radio. Select a genre (see next section). Go.

The Music

I particularly like the SwissGrove.ch station for the background music of my classes (in iTunes under “jazz”). The music on this station has a steady beat without too many fluctuations. It has music both with and without lyrics, so as long as the volume is low it shouldn’t distract your student’s ears. Another good style is Bop or Hard bop in the jazz category, for the same reasons as listed above, and it’s almost certain not to have a singer.

I always mute the sound when we start a listening activity to help reduce distractions. When it comes time for students to do some speaking practice, I turn the volume up a bit and let them go. It’s never too loud for me to hear and observe my students’ practice as I circulate around the room.

Conclusion

When I use background music, I see a marked improvement in student willingness to participate and practice. They are generally more alert and active as well. I find the station selection to be of particular importance on rainy or overcast days when my students tend to be even more sluggish than usual. Music can be a great pick-me-up for both students and teachers, and I highly recommend you consider using it in your classes.

References

Cohen, Y., & Norst, M. J. (1989). Fear, Dependence and Loss of Self-Esteem: Affective Barriers in Second Language Learning Among Adults. RELC Journal, 20(2), 61–77. doi:10.1177/003368828902000206

Newman, H. M. (1982). The sounds of silence in communication encounters. Communication Quarterly. Retrieved September 14, 2012, from http://www.tandfonline.com/doi/abs/10.1080/01463378209369441

Tsui, A. B. M. (1996). Reticence and anxiety in second language learning. In K. M. Bailey & D. Nunan (Eds.), Voices from the Language Classroom: Qualitative Research in Second Language Education (pp. 145–167). Cambridge University Press.

 

New Section: Course Notes

I’m trying to work through my courses in a more useful format. While this blog isn’t perfect, I don’t exactly have the time to work on learning a new format. If I did, I’d be going with this utterly incredible workflow:

So instead I’m going to have to settle for at least linking around here to articles and things I’m working through. I aspire to the above efficiency. Someday.

Reflections on Semester 1: No Limits.

After a semester as a student again, I’m more aware of some of the problems in my own classes, some of which is my own making. In my next few posts I’m going to focus on issues that were particularly frustrating and consider how I would have approached them, and also how this applies to my own teaching. The first post is below.

Limitations

One of the most frustrating things I encountered this semester was limitations. In one of my classes, I was limited in which journals I could use for my research assignments, and as a result, doing research became very difficult. While 30-50 journals to choose from may seem adequate on the surface, other limitations in the titles of the articles, content, or focus of the journals themselves made the projects near impossible to do in a timely manner. If I had access to journals in their paper forms, skimming for main ideas, themes, etc, would be a lot easier- pick up the journal and look at the cover. When working with journals that are a mix of on and offline, and some that haven’t updated since 2007, it’s hard to “browse” for themes as one could at a library. To visually “browse” each journal via Google Scholar is likewise a daunting task.

While the intent of the exercise, as far as I could surmise, was to familiarize us grad students with the current issues in what the professor determined were the major publications in the field (which I should hope she is a capable of determining). Okay, this seems like a reasonable goal. I’m not sure, however, that throwing a list of about 4 dozen titles at the students and telling them to determine the themes AND write a paper about it in under 25 hours is a reasonable request. And I have access to some of the fastest internet anywhere, when I’m home. *sigh*

Given the educational goal, how would I have run this exercise? I may have required 50-60% of the research to be pulled from the reading list instead of 100%. It seems to me that there is much crossover in discipline and much can be gained from drawing on sources parallel to education and technology. In any normal literature review, a scholar would pull from all appropriate, peer-reviewed sources, books, etc, in an effort to give strength to her position.

Another aspect may have been giving a list of recent trends and having students make the connections between the trend and how technology is being applied, or seeking consistency in the technology used to create or support this trend. Another possibility would be a list of old trends and have students examine what has supplanted it and how.

Finally, I would have provided students with explicit access to examples of good, better and great work with explanations as to how each example met the criteria I was expecting, especially if I were going to be a stickler for format on equal grounds to content.

Applying it to My Classes
Students new to a content area can find a lack of limits paralyzing. This is particularly true if they don’t have a clear vision of the goal and requirements of the project. Limiting my students to a single method and content stymies creativity and only assists my grading ease. Giving student freedom in either what they present or how they present it (keeping the course goal constant) allows freedom for variation in interest or ability. When both the content and the method present challenges, but only one is relevant to the course, either teacher, student, or both end up missing the educational goal of the activity.

An example of this is most certainly the mandatory online homework that I had to assign to my students last semester. The vast majority didn’t even register, let alone do the assignments. As a result, most students, and certainly the students most in need of it, missed out on the practice that could have helped them on their assessments.

The other issue is assessment. My program doesn’t have measurable outcomes, and as such it is impossible to identify whether or not students succeeded. Having measurable goals and keeping these in mind make assessment easy. Without them, assessment is subjective at best and more likely impossible.

Your Take
How have limitations served or hampered your students’ learning? How can you change how you limit your students or what you limit in your classes to help your students reach their educational goals?

Is there anything I missed? Do I have any wrong assumptions or reactions? What do you think I could be doing better?

The importance of automated backups

Why any professional outfit running a CMS for teachers does not have an automated backup for data recovery is beyond me.

Also, teachers should be forced to set up a backup email where they an be sent the data from the grade book, etc, automatically on a daily basis.

We should be past the days of asking, “Did you make a backup?”

Website Registration Processes

Site registration is something, it seems, many users just want to do as quickly as possible so that they can get on with doing what is actually important to them. Myself included. I’ve noticed this previously with my students, and when I was using Moodle, I tried to cut out all the unnecessary registration information to both expedite and simplify the process.

This year, with the online course book that we’re using at my university, students have a multi-step registration process, which is, in my opinion, setting students up for failure. Parts of the registration are available in their L1, but not until after some critical decisions have been made. Additionally, it is not obvious how to change the language settings (and I didn’t catch it until I walked several students through the process).

In fact, many of my students have indeed selected the wrong product (the 1st edition of a textbook rather than the 2nd, as the title on the text for the first edition is the same as we’re using in class, whereas the edition we’re using in class shows a higher-level textbook.), and because they’re both unlikely to read the English instructions and unable to, they continue forward, blindly, just doing what needs to get done so that they can get to their online homework assignments.

In my own classes, there was a rather convoluted process for finding all the information I needed for my program, and a couple of steps with special instructions that I missed, because, like my students, I wanted to get to the goal and I missed the process.

My take home: People are going to ignore registration instructions, look for the red asterisks that indicate a mandatory field, and get through the process as quickly as possible.  It’s the same reason people click the “I Agree” box on EULAs. We assume familiarity indicates sameness, and we assume we don’t need to think about the process. Fix: call out special instructions or information separately, bringing awareness to it. Make it un-ignorable.