Last Fall, in preparation for teaching ds106 this Spring, I blogged about a number of trends in my Spring 2011 course evaluations and how I wanted to work on my relatively poor scores when it comes to at least two elements of the course: “Clear Criteria for Grading” and “timely return of graded materials.” (We got course evaluations back much much quicker this year thanks to the course evaluation system moving online.) Turns out despite the best laid plans of mice and menaces, my numbers in both categories went down slightly. For example, “clear criteria for grading” was 3.95 out of 5 last year, and was 3.93 this year, which kinda pisses me off because I spent a lot more time framing expectations this time around. That said, grading in ds106 is very much part and parcel of one’s engagement in the work happening around the community, and despite how much I push this it always comes as a surprise to students at the end of the semester when it comes to grading.

The other area, “timely return of graded material” was an average of about 4.25 out of 5 last year across both sections, this year it was 4.13 for my larger, combined class. I am not as surprised about this because I pushed off our weekly meetings and didn’t send letter grades out after we met. I think part of my problem with both these categories is I am torn between totally abandoning the traditional notion of the letter grade but still making sure students have a clear sense of where they are throughout the semester—which is what I do three times a semester in individual conferences. This approach takes a ton of time, but I figure the more intimate, focused feedback on their progress and work is far more useful than a hard and fast grade. But I may be wrong on this count. I ride them regularly in their comments and through email from week 2 on so they don’t fail miserably out of the gate (which works very well, I highly recommend this—particularly for online students), the feedback is regular and sometimes brutal

As for the other categories for instructor evaluation, there were some my scores increased significantly others where it fell marginally. For example, this semester the class “met on schedule” was up .18 points from an average of about 4.62 to 4.8. Not sure what to say here, we met pretty regularly last year as well. Also, the “presented material in an organized fashion” was up about one-tenth of a point from an average of about 4.23 to 4.33—I think this has a lot to do with all of Alan Levine’s amazing framing of the assignments each week—I benefitted similarly from teaching alongside Martha the previous year as well.

I’m a little sad to report that my enthusiasm score went from an average of about 4.93 to 4.89—I wonder if this a case of a bit of ds106 ennui. This is most distressing for me, enthusiasm is all I got—it’s my life!

You can see all of the Instructor Evaluation details below:

The second part of the evaluation is the focus on the course experience. And I was pretty impressed with the results this time around because all six categories increased, and the separation of course experience from the instructor is promising for ds106. My larger take away from these scores—which I fully understand are subjective, unreliable, and potentially earth threatening— is the idea of the course and its usefulness transcending any one idea of an instructor, which is nice.

“I acquired substantial knowledge and/or skills in this course.” 4.57 –> 4.67
“I found instructor’s feedback useful”. 4.63 –> 4.85 (this is odd given how much I got tagged on feedback in my evaluation)
“I was encouraged to ask questions about the course material.” 4.79 –> 4.81
“I found the instructor to be helpful in clarifying difficult material.” 4.68 –> 4.85
“I was encouraged to reflect critically on course content.” 4.69 –> 4.88
“I found the instructor to be available outside of class for help (e.g., during office hours, special appointments, via e-mail, telephone).” 4.78 –> 4.84

Across the board the experience of the course ranked higher than last year, and it also had a 4.82 overall score which is significantly higher than the UMW average of 4.38.

The only thing I did radically differently this semester was to provide every student with the option to take the class face-to-face (f2f) or entirely online. Of the 34 who finished the class, 19 took it f2f and the remaining 15 took fully online (sometimes a few of these online students came to class for the fun of it). So close to a 60/40 cut when given the option to take it f2f or online. What was unfortunate about this go around, unlike last semester, is that I can’t distinguish the feedback from f2f and online students given they’re all in one section—last semester there was a f2f and a separate online section. That’s another point to consider. I had about 10 to 12 more registered students in a single session of ds106 than I usually do (last year I was lucky enough to get paid for two sections, but that was “corrected”) so I decided to overload to see how that would go, and in retrospect it wasn’t that much more work because I did a good job of keeping the onus on them to be part of the community, comment, interact, and basically forge their own relationships within and beyond the class. Placing the responsibility back on the students and following up with them on it periodically is rather effective for managing a very mild overload of students in my experience (don’t imagine it can or should scale much beyond a handful of students).

More and more I think the distinction between taking d106 online and f2f is arbitrary in terms of the value of the experience, and for me that is a radical development. ds106 is a web native class, it is born of and on the web making the f2f experience potentially interesting and reassuring, but at the same time vestigial. I like the f2f classroom, I like getting to know students their and having the physical space to congregate in. That said, based on my own experience I’m pretty certain it would be erroneous to privilege it in this regard.

Finally, here are some anonymous student quotes from the evaluations that were pretty fun:

It doesn’t get much more awesome than this one—attack the evaluators!:

Attempting to evaluate my teacher in this manner is flawed and unacceptable. To pretend that my teacher, Jim Groom, deserves to be evaluated according to a quantified list of predetermined measures would be an assault to his viability as an instructor, an advisor, and a mentor. Although you wrap up this ‘course evaluation’ with a pretty bow (telling me my input is important, that these evaluations matter, etc.), I absolutely refuse to reduce myself to participation in your quantitative course evaluations and ‘outcome assessments.’ Do you really believe I can summarize my teacher’s abilities with a series of “5s,” symbolizing his extraordinary capacity to inspire students to learn? I sincerely hope you do not.

And this one ain’t so bad either:

This has been the most work intensive and the most rewarding course I’ve taken at UMW. It is unlike anything else I’ve taken before, and I’ve gained so much valuable knowledge about Web2.0 and online presence, not to mention the increased confidence in my creative and technical abilities. I will use the skills I’ve learned in this class for many years to come, no matter what profession I end up working in.

The idea of the intensity of the work is a recurring theme in the evaluations (which makes me happy, get your monies worth folks):

Overall I really enjoyed this class. At times I thought the work load was a little bit much for a 100 level class, but Jim was very understanding of our busy schedules.

Where is written that all 100 level classes are less work and by default easier? This is a fallacy of some kind, right?

The work was tedious most weeks, but the hard work was well worth it. I learned so many useful skills in this class. Allowing us an extra week for video was truly helpful–I recommend doing that again.

This is a great class that overloads right near the end, starting with the video weeks. Seriously, those burnt me out hardcore. The radio thing was already a TON of work, then 30 stars of video, THEN 15 remix stars ALL WHILE I’m supposed to be doing final projects, in addition to my OTHER CLASSES and work and a social life? C’mon.

This course has been very enlightening in terms of things that can be accomplished through the web as a medium. However, the class is so fast-paced, if you don’t have the time for it in your schedule, it is advised to NOT take it. It requires so much time and energy through some parts of the semester and can be difficult to juggle the coursework amidst work for other, more crucial classes! I love the idea of the course though and it has definitely been an interesting experience!

And the money quote which is at the heart of ds106:

I LOVED this class. It really opened my mind to my potential creativity