Assessment & Grading & MGMT

Documentation Of Participation vs. Evidence Of Learning

/ magisterp / Leave a comment

I came across a 1993 article on student self-reporting (Darrow, et al.), and spent some time thinking about the idea that became the title of this blog post. As I’ve begun diving deeper into the “ungrading/gradeless” sphere of self-assessment, self-grading, and portfolios, I can say that at first I pretty much was getting the former documentation of participation, not the latter evidence of learning. Earlier this year, my student teacher and I spotted students uploading some questionable “learning evidence” into their portfolio, like notebook pictures with the day’s greeting copied from the board during the first five minutes of class.

This is not evidence of learning.

I’d go as far to say it’s a stretch to even call this something like participating. Copying is the absolute lowest writing skill for first year high school language learners, and this 5-minute routine merely sets up actual participation once class really begins. So, that was obviously documentation of some kind (vs. evidence of learning), and we then steered students towards a more productive direction of getting us evidence of learning. However, not everything students uploaded was as obvious. Take, for example, a Read & Summarize statement. Yes, the student was doing something in class, but was that necessarily doing anything for learning? It’s certainly possible, but just as likely not. The point here is that the difference between documentation of participation and evidence of learning really depends on the quality of what students add to their portfolio. If we just treat it as completion, that’s basically what we’ll continue to get: documentation of participation, which can actually lead to disengagement and lack of participation. As much as school can be school, kids really do find meaningless work worthless, and tend to find meaningful learning valuable. Even the cool kids. It’s important in a portfolio system to provide feedback on what students add so that you ensure meaningful learning occurs.

Easier said than done, but it’s time well spent.

As far as I can tell, there are only two ways to determine if what students add to their portfolio is, indeed, evidence of learning (and not documentation of participation). The first is an objective comparison to previous work, whether that’s on the teacher or the student, and the second is an honest rationale from the student’s end (explaining why what was added shows learning). I find the former tricky in a language class. For example, if you were to use the same text and have students keep submitting assignments based on that throughout the grading term, how sure are you that students are even processing the language anymore (vs. based on memorized English understanding of the text)? One cumbersome way could be to use a core set of vocabulary at the start of the term, and then write different texts with that same core set throughout the grading term that students interact with and complete assignments for. That might do the trick, but even then you’ve got to look at the students who ace the assignments in the beginning. How could they possibly show learning if they’ve already…learned…all that from the start? Also, a picture of a Quick Quiz result or something might just be participation, even if the student is showing you they understood all the Latin. Understanding Latin for 10 minutes during one class isn’t necessarily evidence of learning. Again, you’d need to compare those results over time to make the claim.

So, the comparison to previous work is tricky if not just time-consuming. That’s why I prefer getting students to write some honest rationales explaining why what was added shows learning. It’s all going to be individual anyway. Might as well embrace that.

Quizzing For Learning vs. Quizzing To Get A Grade

/ magisterp / Leave a comment

I was talking to a colleague about an assessment idea I had. The scenario began “if I were a math teacher…,” but really, this idea applies to anyone who gives quizzes. Many teachers I observe who assess like this usually hang out at their desk while students take the quiz. Sometimes it’s timed. Sometimes there are “after the quiz…” instructions on the board. In the literature, this is called an obtrusive assessment, with class on pause, sometimes the entire time.

So, if I were to ever assess like that, instead of hanging out at my desk, I’d start circulating the room, stopping at each student to point out a quiz item they should review (e.g., “Ja’den, spend more time on #3”). And I’d do this the entire time, just walking around, essentially doing all the correcting I would’ve done during my planning period, and even providing some feedback. It’s kind of like a more involved individualized Monitor Assessment. My colleague was wondering how this “real-time rolling assessment” would really show what students know and can do. We talked a bit. Questions were asked like “with so much scaffolding, how do we know the student can do anything on their own?” The truth is, they might not, but how is that any different? In fact, during that whole discussion I forgot to consider what the “real-time rolling assessment” was being compared to. That is, how is a give quiz/collect/correct/hand back procedure any different, really, for finding out what a student knows or can do?

It comes down to process.

Continue reading →

Punished By Rewards & Advantages To The Single-Point Rubric

/ magisterp / Leave a comment

As if researching how to eliminate grading and reduce assessment couldn’t get much better, I’ve now got something else. Alfie Kohn’s 1993 masterpiece really ought to be required reading for every educator. Coming up on its 30 year anniversary, the author at the time reviewed studies dating just as far back to the 1960s. This post is gonna focus on self-assessments. When it comes to students self-assessing, evidence suggests that the more students think about HOW WELL they’re doing (vs. WHAT they’re doing), they do it poorly.

That’s crazy-unintuitive, right?!

Continue reading →

Comprehension Establishers & Question Types/Possibilities

/ magisterp / Leave a comment

I end up learning at least one thing each year from my student teachers, whether it’s some insight while observing, some reflection when we’re planning, or some new activity or strategy they suggest. Here’s a revelation worth looking into…

When scripting out some questions back in October, one example I gave was asking “class, which word means ‘again?’ Is it aliquid or iterum?” After a few more like this, my student teacher said “oh, it’s kind of like a comprehension check acting as a comprehension…establisher.” I paused for a moment, then realized yes, that’s exactly what that is. She put a name to what I’ve been doing for years, going way back to the 2016 sneaky quizzes when I’d use the T/F statements to establish meaning of words.

Comprehension Establishers establish meaning in the form of a question.

The difference in purpose between comprehension checks and establishers is subtle. Establishers aren’t intended to evaluate student understanding. They’re asked in a way that all but guarantees students make a form-meaning connection (e.g., “What word means ‘obscure,’ nocte or obscūra?”). A comprehension check, however, is often exactly that: to check whether a student understands, and if they don’t, then we establish meaning right away. In that sense, can an establisher bypass the check and then establishing meaning? Absolutely, but then there’s variety to consider. Might as well get some experience with both.

Question Types/Possibilities
Also discovered when scripting out some questions, it became clear to me that there are often too many possibilities. Instead of brainstorming every possible one, it’s probably more beneficial to settle on a couple question types and cycle through them while reading. For example, using one sentence, Mārcus ōrdinārius esse nōn vult, we could ask each of the following:

Contrary-To-Fact Personalized Q: vellēsne esse ōrdinārius?
Comprehension Establisher Q: Which word means “to be,” esse or vult?
Comprehension Checks: What does esse mean?
Content Q: What does Marcus not want?

But should we ask that many questions for one sentence? If so, should we ask all four questions for EVERY sentence in the chapter? I’m thinking “no,” and “no.” While on the one hand it would appear to provide the student with a great deal of support, on the other hand this process would drag out quite a bit. My recommendation would be to ask just ONE of those question types PER sentence and see how it feels. You might find that even one of those questions per sentence ends up being too many while reading. If so, scale it back to a question per section of two-three sentences, and then just cycle through the four question types. For example, if a short chapter has eight sections of sentences, you’ll ask a comprehension establisher q, a comprehension check, a contrary-to-fact personalized q, a content q, and then repeat. My advice is to identify the contrary-to-fact personalized q’s first, since it doesn’t always make sense to ask those. Then, fill in the rest. Print these out, and stick them in the book you’re reading. Remember, unused scripts already served a purpose: to get you thinking of how and what to ask students.

Start Here

/ magisterp / Leave a comment

The most useful professional development (PD) I’ve had over these past 10 years in education has been from presentations, workshops, and blogs that have given me a “start here.” It’s usually in the form of someone figuring out a really effective way to do something, then putting it into some kind of ready-to-go format, whether that’s a packaged method, or list of steps. The “start here” works because it’s the culmination of trial, error, and revision. The “start here” works because it represents the essential. When I’ve used someone else’s “start here,” it’s been really effective. Naturally, there’s adaptation and I’ve been able to put my own spin on things, but only after I’ve implemented whatever was presented to me. So what’s the problem?

Some teachers begin to change the “start here” right away.

For example, if I share a cocktail recipe with you called “The Lance Drink,” and upon seeing .25oz Sfumato in the ingredients you decide to just leave it out, you haven’t actually made The Lance Drink. You’ve certainly made a cocktail. It’s close, but something else. You’ve mixed together ingredients of which the outcome is unknown…and there’s a good chance it might not turn out very good. Let’s say you love vodka. It’s in every cocktail you make, no matter what. When I give you my recipe, you sub vodka for The Lance Drink’s rye base. Why? That’s what you’ve always used. It’s what you’ve always done. So you mix…you sip…but you immediately spit it out because vodka is a horrible combination with the other ingredients. You might even say “gee, this Lance Drink isn’t so great.”

Teaching is a bit like that.

Instead of going with something tried and true, teachers tend to hold onto stuff that just doesn’t mix, not giving the “start here” a real chance. Sometimes, they might go as far as to claim that the “start here” doesn’t work (or whatever), mischaracterizing whatever was presented to them. In the worst of cases, other teachers that never got the original “start here” just listen to the ones who changed something right away, and shun the changed version before they can try the original, effective one.

The next steps—for anyone who works with these teachers—become searching for how to reconcile old principles in the changed version with new ones that the original “start here” was based on. Sometimes there’s no solution. The principles are too conflicting. Sad. Yet it all could’ve been avoided by just taking the “start here” and rolling with it. I’ve actually heard back from teachers who’ve experienced both, mostly when it comes to grading practices. Instead of rolling with the “start here,” they tried some weird combo, thought things didn’t work, then gave up only to revert to old ways. Then, sometime later, they gave things another try—exactly how it was presented—and come to find out they’re all of a sudden embracing the change. Again, it all could’ve been avoided.

So, in sticking with the metaphor, what’s your vodka? Let go of that, and why not give rye a try next time?

Grades: Going, Going, Gone!

/ magisterp / 3 Comments

Here’s a quick report having gone nearly 100% gradeless. I say nearly because at my school, the halfway point of the quarter (i.e., progress reports) requires a grade. So, as of right now there’s a course grade that shows up. This practice isn’t quite in line with a true ungrading approach that would have a grade only at the very end of the grading period. I’m nearly there, and have a feeling this is as far as I’ll go, too. But that’s not a problem. There’s already been a big difference in the most important areas, and I expect things to get even better.

Continue reading →

SBG: On Point With Assessment, Behind The Times With Grading

/ magisterp / Leave a comment

But first…earlier this week, I shared a recent post on using portfolios to grade equitably, and some dude characterized me as a cowardly idealistic privileged and overeducated white savior who claims to have some solution to problems that minorities face. That’s a lot to unpack, and I’ll leave most of it alone. It’s true that I’m a college-educated white man, placing me in one of the highest privileged boxes possible. No one, though, is claiming to solve society’s inequity with a handful of grading practices in school. Perhaps more importantly, though, it’s downright naïve to think that teachers have no influence and suggest that they can’t do something about a broken system. Grading is a systemic problem, it’s broken, and we’ve known about that for over 100 years (Rugg, 1918). Many teachers should feel empowered to do something about it in the space they have control over: their classrooms (and possibly school).

I now just feel sad for that dude of so many words who wrote such uncalled-for ad hominems. I hope he finds a way to deal with whatever pain he’s going through. I’m gonna stick to using this admittedly privileged platform to share what I’ve been reading and learning about with a just-as-admittedly privileged background in education and a current Ph.D. pursuit. Hope you get as much out of it all as I have, and can use it to enact change wherever possible…

Standards, Assessment, Grading
We’ve been hearing about standards-based grading (SBG) for decades. It’s a massive improvement from whatever was going on in most classrooms prior to the 90s. Thing is, though, some educators have already moved beyond SBG in terms of grading. Ironically, standards-based grading is no longer the best option for grading! But that doesn’t mean it’s useless. We’ll get to that.

What’s been replacing SBG, though? It’s known as “ungrading.” But even in an ungraded system, teachers are still assessing. Assessments might not look what you’d typically expect. Or, they’re pretty much the same just with no points. Regardless, they’re certainly part of instruction as teachers and students focus more on learning content (and not points, scores, or grades). And a big part of that is standards.

Focusing on content probably involves standards in every case, even if a teacher doesn’t formally have a system of standards. That is, whatever the teacher expects of students, and whatever it takes to learn the content, could be and probably already is expressed as a standard, somewhere. Standards are a good way to organize learning. Within this framework, then, standards have a big role to play, just not in grading…

Continue reading →

Equity In Portfolios

/ magisterp / Leave a comment

Averaging scores benefits only two kinds of students: those who show understanding consistently, and those who come into the classroom already understanding the content. If by chance the inequity of that is unclear, let me explain…

Let’s start with every other kind of student, like the one who comes into class with less understanding—for any reason outside of the teacher’s control—broadly described as being less-privileged. A less-privileged student with lower understanding will have lower scores than a more-privileged student who already has more understanding. This is a fact. As the year goes on, the student with lower understanding certainly has the potential to learn content and get higher scores. However, when all the scores are averaged, the less-privileged student will have a lower grade even if making large gains over time.

Pause here.

Now, consider the kind of student that averaging benefits: one who comes into the classroom already understanding content and who starts off with high scores, not low ones. As the year goes on, this already-successful student will have their high scores averaged, and end up with a higher grade than a less-privileged student even if making zero gains over time. This last point is a research interest of mine, and one that isn’t given enough attention when we talk about grading for equity. Whereas the common thinking with a standards-based approach is that it doesn’t matter how a student learns the content and meets the standard, only that a student learns the content and meets the standard, such thinking doesn’t account for any massive gains that still fall short despite conditions outside of school. Nor does such thinking address the already-successful student who can meet the standard with no effort at all. Granted, grading effort/participation is generally a no-no, but what message is being sent if a student can meet standards without learning anything? If they’re privileged enough to have knowledge and understanding, where does individual growth come in when you think of the lifelong learner that so many schools claim to produce?

Cue portfolios!

Continue reading →

Do Now/Activator/hodiē

/ magisterp / Leave a comment

For a few years now, I’ve been starting class with a calm, focused, 5 minute task for students to do while I take attendance and recover from our not-much-transition-time between classes. This is crucial. I greet students at the door—itself a high leverage practice that tells us so much about how a student might be doing that day and what their needs are—then, when official class time begins, I walk in, and project the task. I say nothing to begin. There’s no corralling, no raising my voice, no vying for attention. Students know that once the greeting is projected, it’s time for Latin. If anyone’s still in that “class transition” mindset instead of a Latin class one, I casually walk around, perhaps supplying them a pencil, pointing to our scrap paper area, or motioning towards the task, and we’re off and running with the start of class. This is part organization, part content, and part classroom management. N.B. this opening all takes place on my projected ONE doc. See this video from 2020 for more details. It’s basically stayed the same, even with in-person teaching.

What’s the task?
Nine out of ten times it’s to copy the date, weather, and a short greeting into notebooks added to throughout class, which serves as some optional portfolio learning evidence. This is the start of the weekly sheet/packet routine. While I no longer use weekly sheets, many things that used to be on them still end up in a student’s notebook, just with less structure and repetition. I like the flexibility now to include all of the weekly sheet content during a given class, or none of it besides the date. Last year, I added a note or two below the date, or some commentary about the school week’s or day’s agenda. This whole idea is really an indispensable way to firmly anchor the start of class, especially when there’s some kind of short task to get things going. Recently, though, I was thinking how to use this class opener for a more robust text. Nothing fancy, but there’s opportunity there. Here are my ideas for this year:

  • Magister P [discipulīs/classī Venetae/omnibus, etc.] salūtem dīcit
    Yeah, why not? If students have been copying “salvēte omnēs” all these years, might as well infuse some conventional Roman stuff each class. We rarely have characters writing letters to each other in class stories, but maybe we could start!
  • More weather descriptions & Q/A adverbs
    Instead of “pluit” or whatever, some commentary/question about rain would be a nice springboard for a quick discussion. In terms of adverbs, I’m thinking of drawing more attention to the Q/A posters by using phrases such as “mihi haud placet.”
  • Center piece: “describe the object, in Latin”
    Several years ago, I started Friday classes with a “guess what it is” or “what’s in the box?” kind of prompt. Drawing from that idea, I’ll either place an object on display, or put an image of something below the day’s greeting, having students describe what they see. This is a different kind of writing we don’t do much of, and I’m hoping that doing more of it will produce rich descriptions of items and characters in our stories.

The More You Hold Onto, The Harder The Shift

/ magisterp / 1 Comment

For this post, I had my favorite education topics in mind: grading and second language teaching. Just like the language teacher shifting focus to comprehension and maybe communicative purpose (and away from grammar, drills, paired speaking activities without purpose, etc.), any teacher shifting focus to learning (and away from grades) must change at least some of their practices for a successful and smooth rollout. How much change and what kind? The title says it all, but let’s take a closer look…

Principles & Assumptions
Key to shifting practice is adhering to certain principles and letting go of some assumptions. Otherwise, there’s not much of a shift at all. In fact, the more a teacher holds onto their old principles & assumptions, the harder it will be to make any kind of move. For a few years now, I’ve been recommending overhauling a few key practices so that new ones run smoothly. This is against common advice to just try something new little-by-little, I know. However, while that sounds appealing, in my experience the results are almost never what anyone wants. Consider the language teacher who adds tiered texts and embedded readings, yet holds onto measuring how well students identify verb endings. Sure, more-comprehensible texts is a step in a different (and dare I say “better”—gasp!) direction, but those grammar tests & quizzes under old principles will hinder the new practice.

It’s the same with grading.

If a teacher wants to try something new but holds onto aspects of their old system, there’s likely a conflict of principles, even if the teacher wasn’t aware of the old principles (which most often the case because no one really teaches teachers anything about grading). For example, attempting to teach for “mastery” while setting the gradebook to average scores creates a problem: the student who eventually masters content still has their previous, lower scores in the mix. That doesn’t make sense. While on the one hand, it’s appealing for the teacher to shift their thinking in terms of having standards to master, on the other hand the shift won’t fully be realized without adhering to the principles that make the shift actually work. In this example, the teacher would have to truly evaluate student work to make sure the most-recent learning evidence does show mastery, and have the grade reflect that. The computer can’t do that. Sure, it can automatically update the grade with a standard’s most-recent score, but that’s not the same. The computer doesn’t know whether the student had a bad day, or whether a really high score was a fluke. That’s why teachers need to collect multiple pieces of learning evidence and really know their students.

Of course, that’s if you bother with grades and points in the first place!!!!

From what I’ve seen and read in the literature, ungrading is where everything’s headed. We’ll have to wait until SBG is the dominant paradigm first, though, but I do predict more educators will recognize the ineffectiveness of and harm that grades do, just like what will eventually happen with grammar and language teaching.

So, what’s something you’re holding onto that’s preventing a smooth shift to something new?