Most tricky questions are the misguided product of a teacher thinking they’ve created a valid or rigorous assessment. Validity is when the assessment measures what it’s supposed to measure. This usually means that assessments show that students know what was taught. When it comes to teaching a language, teachers lacking Second Language Acquisition (SLA) training tend to select the wrong thing to be measured (e.g. grammar, cultural facts, etc.). These things usually include tricky details, which lead to tricky questions. Validity then becomes an issue when these teachers use such assessments as evidence that they successfully teach “communicatively” or “for fluency,” when they’re only assessing memory and knowledge about the language system and its speakers. Rigor then muddles things up.
Rigor is not well defined in most school systems, but people (i.e. parents, admin, evaluators, colleagues, etc.) seem confident when they BELIEVE it’s not there. As such, teachers are under pressure to create assessments that seem rigorous, but these assessments just end up being longer (i.e. obtrusive), complex, and downright sneaky. Here’s an example I lifted from a teacher’s assessment. It’s a weak example, but serves our need for the purpose of discussion:
A) Find the matching pairs.
It’s 11:00am a las ocho y media
It’s 6:00 son las once de la mañana
at 8:30 son las once de la noche
It’s 11:00pm a las seis
This teacher, for whichever reason stated above, wants to assess whether students know how to say “in the morning” and “at night” in Spanish. They’re also concerned with how well students know their numbers for telling time. First of all, this sort of thing doesn’t need to be taught or assessed; only the numbers 1 and 2 are the most frequent, we rarely see numbers spelled out, and it’s just boring. That’s right. Time lessons are super boring, and boring doesn’t lead to acquisition. OK, moving on.
Regardless of choosing a pointless thing to measure, the only difference on this assessment between the two times of the day is the letter “a” and “p” of “am” and “pm” denoting morning or night. These are pretty close and could be easily overlooked among 20 other matching items. There’s no reason to make this a puzzle; this is an assessment. If the teacher really wanted to assess the difference between “in the morning” and “at night,” there’s no need to use the exact time of day since students who know the difference might just match the wrong one mistakenly. This is a tricky question. Actually, it becomes 2 tricky questions since one mistake becomes 2 wrong. If this matching assessment is graded out of 100, that just brought down the grade while not really telling the teacher what they wanted to know.
Furthermore, there’s absolutely no rigor. There is no Higher Order Thinking (HOT) that you would expect from something rigorous. There is, however a sense of “lets see if they can get this one,” which suggests a challenge. Challenges are rigorous, but if teachers find themselves thinking this way when it comes to assessing, they’ve already set up students for failure.