100% Coverage ≠ 100% Comprehension

A question by a member of the Latin Best Practices FB group prompted me to look into text coverage, which ultimately led me to comprehension. These are two ideas that a lot of people have misinterpreted, much like the “4%er” figure, and even “90% target language use.” I’m thinking people have a hard time with mathematical concepts, and maybe we should avoid percentages moving forward. But first, we should take care of what damage has already been done by looking at simple examples right away:

Text Coverage
Text coverage is measured by tokens. There are five tokens in the sentence “the bird sees the cat.” Two of the tokens in that sentence happen to be the same word. Therefore, “the” represents 40% text coverage. If the reader doesn’t know “the,” they have a text coverage of 60%. The reader who knows everything except “cat” would have a text coverage of 80%.

Comprehension
Comprehension is a different idea entirely. If the reader who doesn’t know “cat” were asked “what does the bird see?” and it were scored, they’d have a comprehension score of zero. If they were asked two questions about the bird, and two questions about the cat, their score would be 50% comprehension with their 80% coverage of the text. Not the same thing.

Reading
Laufer et al.’s research shows that learners need a text coverage—not comprehension—of 98% ideally to read with ease (and 99-100% whenever possible), but that’s just getting through the reading. That 98% figure is just the start of comprehension.

Hold up.

Yeah, that’s right. Knowing 98% of a text—STOP!!—Remember the first section on tokens. It’s not 98 out of 100 different words, but 98 of 100 tokens (i.e. some words probably repeat). So, knowing 98% of a text doesn’t even guarantee comprehension of what is read. That’s quite the trip, isn’t it? It gets worse when we look at some findings from one of Eric Herman’s Acquisition Classroom Memos on exactly how [in]comprehensible reading can get with what seems like decent text coverage.

There’s a lot in that chart, but compare the text coverage to comprehension scores. Even 95% text coverage can get woefully low comprehension (55%). Keep it mind that the higher scores are still in the “most” range, as in learners are understanding most of what they read when they know 95%+ of a text. Also, those vocabulary sizes are incredibly high for what the majority of K-12 teachers should expect from their students. Eric also adds some context to the research:

“For the most part, the above reading studies were done with high proficiency students, ungraded and academic texts, and count word families. A reasonable prediction is that even higher text coverage and vocabulary size numbers are required to enable adequate comprehension of graded texts by lower level proficiency students. And this is not considering levels necessary for a confident and pleasurable reading experience, which would undoubtedly be even higher!

Higher would be 100%. Let’s make sure we set the record straight:

  • Students need to know 98% of a text to read it with ease.
  • Reading with ease from knowing 98% of a text can still result in much lower comprehension scores, like 70%.
  • Coverage ≠ comprehension

Providing students with texts of 98%…even 100% coverage of known words is step zero. It’s actually the minimum hope we could have for students reading with ease with high levels of comprehension. It turns out that text coverage isn’t very important to look at, because even knowing 100% of the words doesn’t guarantee 100% comprehension. It all goes back to vocab as top priority, sheltering whenever possible so gradual exposure to new words increases vocabulary without the burden of incomprehension. What does this mean for class? Probably using even fewer words than you think! Students can’t magically learn thousands of words, so if we expect them to comprehend high levels of what they read—especially during any kind of independent reading—we must use and create texts with a very limited number of words.

Something Strange…

Before I started teaching in 2013, I joined the moreTPRS Yahoo list serve. Then in 2015, I joined Ben Slavic’s PLC. At that time, there were daily—DAILY—conversations about Second Language Acquisition (SLA), with really tough questions being asked, answered, and debated ad nauseam. Then Tea With BVP was launched, with weekly shows until 2018—here are my clips down to the nuts & bolts. N.B. That show was rebooted in a different iteration as TalkinL2 until just about a year ago. Needless to say, I learned a great deal in those five years; far more, in fact, than in my MAT program (no offense, just a result of SLA expert-lacking faculty nationwide).

I cannot overstate this enough when I say that those early years were simply *crucial* in the development of what we all learned about SLA, and teaching languages. Furthermore, what we now know has also been around for like 10, 20, 30, even 40 years beforehand! That is, most scholarship in the last decades haven’t really changed the game of what’s been discussed since the 70s. The problem? This stuff wasn’t (isn’t?) mainstream—at all. That “golden age” of my SLA development involved the constant interaction with perhaps 200, maybe 300 teachers, almost all of whom I can reach out to with a click. I also had direct access to researchers, their ideas, as well as teachers implementing and arguing about what is, essentially, “best practice” for teaching languages in schools. When we didn’t understand, we emailed and got answers. And we were fringe. Having been exposed to the same ideas over and over—not just Krashen & VanPatten—thankfully from a variety of perspectives courtesy of Eric Herman, a solid understanding of universal truths (as much as we can call them that in the field of SLA) was being discussed.

Don’t get me wrong. There was a LOT of disequilibrium that nearly everyone had to face (e.g. “wait, so you’re saying that…”), and it was not without major headaches. After all, teachers’ worlds were literally being turned upside-down (no, I do mean literally like what you thought was the cause was just a result). Even the researchers’ ideas were challenged—by “mere” teachers no less—and fruitful discussion emerged, like when Carol Gaab demanded a concrete response about whether co-creating a story was a communicative act (i.e. had purpose). N.B. Yes, it is; entertainment. There was significant growth at that time, but it wasn’t all roses. We could call a spade a spade, or at least all come to agreement that a heart was definitely not a spade, and then talk about how to make that heart into a spade. Granted, this was within the fringe group of hundreds of teachers who had that shared experience, but there was definitely some kind of “tell me if what I’m talking about is complete nonsense” agreement. Whatever it was, it now feels like he heyday of theory to practice and critical looks at each others’ teaching.

Then…

Continue reading

Flashcard Blitz

As a comprehension-based and communicative language teacher, I’ve largely dismissed promoting any use of flashcards due to their connection with memorization. Beyond disappointing research about this kind of explicit learning, my classroom experience has confirmed that the more students are aware of language, the less fluent they seem to become. For example, the frequent note-taking academic students are typically those who can’t respond without second-guessing themselves and checking said notes, overly concerned with accuracy, etc., which slows them down quite a bit. Above all else, teaching practices requiring memorization lead to inequity since individual differences can’t be accommodated. Then, Eric Herman lobbed some mind grenades in Acquisition Classroom Memo #39. He can be trusted to do that, and we’re all better teachers for it…

Continue reading

Backward Design: Bad For Languages

TLDR; Don’t use UbD, especially this next year. COVID-19 messed with everything, so keeping the same expectations is unreasonable. Let’s face it…there’s not going to be any miraculous “catch up,” nor should we expect that. Instead of guessing where students will be in the fall, and how far their proficiency might develop with all the disruptions, try Forward Procedure.

I began writing this post after seeing calls from a lot of language teachers seeking tech tools as answers…to all the wrong questions. Rather than trying to maintain what we’ve done, we’re gonna need to make considerable adjustments to our expectations. Curricular design is one of those.

Backward Design
Sure, it makes perfect sense. You start with the result you want for your students, then go backwards from there, planning learning experiences along the way. It’s been recognized as good teaching across all content areas for at least a decade, and has been around since the late 90s. This is “textbook” best practice. In fact, it’s literally a textbook…

Continue reading

Demonstrations: Providing Compelling Input

I’ve known that Krashen et al. suggested long ago that using Total Physical Response (TPR) to teach basic dance steps, martial arts, magic tricks, etc. results in compelling input. I’ve seen presenters talk about the idea of doing so, but haven’t really seen it much in classrooms. Now, I’ve performed one of Eric Herman’s magic tricks, but I didn’t really think I knew how to do anything that I could instruct students to do.

However, Tuesday was one of those perfect times to try something new during a weird day because the rest of the week was midterms. Since I got into archery this year, I decided to bring in my bow to demonstrate basic assembly, shot cycle, and target point values. Yes, I cleared this with security as well as admin, and no, I didn’t bring any arrows.

The experience was fantastic.

Students were captivated for a solid 45 minutes, and there’s no surprise why. Humans are naturally curious learners. It’s just that the school system has destroyed the joy of learning. If we can pause that “school feeling” for a moment, we bring back the joy. After my demo using common vocab, I projected a list of archery-specific phrases, and we co-created a quick text on archery. From there, I put together a more comprehensive packet. However, I wasn’t teaching words. I was teaching about archery. That’s the content.

In pedagogical terms, this is content-based instruction (CBI). Students asked a lot of questions about the bow. Why? It’s cool. In comparison, though, they didn’t ask as much about Roman apartment buildings last month. Why? That’s kind of boring, no matter how well we connect the content to their lives. Of course, exploring Roman content works the same way as exploring archery. It’s just that it takes someone with particular interests to get as excited about Romans. However, I’m not convinced that this should be either/or. I’ll both continue to explore Roman content (in Latin), as well as teach about other content (in Latin).

I’m now looking for other things to demo. Drumming might be one. After performing that card trick, I suppose I could teach it. In all of this, I’m reminded of how beneficial it is to include students in the demo process (e.g. distribute toy bows, drum sticks, decks of cards, etc.).

So, What could you teach your students?

2018-19 Syllabus & Latin Program Mission & Vision

Unless you’re an island of one, a program Mission & Vision is a good idea to keep the department heading in a similar direction, even if things don’t start out that way. I put a lot of time into crafting the document last spring, and just had some help from my admin for the final touches. Once that was squared away this week, I could hand in my 2018-19 Syllabus. Let’s unpack all that…

Continue reading

Studies Showing the Ineffectiveness of Grammar Instruction & Error Correction

Greek_1st_declension_contract_nouns

How effective is studying these “rules?” Research shows “not at all!” What was lurking beneath all that studying for those claiming it did, in fact, work? Comprehensible Input (CI).

**Updated 1.29.2020**
with Rothman (2008) 

All of this research has been shared by Eric Herman, either in the Acquisition Classroom Memos, his book, “Research Talks…,”or from my direct requests. Thanks, dude! As you’ll see, there is very little support (none?) for explicit grammar, or traditional rule-based language instruction. Even effectiveness aside, it should be clear that the practice has no place in inclusive K-12 classrooms (and probably beyond), since affective factors—alone—are shown to result in enough negative consequences. N.B. The highly-motivated independent adult learner can, and probably will do anything they want, and/or feel is helping them regardless of any proof. K-12 students are NOT those people.

Continue reading

The Problem with Non-Targeted, Targeting 1, and Targeting 2

In 2013, Stephen Krashen wrote an article, The Case for Non-Targeted, Comprehensible Input, about the problems of the traditional “rule of the day” grammar syllabus. Krashen not only wrote how this “targeted” grammar and vocabulary has disadvantages, but also how TPRS reduces such problems, even ending the section with:

“Although TPRS probably succeeds in reducing the problems of the grammatical syllabus, there is another possibility: Non-targeted comprehensible input.”

At this point, it appears that the “targeted” nature of TPRS and non-targeted are—probably—on par, and that it’s really just an option of what appeals to you…

Continue reading

Unit Test “Mastery” (UTM)

Unit Test “Mastery” (UTM) is a symptom many teachers and students suffer from. The teacher:

  1. presents content (Present)
  2. provides a learning experience (Practice)
  3. announces an assessment
  4. assesses students (Produce)
  5. chooses remediation based on low performance, or moves on
  6. repeat

The consequences of UTM is that students appear to “master” the content either right away or after the remediation, which itself is usually misinterpreted as assisting a “struggling student.” The teacher then moves on, and students seldom run into the same content, even from what you might expect from cumulative courses (e.g. one-off math/science concepts, or that perfunctory “transportation unit” in which students are given a vocabulary list for all possible—and likely outdated—ways to get around Madrid, etc.).

This symptom seriously misleads the teacher. It’s one source of validating teaching practices that don’t actually produce results they seem to be producing. For example, most language teachers attribute their understanding of language to how they were taught, yet they’ve probably just been exposed to the language daily over time, teaching similar (same?) content year after year. This looks like proficiency, yet is probably just daily recall of translated and memorized information!

In reality, communication isn’t really something anyone can master, at least not in the subject-matter-learning sense used in other content areas. There’s a lot of pressure to make language courses fit what’s expected in school, but the model fails when we have inclusive classrooms based on universal human traits, and not intellectualizing language. The best teachers are able to resist that, educating their administration, or at least find the wiggle room to provide input and encourage interaction in a second language during the school day—something all humans are hardwired for.

I encourage everyone to find alternatives to traditional units accompanied by lessons with limited flexibility. Instead, meet students where they are, and move forward. One way to think about curriculum is basing it on vocabulary frequency, but not thematic (e.g. Greetings, Getting Around, Sports, etc.). Chris Stolz has shares how Mike Peto’s entire department has taken this to an extreme with fantastic results! All of these ideas are supported by what Eric Herman has coined “Forward Procedure:”

Forward procedure is process-oriented. It focuses on where students are. That doesn’t mean you can’t have tests, but those are not pre-determined. They are created in response to what has happened in class and tailored to where students are. If there had to be an element of “standardization” between sections, this would be to agree to use the same test format, but not the same content (e.g., sections hear a different story and do a timed rewrite). Rather than focus on something to cover, it focuses on giving students what they want and need in that moment to learn. It is the approach that makes a teacher most responsible to the learner. In a second language, communicative classroom, this is a much better fit. To quote Savignon (1976): “Above all, remember that for it to be real, communication must be a personalized, spontaneous event. It cannot be programmed – but you can make it happen” (p. 20).