Methods & Results: To What Do We Attribute Success?

Not every teacher shares how well their students are doing—probably out of fear of being criticized—and I don’t blame them one bit. This data is often kept under lock and key, so it’s hard to get a sense of whether all the talk amounts to something. SPOILER ALERT: it does. The reports I’ve seen on how well students have been doing under a…NOT…grammar-translation approach tend to attribute success in different ways, though. Today, I’m looking at two such programs to see if we can narrow down what contributes success:

Program 1:

  • 69% of Latin V students score Intermediate Mid (I4+) on ALIRA
  • Focus on reading
  • Translation of what is understood (vs. in order to understand)
  • Uses LLPSI (Lingua Latina per se Illustrata)
  • Uses novellas & other sources of input
  • Speaks Latin whenever possible (i.e. judicious use of English)
  • Establishes meaning in English (i.e. fēlēs = cat) when students ask
  • CI is necessary, but not sufficient for acquisition
  • Extensive interaction is most important

Program 2:

  • 64% of Latin IV students score Intermediate Mid (I4+) on ALIRA
  • Focus on reading
  • Translation of what is understood (vs. in order to understand)
  • No textbook
  • Uses novellas & other sources of input
  • Speaks Latin whenever possible (i.e. judicious use of English)
  • Establishes meaning in English (i.e. fēlēs = cat)
  • CI is necessary, and sufficient for acquisition
  • Interaction is important

The results are very close by the end of each program, and there’s definitely more in common than not, but what isn’t in common makes for differently-enough teaching and learning environments. Both are just as successful, but what can we attribute that success to? Let’s look into those differences a bit more…

Continue reading

100% Coverage ≠ 100% Comprehension

A question by a member of the Latin Best Practices FB group prompted me to look into text coverage, which ultimately led me to comprehension. These are two ideas that a lot of people have misinterpreted, much like the “4%er” figure, and even “90% target language use.” I’m thinking people have a hard time with mathematical concepts, and maybe we should avoid percentages moving forward. But first, we should take care of what damage has already been done by looking at simple examples right away:

Text Coverage
Text coverage is measured by tokens. There are five tokens in the sentence “the bird sees the cat.” Two of the tokens in that sentence happen to be the same word. Therefore, “the” represents 40% text coverage. If the reader doesn’t know “the,” they have a text coverage of 60%. The reader who knows everything except “cat” would have a text coverage of 80%.

Comprehension
Comprehension is a different idea entirely. If the reader who doesn’t know “cat” were asked “what does the bird see?” and it were scored, they’d have a comprehension score of zero. If they were asked two questions about the bird, and two questions about the cat, their score would be 50% comprehension with their 80% coverage of the text. Not the same thing.

Reading
Laufer et al.’s research shows that learners need a text coverage—not comprehension—of 98% ideally to read with ease (and 99-100% whenever possible), but that’s just getting through the reading. That 98% figure is just the start of comprehension.

Hold up.

Yeah, that’s right. Knowing 98% of a text—STOP!!—Remember the first section on tokens. It’s not 98 out of 100 different words, but 98 of 100 tokens (i.e. some words probably repeat). So, knowing 98% of a text doesn’t even guarantee comprehension of what is read. That’s quite the trip, isn’t it? It gets worse when we look at some findings from one of Eric Herman’s Acquisition Classroom Memos on exactly how [in]comprehensible reading can get with what seems like decent text coverage.

There’s a lot in that chart, but compare the text coverage to comprehension scores. Even 95% text coverage can get woefully low comprehension (55%). Keep it mind that the higher scores are still in the “most” range, as in learners are understanding most of what they read when they know 95%+ of a text. Also, those vocabulary sizes are incredibly high for what the majority of K-12 teachers should expect from their students. Eric also adds some context to the research:

“For the most part, the above reading studies were done with high proficiency students, ungraded and academic texts, and count word families. A reasonable prediction is that even higher text coverage and vocabulary size numbers are required to enable adequate comprehension of graded texts by lower level proficiency students. And this is not considering levels necessary for a confident and pleasurable reading experience, which would undoubtedly be even higher!

Higher would be 100%. Let’s make sure we set the record straight:

  • Students need to know 98% of a text to read it with ease.
  • Reading with ease from knowing 98% of a text can still result in much lower comprehension scores, like 70%.
  • Coverage ≠ comprehension

Providing students with texts of 98%…even 100% coverage of known words is step zero. It’s actually the minimum hope we could have for students reading with ease with high levels of comprehension. It turns out that text coverage isn’t very important to look at, because even knowing 100% of the words doesn’t guarantee 100% comprehension. It all goes back to vocab as top priority, sheltering whenever possible so gradual exposure to new words increases vocabulary without the burden of incomprehension. What does this mean for class? Probably using even fewer words than you think! Students can’t magically learn thousands of words, so if we expect them to comprehend high levels of what they read—especially during any kind of independent reading—we must use and create texts with a very limited number of words.

Text Coverage & DCC’s Top 1000

**Updated 2.25.21 with details from this post**

The DCC frequency list is often consulted for choosing which words to use when writing Latin for students. It certainly makes sense to use ones they might encounter over and over again vs. those they might not, but *how* frequent are these frequent words? In particular, I was curious what a student could probably read having acquired the Top 1000 words on DCC’s list. Here’s some quick background…

Continue reading

“…& Classical Humanities”

Almost every degree and teaching license I know of related to Latin attaches “& Classical Humanities” to the end. That is, it’s rare to study and teach the Latin language without also studying and teaching Classical Humanities.

Why is this?

I know, I know. They’re two peas in a pod. It might seem obvious since the culture most associated with Latin is Roman, part of the Classical era. Yet Latin has been around for thousands of years, right? Many cultures have used Latin, and not all of that Latin has been about the Romans, either. Consider teaching Classical Humanities without Latin. It’d be a history class focus on a particular time period, right? That’s like a history class on 18th century Spain. Now consider a high school Spanish language class. Surely, students don’t learn only about the18th century, much less Spain’s entire history, or even focus on just Spain at all! There are tons of Spanish-speaking cultures that have written about a ton of different stuff, and Spanish language classes take that into account.

Why not Latin?

Of course, the context of a Spanish class seems different, but is it, really? The Latin language didn’t die with the fall of the Roman Empire. In fact, non-Roman cultures have now been using Latin longer than the Romans existed. I’m not saying there are now more texts written by non-Romans than Romans. Then again…

TheLatinLibrary.com
The second I wrote that, I suddenly realized I had no idea whether it could be true. In what a colleague would say is very “on brand” of me, I ran some numbers through Voyant Tools, taking all ancient Latin texts from TheLatinLibrary, and comparing the total words to the Miscellany, Christian, Medieval, and Neo-Latin categories. It must be noted right away that this doesn’t represent all of the world’s extant Latin. In fact, I’m reading a work of elegiac couplets from the 15th century by Vincent Obsopoeus that’s nowhere to be found on TheLatinLibrary. There are thousands of words of Latin in there, but it won’t appear in my data. You won’t find works like Cornelia, or Ora Maritima, either. There’s no Hobbitus Ille in the data, and women are utterly underrepresented, with perhaps just Sulpicia and Egeria included in TheLatinLibrary at all. So, my source has its flaws, yet what we have of ancient Roman Latin is all there, or nearly all there, and these estimates* help put things into perspective. Of course, I went into this wondering if the world has surpassed the Romans in writing of Latin—prompting more inquiry into why the Classical Humanities are still a focus in high school Latin study—and the truth is undeniable, especially when acknowledging there’s so much recent Latin unaccounted for. Bottom, line, far more Latin has been written since the Romans than what you see here, which is already more than what we have from them:

Continue reading

A Solution To Asking Wrong Questions (e.g. “How Do You Teach X?”): Focus & Flip

Ask 10 teachers how they teach X, and you’ll probably get 10 different responses. However, if you flip it, and instead ask “how do students learn X?” you might get what in many cases is the only answer. Furthermore, it helps to focus the question first because most “how do you teach X?” questions are way too broad. Teachers can’t possibly teach everything about X, so there’s gotta be a more specific outcome to the question. What is the point of X? Or, what are students expected to do, or know about X? For example…

  1. Take a question teachers always ask:
    • How do you teach the subjunctive?
  2. Focus it:
    • How do you teach students to identify subjunctive verb forms?
  3. Flip It:
    • How do students learn to identify subjunctive verb forms?

In this case, the answer is quite simple: students must memorize verb forms. There’s no way around that one. Humans won’t spontaneously infer which verbs are subjunctive. To identify them, students will have to be shown what they are, commit them to memory, and then recall from memory. So, the teacher who expects students to identify subjunctive verb forms needs to provide them, and hope their students have good memories (oh right, that last part is out of their control). Not a very reliable thing to expect, it turns out.

Consider back to the alternative, too. Just think of all the different answers you could get to “how do you teach the subjunctive?” They’ll probably all be from the teacher’s perspective, like descriptions of activities, and have nothing to do with the actual learning that must go on, too. This is probably why so many teachers reinvent the wheel year after year. The teaching isn’t actually addressing what students need to learn. Of course, that grammar question is a bit silly since the focus doesn’t have much use. Let’s look at a related question with a more useful purpose…

  1. Q: How do you get students using the subjunctive?
  2. Focus: How do you get students to speak using accurate subjunctive verb forms?
  3. Flip: How do students learn to accurately speak using subjunctive verb forms?

This answer is also simple: time & exposure. Accuracy, especially in speaking, isn’t expected for the first years (3-4+), with or without any “error” correction, either. For any language to come out (output), students need lots of examples coming in (input). So, the teacher who expects accurate use of subjunctive, then, needs to ensure that there are tons of examples of subjunctive verb forms in what students listen to and read. Oh, and they also need to have patience. Any teacher who expects—and gets—beginner students speaking accurate subjunctive verb forms either doesn’t know the research, measures that in isolation and moves on, or is seeing short term memory results. Yet also probably holds “review” sessions each year!

So, give Focus & Flip a try!

Something Strange…

Before I started teaching in 2013, I joined the moreTPRS Yahoo list serve. Then in 2015, I joined Ben Slavic’s PLC. At that time, there were daily—DAILY—conversations about Second Language Acquisition (SLA), with really tough questions being asked, answered, and debated ad nauseam. Then Tea With BVP was launched, with weekly shows until 2018—here are my clips down to the nuts & bolts. N.B. That show was rebooted in a different iteration as TalkinL2 until just about a year ago. Needless to say, I learned a great deal in those five years; far more, in fact, than in my MAT program (no offense, just a result of SLA expert-lacking faculty nationwide).

I cannot overstate this enough when I say that those early years were simply *crucial* in the development of what we all learned about SLA, and teaching languages. Furthermore, what we now know has also been around for like 10, 20, 30, even 40 years beforehand! That is, most scholarship in the last decades haven’t really changed the game of what’s been discussed since the 70s. The problem? This stuff wasn’t (isn’t?) mainstream—at all. That “golden age” of my SLA development involved the constant interaction with perhaps 200, maybe 300 teachers, almost all of whom I can reach out to with a click. I also had direct access to researchers, their ideas, as well as teachers implementing and arguing about what is, essentially, “best practice” for teaching languages in schools. When we didn’t understand, we emailed and got answers. And we were fringe. Having been exposed to the same ideas over and over—not just Krashen & VanPatten—thankfully from a variety of perspectives courtesy of Eric Herman, a solid understanding of universal truths (as much as we can call them that in the field of SLA) was being discussed.

Don’t get me wrong. There was a LOT of disequilibrium that nearly everyone had to face (e.g. “wait, so you’re saying that…”), and it was not without major headaches. After all, teachers’ worlds were literally being turned upside-down (no, I do mean literally like what you thought was the cause was just a result). Even the researchers’ ideas were challenged—by “mere” teachers no less—and fruitful discussion emerged, like when Carol Gaab demanded a concrete response about whether co-creating a story was a communicative act (i.e. had purpose). N.B. Yes, it is; entertainment. There was significant growth at that time, but it wasn’t all roses. We could call a spade a spade, or at least all come to agreement that a heart was definitely not a spade, and then talk about how to make that heart into a spade. Granted, this was within the fringe group of hundreds of teachers who had that shared experience, but there was definitely some kind of “tell me if what I’m talking about is complete nonsense” agreement. Whatever it was, it now feels like he heyday of theory to practice and critical looks at each others’ teaching.

Then…

Continue reading

“Not-Reading” Synonyms

It would take a proficient Latin speaker about 7 hours to read Caesar’s Dē Bellō Gallicō—in its entirety*—at a slow pace (i.e. half the average reading speed).** For comparison, a proficient English speaker reading at the same pace would take over twice as long to get through The Hate U Give (~15 hours). One of these texts is level-appropriate, and now commonly used in 9th grade classes along with 4-5 other full length books and many other short texts throughout the year. The other is nowhere near level-appropriate, yet commonly used in 11th or 12th grade classes as roughly half the year’s focus—certainly not in its entirety—with selections comprising just 13% of the full text. It should be clear which is which, and any K-12 teacher who says their students read Caesar is being as truthful as today’s outgoing president, who has mislead and lied over 29,000 times in office.*** Yet if not an outright lie, the claim of reading Caesar is still highly misleading, and should be addressed ASAP…

Continue reading

The Problem Is Vocab, Not Grammar

This post is not about teaching grammar. This post is about its role in comprehension. Grammar can tell you a word’s function, but what impact does that have if you’re struggling to understand what words mean?! It’s still all about words. In fact, all words contain grammar. If you know what a word means, you’re a little bit closer to acquiring its grammar each time you encounter it. In this post, I use a language I’ve made up for other demonstrations, aptly dubbed Piantagginish, to show how vocab—not grammar—is the real problem regarding comprehension. The pedagogical takeaway is to avoid vocab overload, and shelter vocab whenever possible…

Continue reading

CI, Equity, User-Error & Inequitable Practices

I don’t agree that the statement “CI is equitable” is harmful. Yet, I also don’t think the message behind “CI isn’t inherently equitable” is wrong, either. John Bracey said one can still “do racist stuff” while teaching with CI principles. Of course, we both know that’s an issue with content, not CI. Still, I get the idea behind that word “inherent.” In case you missed the Twitter hub bub, let me fill you in: People disagree with a claim that CI is “inherently equitable,” worried that such a message would lead teachers to say “well, I’m providing CI, so I guess I’m done.” I don’t think anyone’s actually saying that, but still, I understand that position to take.

Specifically, the word “inherent” seems to be the main issue. I can see how that could be seen as taking responsibility away from the teacher who should be actively balancing inequity and dismantling systemic racism. However, teachers haven’t been as disengaged from that equity work as the worry suggests. I’ve been hearing “CI levels the playing field” many times over the years from teachers reporting positive changes to their program’s demographics. What else could that mean if not equity? But OK, I get it. If “inherent” is the issue, maybe “CI is more-equitable” will do. If so, though, at what point does a teacher go from having a “more-equitable” classroom to an “equitable” one? And is there ever a “fully-equitable” classroom? I’m thinking no. So, if CI is central to equity—because you cannot do the work of bringing equity into the classroom if students aren’t understanding (i.e. step zero), and nothing has shown to be more equitable than CI, well then…

For fun, though, I’ll throw in a third perspective. Whereas you have “CI is equitable” and “nothing makes CI equitable per se,” how about “CI is the only equitable factor?” I’m sure that sounds nuts, but here it goes: Since CI is independent from all the content, methods, strategies, etc. that teachers choose, as a necessary ingredient for language acquisition, CI might be the only non-biased factor in the classroom. Trippy.

I don’t think that third perspective is really worth pursuing, though, so let’s get back to the main points. Again, I understand the message behind “CI isn’t inherently equitable” as a response to “CI is equitable.” However, I suspect the latter is said by a lot of people who aren’t actually referring to CI. Don’t get me wrong; some get it, and are definitely referring to how CI principles reshaped their language program to mirror demographics of the school. However, others are merely referring to practices they think is “CI teaching.” This will be addressed later with the Dunning-Kruger Effect. Otherwise, let’s talk equity…

Continue reading

1969: 50 Years of “4%ers”

Just a few months after the moon landing, Superintendent John Lawson (Shaker Heights, OH) gave a speech at the Symposium on Foreign Language Teaching at Indiana University. Its age certainly shows. Then again, were it not for the typeface, you’d think some of these statements appeared yesterday in a blog! I find it striking that such “progressive” and “controversial” ideas have been discussed for 50 years, pretty much coinciding with the civil rights movement, yet without much fundamental change to either. There’s no excuse for the latter. As for second language teaching, that’s slightly more understandable considering the field of Second Language Acquisition (SLA) was hardly established by the late 60s.

To give you a sense of how relevant Lawson’s ideas are today, look at this statement addressing the importance of compelling topics, and what now has become criticism against using unadapted texts driving the AP Latin problem:

There’s also a section, while brief, managing to address topics like teaching to the test, teacher perception of status in their field, elitism, exclusivity, ineffective pedagogy, compellingness, connectedness, comprehensibility, and confidence. All that back in 1969. Holy moly, right?!

That speech also happens to be the source of the “4%er” term that Keith Toda just shared in his latest (and last-for-a-while) blog post. Now, Keith is somewhat of a self-proclaimed man of the shadows not really active on social media, so my first thought was that he didn’t know the “4%er” term doesn’t really come up these days. In fact, I had to go back to a 2015 moreTPRS list email to search for the references contained in here! But maybe that term is exactly what teachers need to be reminded of right now. Let’s start with its history:

Continue reading