Category Archives: Teaching

Chinese Keju Exam Angst…with a tangent

Leave a reply

Every now an again it’s nice to check to see how non-US cultures handled education. A great example is the Chinese Civil Service or Keju exam. Lasting from the 7th century AD to the last dynasty in the early 20th century, this was an exam which determined if a person was academically qualified enough to become a well-paid government administrator/bureaucrat. Today we have counterparts like the SAT (how good a college you can go to), MCAT (how good a medical school) and the LSAT (law school).
On the positive side, exams like these theoretically allow anyone with the resources and ability to learn enough material to pass to become credentialed. But Dr. Hoi Suen from Penn State feels that this kinds of high stakes exam will inevitably lead to problems.

“With approximately 1,300 years of history and extensive official and unofficial records that were kept throughout this period, China’s is the only examination system that can provide us with a glimpse of what might be some long-lasting chronic problems of high-stakes, large-scale testing programs as well as of the efficacy of attempts to remove unintended negative consequences” — Suen (2006)

The Problems

So what were they? Pretty much what you get with the SAT/MCAT:

  • Keju test handbooks similar to the Kaplan handbooks
  • Violent or suicidal behavior from students who either failed the exam or were studying for the exam
  • Massive cheating in the form of
    • students memorizing entire essays and poems to copy down later
    • bribing test proctors and graders – sometimes they looked for “key phrases” in essays so graders would know who to score highly
    • hiring fake test takers
    • bringing in crib notes
  • Security implementations such as body searches, isolated exam rooms, anonymous forms and physical punishment for convicted cheaters.

Baton Effect

In addition to these issues, Suen also found something called the baton effect – which basically says that Chinese society focused only on learning material on the test, which in this case meant literature and poetry at the expense of medicine or technology. The baton effect shows that high-stakes testing actually can influence what people learn.
Think of how many people today who complain that we don’t study literature and poetry enough because the US students are too concerned with studying “for-profit” fields like accounting, medicine or the law.
So the problem may not be that testing will inhibit education, but that it can be TOO influential. You ideally want a test that matches what your society really needs, and yes there are plenty of diverging views out there.

Any Solutions?

Of course, China and the U.S. are not the only high-stakes testers out there. Most countries today have some sort of high-stakes test for university admissions and some used to have them for high school. And I think it’s safe to say that once a test becomes high stakes enough to count, you will get the cheating and the destructive psychological behaviors described. I’ve heard great academic dishonesty stories from some of my non-US colleagues.
But what are the alternatives? Traditionally the alternative has been “the old boy’s network” or whatever variant you have in mind. I need to hire someone for a task and I check in with my social network to see what “qualified” applicants are out there. Actually, when the population is small scale, this might actually be the best solution.
But once your population gets too large, caste think tends to set in. Anyone not born in the right circles would somehow have to find a patron (possible, but not easy).
Is there a third way that’s more equitable? Maybe the ultimate solution is just to open more pathways to success. We’re not all meant to be doctors or lawyers or government workers, so why should we all be trying? Wouldn’t a system that rewarded something other than academic performance be interesting?
For instance, a highly-skilled welder may actually be well-paid and know quite a bit about metallurgy, gas chemistry and structural engineering. Welders may even need to receive continuous training to keep up with the latest techniques…Some welders even wield their torches to become metal craftsmen (and their art may command high prices).
But how many professors or lawyers want their children to grow up to become welders? (Actually see P.S. 2 for my reasons why not).

Class and High Stakes Exams

All of this speculation leads me to think that the biggest reasons for the problems encountered by the Keju and the SAT/LSAT/MCAT is that they are entries for people to gain or maintain a relatively higher social status. Hence there is much more competition in them (as well as a very strong desire to circumvent the system).
There are actually lots of other high stakes certification exams like the CPP (Certified Payroll Professional) and ones for welding, yet I don’t think the issues of academic dishonesty are quite as prevalent. They’re challenging, but not as many people take them.

Postscripts

P.S. 1 – An interesting development recently is that “chef” has become a much more glamorous profession thanks to the rise of cooking channels. One person admitted that he was glad to have found cooking…because he really hated school.
P.S. 2 – Another annoying quasi universal is that societies often set the highest social class to those who do the least work. Cognitive labor is always above manual labor, and no labor at all can be the best. I think that part of it is that manual labor can be a bit dangerous (welding accidents are more likely than attorney accidents). And it is cleaner, which counts for a lot in our subconscious mammal brains….still many societies have missed out on key technical innovations because the philosophers “didn’t want to get their hands dirty” doing actual experiments.

Cognitive Load Theory…I Finally Found You

Leave a reply

A few weeks ago, I was complaining that the version of instructional theory I knew did not reference specific mechanisms of cognition (Help Wanted: Linguist Seeking Cognitive Components). But my horizons recently got expanded to include Cognitive Load Theory. Actually I first found it in The Cambridge Handbook of Multimedia Learning (Chap 2) in an article written by John Sweller. The key concept I like

Learning has been defined as an alteration in long-term memory. If nothing has altered in long-term memory, nothing has been learned.

Specifically cognitive load theory says that knowledge (I assume facts/procedures in this case) is stored with some organizational structure attached (possibly all hierarchical schemas or schemas plus other structure). In contrast, new information has to be filtered through working memory (a type short term memory) Sweller proposes that when receive information, the working memory will try to get to a schema from long term memory to reduce cognitive load (I would say you try to recognize first, then learn new information). Interestingly, he proposes that if the learner’s memory can’t get a schema, the learner may first try to see if another person has one available (an instructor, a peer or the textbook). This fits the social aspect of learning, but it slightly contradicts the constructivist approach in that constructivism does not really assume that the learner is looking for a close match internal content organization. They assume the learner is constructing everything from scratch. On the other hand, if the learner already has a schema in place it is easier to process new additions (the more you know, the easier it may be to learn more). On the other hand, CHANGING a schema to fit new information can be pretty tricky. Some non-intuitive predictions I found interesting

  • Redundant/duplicate information adds cognitive load – because you have to process ALL the material before you can determine it was duplicate. Instructional designers sometimes advocate showing the same information multiple ways to help different learner types, but you can get into overkill territory if you’re not careful (been there, done that)
  • Worked examples critical – Sweller cites research that learners may need to see fully worked example problems to best learn the technique. Asking learners to "recreate" a technique from scratch may not be as productive. On the other hand, you do have to help learners transition to solving their own problems. Interestingly, although Sweller does not address the creative arts, it is interesting to note that art is usually taught by showing many examples of how a "design problem" is solved.
  • Experts actually store a lot of "factoids" – But Sweller contends that experts index factoids in such a way so that they can recall the correct one given a current problem they’re solving. It’s different from being able to recite a random list of trivia. But you still need to get the factoids in there at some point…

I think this theory is on the right track, but there are a few valid criticisms I think a constructivist could make:

  • There is no overt role for motivation or emotion – although I generally feel that motivation is something that either enhances or interferes with the ability of learners to place content in long term memory. However, a complete model should take this factor into account somehow, and I actually think a model like this could easily accommodate affective factors as a factor affecting memory storage.
  • Assumes all hierarchical schemas – Actually I think this is what the EXPERTS store (but only "left brained" knowledge). Novices may be storing facts as "unstructured lists" and need help sorting what they know into appropriately structured schemas. Still some creative processes involve a "subconcious" or "right brained" mulling of the problem with strange tangents that is not well understood.
  • Cognitive load theory may be more math/science geared – That is his focus is on learning a set body of facts and procedures. He does not really address issues like creativity in the arts or multiple points of view in sociology. On the other hand, even policy studies rely on being able to interpret facts and figures.
  • Does not acknowledge cultural differences – Even if people are born with the same brain, they don’t get the same upbringing. Conflicts between home culture and academic culture can interfere with learning (because of affective issue). Acknowledging cultural differences can enhance opportunities for learning (especially for the instructor)
    I suspect Sweller would NOT believe different cultures store knowledge with different mechanisms. Different cultures may have different schemas (e.g. the tropics subdivide fruits into "hot" and "cold" varieties for various reasons), but they’re still schemas!
  • Does not acknowledge "inborn" learner differences – On the other hand, some people may wonder if such a thing exists!

Help Wanted: Linguist Seeking Cognitive Components

Leave a reply

As my instructional design colleagues already know, I moved straight from theoretical linguists to Web design, then on to instructional design. I don’t recommend this for everyone because I will be the first to admit I was weak on pedagogical theory. In fact, I had to “construct” my own meaning of “constructivism” and it was full of “cognitive dissonance” (I thought concepts contradicted each other). I’m still not sure I have it right, which is why I’m still a “linguist among constructivists.” Here’s why:
As a good theoretical linguist, I’ve always accepted the Constructivist premise that learning is “is an active process in which learners construct new ideas or concepts based upon their current/past knowledge.” [http://tip.psychology.org/bruner.html] or “must actively “build” knowledge and skills (e.g., Bruner, 1990) and that information exists within these built constructs” [http://chiron.valdosta.edu/whuitt/col/cogsys/construct.html].
This mirrors the Chomskyan language acquisiiton model that assumes that children acquire language by listening to adults (not by overt instruction by the way, but by children processing the raw signal and concocting their own grammar).
So far so good, but the pesky linguist in me immediately asked exactly WHAT kind of structure is a learner constructing? Does it have parts? Do they come in more than one type depending on complexity level (following Bloom’s taxonomy? or verbal vs. kinesthetic?)
After all, linguists divide the language component into components like phonology (sound), morphology (word structure), syntax (sentence structure), semantics (literal meaning) and pragmatics (actual meaning). There must be even MORE components for something like critical thinking or algebra.
Yet, most typical sources on constructivism do not really specify this at all (although I do see the reference to concept map and schema). The closest answer I’ve gotten on the constructivist road is it’s “very complex and counterintuitive”. [http://chiron.valdosta.edu/whuitt/col/cogsys/construct.html]…I’m sure that’s true.
I finally realized after a while that most constructivists assume a “holistic” model in which defining the parts is not necessarily critical. Theoretically, if the child is in the correct learning environment, then the right structure will be built.
At this point, I will have to say that the insight that parts add up IS important. Learning does involve a complex interaction of perception, cultural biases, physical health, previous mental structures and motivation (connation). Mess any one of these up and the learner will more than likely have problems.
But in the end, I can’t abandon the idea of defining components of cognition and learning. After all, HOW do we define the optimal environment if we don’t understand all the components of the environment? Which strategies can we deply to maximize the functioning of each component in the learner? And if we assume learner differences, what are they exactly?
Some say the actual cognitive model might be “too complex” to work with at this time, but if the meterologists can sort through a complex mix of climatological data (carbon emissions, sun spots, humidity levels, season, wind flow, volcanic emissions) to make a weather forecast…I have faith that we can do the same. Meterologists keep refining their models, and so can we. I think it’s important to try.
P.S. What do I think is being constructed in a learner? My best guess is that the learner makes a change somewhere in long term memory and that it varies depending on the content. Choices include semantic memory (facts), procedural (how-tos) and autobiographical (single events). Of course, this also has to go through the perception channels to short term memory to some sort of internal processing. And I don’t necessarily understand how memory chunks are stored and organized.

Linguistics Team/Discussion Activities That Work

Leave a reply

Because my working environment is a strong advocate of team learning, I have been experimenting with group activities…some of which actually work. Looking back through my notes, I think the ones that generate the most excitement (and hence talking) from the students are ones which build on something they already know well (like Thanksgiving).
For reasons I discuss below, most focus on sociolinguistics instead of topics like phonetics or syntax which tend to require more formal “mathematical” machinery.

Thanksgiving Ethnography

I asked students to get in groups and compare Thanksgiving traditions based on several dimensions such as patterns in dinner table conversation (topic, formality, interruptions) and treatment of older relatives (as well as side dishes). It’s a chance for students to see cultural differences in linguistic behavior.

“New Jerseyite” vs. “New Jerseyan”

I first asked students to look up “New Jerseyite” and “New Jerseyan” on Google to determine which form was “correct”. Since students used different strategies, I asked them to meet in groups to discuss how they approached the question then we did a summary. In this activity we generally have a good discussion of “prescriptive” vs. “descriptive” grammar since the dictionary mandates “New Jerseyite” but all native New Jerseyans unilaterally reject the word.

Color Chart

I split the group into men and women and asked each group to assign labels to a color wheel with 12 colors. Unlike the stereotype that “women know more color words”, both men and did equally well in this case. My point here was that individuals can diverge from “group” norms in their behavior.

Investigating Missing Fudge

In terms of an online discussion forum, I found that I got the most passionate answers when students were asked to discuss how they would ask roommates about missing fudge. Answers ranged from expected indirect questioning to outright accusations (but only if they knew the person well).

Why I think they worked

I think students found these the most exciting because they were asked to analyze a “common” situation in a brand new way. Not only did students see connections with the content, but scaffolding was built in. Students were able to take old concepts and critically think about them. With newly learned data, critical thinking seems to be harder.
I have done semi-successful group activities based on new data, but the excitement is not the same and you do often see students who don’t participate because they feel lost. It’s not the same.
One of the more interesting cases was when I asked students to solve a morphology problem in pairs (think algebra but with letters). Unlike other activities, students immediately fell silent and worked on the problem individually instead of talking to each other. In this case, their instinct was to work it out “on their own”.
When one instructor told me that she dropped discussion in her Gen Ed class, I wasn’t surprised that her reason was “the students didn’t know enough.” At the Gen Ed stage, students may still be stuck in the low level knowledge and fact stage and not yet to advance to a higher level on content alone. A connection to something they already knew in daily life may have been needed.
As for critical thinking outside “daily life”, maybe that IS something that needs to wait a semester (or at least a few weeks). There may be a time issue involved in moving from level to level.
I’m always amused when a paper talks about the benefits of class discussion and the example comes from a graduate level class. If they’ve made it to graduate school, we can be sure they’ve mastered most of the lower level content already!

Authentic Application for “Cathedral Architecture”

Leave a reply

When I (and many others) are designing instructional materials (e.g. algebra, nutrition, Spanish) I normally focus on finding “authentic uses” for the material. For instance, a Spanish instructor might focus on how to ask for directions or a nutritionist might focus on being able to accurately interpret food labels.
But a recent project was a quiz to help students in an intro Western Civilization course memorize architecture terms. I love cathedrals, but even I was stumped here.
This is a harder question for the art history because most instructors are NOT in the field for practical purposes (i.e. they do it for the love, not the money). Many instructors may feel their topic is interesting enough for everyone to pay attention. Of course, if that were true, I would be in hot demand for my expertise in Distributed Morphology.
Yet I wasn’t ready to give up on the “relevance” angle so for the audience desiring “authentic application” (or the “Why is this relevant?”) I give you:

The cocktail party conversation

You may not be interested, but if you enter corporate America, you will encounter someone who has been to France/Spain/Germany/Britain and has absolutely fallen in love with the cathedral. And why not…they are beautiful buildings. Isn’t it nice to have something to share in a conversation emergency?
This may sound silly, but this is exactly why my grandmother taught her daughters to watch football…so they could talk with the menfolk in their lives. In this case, it actually means I get to enjoy Penn State Bowl games, and yes it has been a conversation helper in some cases.
Instructional Tip: Provide amusing stories for your students to relay!

Appreciate Western Cultural History

There was a time in Western History when most of the populace were practicing Christians and the cathedral was a key social institution. Understanding the usage for the parts of cathedral helps you understand how Western societies were structured in that time period. For students from rural Pennsylvania, there are likely to be differences from how a cathedral operated to how a small-town church (often Protestant) now operates.
On a related note, the innovation of the cathedral is a reminder that the Middle Ages weren’t just a “dark and grungy” period before the Renaissance. The medieval era was a time when major Western social and technological innovations occurred which are still in place today.
Of course this assumes you explain these mechanisms and don’t just provide a list of words to memorize.

You may visit a Cathedral…Someday

We associate cathedrals with Europe, but they do exist in many U.S. cities on the East Coast (although some may have slight differences in form). And you may actually get to go to Europe!
Instructional Tip: Maybe it’s worth examining local cathedrals as “later innovations”.

Building Your Own

In an ideal world of Problem Based (PBL) learning, you would probably get a chance to build or design your cathedral, maybe within a few specific parameters. I know this would probably get the most attention from me. You probably would be learning your parts that way.
However, instructional technologists still haven’t solved the ways to build interactive applications quickly or to convince all instructors of their effectiveness. And it certainly is a more time consuming process.
Interestingly, this isn’t “authentic” per se (unless we admit that many of us do build shrines of some sort at home), but it is kinesthetic and “active”. Sometimes learning should just be fun I think!

What About Beauty?

This is interesting because if you think an item in beautiful, you may be more interested in learning more about it. It does enhance intrinsic motivation. However, there are times when judgments on “beauty” differ.
I do love the cathedral, but I’m not necessarily a fan of other art forms others consider “classics.” Saying “It’s beautiful” won’t motivate me to learn about the “ugly” art forms. However, I do find that learning about the culture behind the art may make it more “tolerabe” and “interesting”…so “relevance” may actually enhance “beauty” in this case.