Wednesday, November 08, 2006

 

Testing and Tribulations

I'm sitting here grading exams, and thinking about exams in general. As some of you know, in most of my classes I no longer give the traditional end-of-the-quarter final. My reasoning for this is that it tests a skill (memorizing a huge number of facts for four days) that has very little application in the real world. I have tried, where appropriate, to shift to a series of exercises including a take-home exam that is going to require realistic legal analysis.

Many, if not most, of my students go on to be prosecutors, and I am often struck by the fact that those hiring for prosecutor positions often tell me that they don't care much about grades-- and even that they find that many of the best prosecutors are those well down in their law school class. Is this an indication of how poorly traditional testing correlates to the skills we are trying to create in a professional school?

Comments:
I think BLS testing is extremely practical in terms of the real world skills it emphasizes.

For example:

Generally we have one test for each class, approximately three months of work is evaluated on three hours of responses.

Whatever is going on in your life- a recent death/birth, sickness, anger, joy — must not cloud your ability to recall and report the relevant information in correct IRAC format.

Then after a three hour test you have to wait six weeks for any feedback.

Then after you receive your grade your satisfaction with that grade is highly contingent upon seeing how you did in relation to the other members of your class.

To me that is almost an exact replica of the practice of law.
 
I am not a fan of the testing here. I have found I am good at analyzing and horrible at memorizing. That puts me in the B to B- range. Granted that's not terrible, but the testing would be better if memorizing wasn't a key element. I forget almost all of the information I cram in there 2 hours after I complete the exam. I welcome a take=home exam with a harder analysis.
 
I understand all of the anti-memorization sentiment and would love nothing more than a good take-home exam, however, there are some things we should be required to know as lawyers.

For exapmle, when a client comes into your office with a problem, they don't want to hear "Well this sounds like a tort, but I'm not sure". Clients expect you to have a pretty good idea about the law. What you should be able to provide is "Well Client, this sounds like a products liability case and in Texas you can/cannot probably recover because of XYZ. I'll have to do some research about your specific situation to have a better idea..."
 
Of course, I teach in a completely different field, literary analysis, but I have the same take on testing as the Razor. Although I think there is important information that a student should know, I want to know if he/she can take a text they have never seen and analyse it, interpret it, get to the bottom of things.

Sure it's great to know authors' names and the works they wrote, but if that is all you can tell me, you flunk. Analysis, three or four levels deep, is the key to success in my field. Can you recognize the larger patterns of dialectics, rhetoric, and discourse that appear in a work?

This is what is important to me, not whether your frat buddy has an old test from the course that you can use to cram with. That's a dead end and the student has cheated him or herself out of their own education.
 
8:49-- You should be able to do that regardless of the testing method at the end of the class where you learned that the person probably can recover because of XYZ. If you think you will retain knowledge to the level that makes a difference in grading on a traditional exam, you have a truly remarkable long-term memory, and that's very rare.

The problem is that to create sufficient differentiation on a memorization test, you need to test on minutiae that is very hard to retain long-term. In other words, the difference between a B and C may be the stuff that the person in practice will have to (and should) look up.

Look, I do give memorization tests, in PR. That is in part because there is a real-life and relevant memorization test (the MPRE) in that field. Criminal Practice is not the same.
 
I don't like end-of-term tests in most professional programs, unless the skills being taught can be adequately synthesized and practiced in 2 hours. That's, by and large, not the case in my field, either.
 
7:27 am:

I used to feel the same. Then I realized I'd have to pay for the additional credit hours. Of course, it's been a while since I've been there, and they may have gone to the flat rate for up to 18 hours or some such thing.

Prof. Osler:

I agree with your reasoning about testing useful skills. That's why I think Big Trial is the best test at Baylor Law.

-B
 
B-

you wouldnt have to pay more. you need 126 hours to graduate, and you pay for 126 hours. the pc quarter itself would be more expensive bc youd be paying for 16 hours instead of 12 for taht particular quarter, but then you would have 1-2 fewer classes to take in subsequent quarters to reach your 126 hours (and in turn 1-2 fewer classes to pay for down the road).
 
That makes sense, but the 1-2 classes you'd lose would be electives, which are very sparse to begin with.

-B
 
how come most of my friends at other law schools get to take outlines in with them? ut, columbia, texas tech, soutwest texas etc.

is there ever a chance that in practice you will never get to consult the law before you give an answer
 
2:22--

While I agree with you about practice (as I said in the post), on the bar exam you aren't allowed an outline, and that may be the best defense of the memorization type of test.

(Southwest Texas has a law school?)
 
I was just talking about this issue with a friend of mine. I don't think that the prototypical 3-hour, memorize-the-entire-UCC type of exam is very practical, since doing any sort of legal work without consulting with others or cracking the books would lead to malpractice in about, oh, 10 minutes. Open books, more time, tougher questions... THAT would be more practical. As a matter of fact, the best grade I've gotten in law school was on the take home exam in one of Prof. Osler's classes.

That said, I do think that there are some classes where this sort of test is proper. Specifically, I agree that PR does require that lawyers memorize and actually internalize the Rules. I would add Evidence, simply because this is the only set of rules that I can think of off the top of my head that actually require strict memorization and the ability to spit out those rules at the drop of a hat. You can't really say, "Hey, Your Honor, can you take a teensy time out while I crack open the Rules of Evidence and pore over the table of contents? I know there was something objectionable there... just give me a moment to find it!" On the other hand, you can't just shrug and make an "aw, shucks" motion with your hands when you can't remember the rule. THAT is also malpractice.

So Civil Procedure? Since I don't believe I would ever fill in my deadlines calendar without double-checking all pertinent rules, memorizing that I have 15 days in this situation and 30 days in this other one seems silly. Since I wouldn't run around filing pleadings in courts without double-checking that I'm in the court of proper jurisdiction, memorizing the amount-in-controversy jurisdiction for a Tarrant County court seems silly.

And don't even get me started on FedTax....
 
Speaking of Tribulations, you don't know how excited I am to have been assigned marijuana for Drug Day in Crim Prac. When you were reading meth and cocaine names, I was willing my name down the list to something fun like acid or paint fumes. But I'll settle for Mary Jane. Call on me at your peril--or don't, I might hijack the class anyway in a fit of REEFER MADNESS!!!--and be prepared for the Phish-phueled Rastacular!

And, um, someone bring munchies.
 
I am way behind the times with this, but I've been so-oo-oooo busy reading for this craaaazy Criminal class I have...and researching meth...how often do you get to have a note on your to-do list that says "research meth"? No, on second thought, maybe I don't want to know.

Anyway, I think some of the best tests, in terms of seeing what we actually know, was in classes like FedTax and BizOrg, where we could bring our statutes in. You still had to memorize the big picture stuff, but you could look up the tiny little details (wait, is that a majority of stock entitled to vote, or of the stock represented at the meeting?). You can't look up every single thing, or you'd never finish the test in time, and you can't really use it as a crutch completely, or you'll run out of time, but it lets you focus your study time on big concepts and not sweat the small stuff as much. I have no idea if that's realistic to practice or not, but it seems like it would be.

Which is not to say that I liked those tests. I did not. But I think they were probably more accurate as to what I actually knew (hence why I didn't like them).
 
I think there's another important issue that we're all missing but that law school exams treat very well. Although you wouldn't fill out a deadlines sheet or draft a petition without checking all the rules, you will be stuck in court from time to time and not have any time to research the answer to a question the judge asks. The skill that I honed most in studying for law school exams is not what is on the test, but the process of figuring out likely questions and how to answer them, figuring out what I needed to memorize and what I was unfamiliar with, and even memorizing a "checklist" of issues I need to remember to hit on. While the law school exam system may be worthless for other reasons, let's give credit where credit is due: in a sense, taking an in-class, no-outline exam is similar to a very broad hearing in court. You know what the general topic is and you know your way of hitting it, but you never know what your opponent or the judge is going to throw out.
 
Post a Comment



<< Home

This page is powered by Blogger. Isn't yours?

#