Department of Mathematics and Statistics
University of Nebraska - Lincoln

How do you teach a reform calculus course, with all the discussion and problem-solving work that entails, and at the same time ensure your students have a strong enough grasp of core algebraic computations to satisfy you, your colleagues, and your client disciplines? Group-work, open-ended problems and classroom discussion, which are the hallmarks of reform calculus courses, are extremely time-consuming. The concern heard most about calculus reform is not that we shouldn't emphasize these aspects, but that the time they demand squeezes out time the students need to learn how to do basic computations of integrals and derivatives.

One partial solution a number of schools are adopting is to use "Gateway Exams” to test computational skills. As we use the term, a Gateway Exam is a competency test which students are required to pass at some specified level of proficiency in order to pass the course and proceed to subsequent courses. Our model of a Gateway Exam is inspired by the tests used at the University of Michigan.

Our goal was to meet head-on the legitimate concern that some students might pass a course with, say, a C grade based entirely on partial credit and group work, and without demonstrating thorough understanding of any topics in the course. In setting a Gateway Exam with a high standard, one must assume that many students will not pass initially, and will need an opportunity to retake the exam. The process of taking and retaking the Gateway Exam will then force many students to assume responsibility for learning material which they might otherwise have neglected in the belief that partial credit from other sections of the course would pull up their average. Thus a Gateway Exam is supposed to focus on core competencies of the course which no one should pass the course without knowing.

At the University of Nebraska - Lincoln (UNL), we have been teaching a reformed calculus course using the CCH materials since 1994. By 1996 we were also hearing concerns expressed by colleagues that many students did not have adequate computational skills. Thus, in Fall 1996, we decided to introduce Gateway Exams to test our students’ ability to find integrals and derivatives.

The argument for giving these tests by machine was clear: Early in Fall 1996 we experimented with paper Gateway Exams for our Calculus II class, and found that the job of setting and grading multiple retakes of paper tests for a class of over 300 students placed a prohibitive burden on faculty and GTA time, and essentially monopolized our Math Resource Center throughout the period of the test. At UNL in each fall semester we teach over 600 students in Calculus I and at least 250 in Calculus II. In the spring the numbers change somewhat, but the total is always above 700 students. If we were to offer tests with multiple retakes to these classes then we had to have a way of setting and grading the tests automatically.

The system we created offers students essentially unlimited retakes, together with unlimited opportunity to practice tests on their own. The tests are delivered over the web and, in principle, can be taken from any computer with an Internet connection and a web-browser. In practice, there is a natural concern about honesty that makes us insist that when students want to take a test for credit they must come to our computer lab where we can proctor the tests and verify students' identities. Of course there is no such restriction on practice tests, and students are encouraged to work practice tests from home, the dorms, or computer labs around campus.

It is hard to overstate the importance of the fact that students can practice our Gateway Exams on their own. It is our sense that students typically prepare at first as they would for any exam. We still offer one chance to take the Gateway Exam in class, on paper, which the very good students pass at once. However a large number cannot initially meet the high standard; the second author found only 31% of his class passed the exam on integration the first time it was given.

Once they realize the expectation is real and the standard is high, students start practicing the exam on the web. The fact that practice tests are drawn from the same database as the tests given for credit, so that students know they are practicing on a test which could have been the test they were given for credit, seems to inspire the students in a way that simply having a long list of problems and answers from a book does not. (In the example reported above, all but one of 88 students met the standard within 4 weeks.)

The software that we use is a self-contained test-server written by the first author. It sets and grades tests which it creates from databases of questions and answers written by faculty and graduate students in our department. Indeed a feature of the design is that the questions and answers are text files which can be written in TeX, Microsoft Word, or any text editor by people who have no involvement in the software design.

Tests include both multiple choice and free-form answers. Free-form questions ask students to enter a formula for the answer in the same format as they are used to doing with their graphing calculators and the system then parses the expression so as to recognize when two formulas express the same function in different ways. As soon as a test is completed, the machine grades it, gives the student immediate feedback on his/her grade, and lists solutions to questions the student got wrong.

In order to make sure the exam is done closed-book and without the use of a sophisticated calculator, and that the person taking the exam is who they say they are, students take the on-line test for credit in our computer lab. After a student has finished a test and before it is graded, the student calls over the lab attendant, who serves as proctor for the exams. The attendant checks the student's ID and, assuming everything is satisfactory, the attendant authorizes the test for grading using his/her own secret password.

The lab we use for the tests has 20 computers, arranged in rows. One row of 7 computers is reserved exclusively for the tests, with students taking exams allowed to spill over to the remaining machines if no one is using them for other classes. Since we give the first version of the test in class, the lab is used only for retakes, and since the lab is open over 50 hours each week, we have over 1000 machine-hours available per week. We have found that has been enough to serve classes as large at 850 (when both Calculus I and Precalculus classes have been taking tests at the same time) on a first-come-first-served basis.

Having practice tests available over the web and the ability to give multiple retakes enable us to hold the students to a very high standard of success. In all of our on-line tests, the passing score is between 80% and 85%, and students receive credit only for passing, with no partial credit on individual problems or for scoring below the passing grade. Most of our faculty have stopped just short of making our on-line tests quite as stringent as the Gateway model described above - success in the on-line tests is not an absolute requirement for success in the course, although our grading schemes count the test as worth approximately one letter grade. Typically one or two students may prove they can pass the course (typically with a grade of D) without passing the Gateway Exam, but most students treat passing the Gateway Exam as a course requirement. However this approach appears more reasonable to students because it has an award for those who pass, rather than a punishment for those who do not pass.

In the authors' courses, 85% of the students have eventually passed the Gateway Exam and if we restrict our attention to those students who continue in the course to the point of taking the final exam, the pass rate on the Gateway Exam is 92%. We're also seeing that success at the Gateway is a very good predictor for success in the course; less than 1% of our students passed the Gateway and then failed the course. Also, only 3% passed the course without passing the Gateway Exam, usually with a D or D+ for a final grade. There's also evidence of the benefits of multiple retakes; the success rate at the Gateway among those who took the test four or more times was almost identical to the success rate of those who only needed between one and three retakes.

It’s important to emphasize, though, that the Gateway Exams we give are one part of the assessment that takes place within the context of a reformed calculus course. Students in this course are also assessed on homework, quizzes, group writing projects, and written tests which explore conceptual and open-ended questions. Adding a Gateway Exam on core computational skills to our repertoire of testing has served our calculus program in two complementary ways. It enables us to simultaneously meet the concerns of those who worry about students losing grasp of algebraic skills, and at the same time, by allowing students to practice and improve their performance on their own over the web, we protect the human interaction of the classroom for exploration of the conceptual aspects of calculus that are beyond the reach of the computer.