The LPE remains the most important tool available. In addition to tests already in place for Arabic, Chinese, Hmong, Italian, Japanese, Russian, and a second version of the Spanish LPE Reading and Listening sections, other new assessments are in progress. Development is underway for Finnish, Korean, Somali, and Swahili LPE's this semester. A new form of the German Reading LPE will be piloted in November.
Beyond the LPE, the following new assessments have been piloted, will be piloted or are now actively administered to students: the Spanish Simulated Oral Proficiency Interview (SOPI), the Individual Language Assessment (ILA) and several Self-Assessment instruments.
The Spanish SOPI
A SOPI is a computerized version of an Oral Proficiency Interview (OPI) administered in a Digital Language Lab (DiLL) to a class of students simultaneously. The primary advantage of a SOPI over a traditional OPI is that the time required to administer the test is significantly reduced. It would take at least six hours to administer OPIs to a class of 24 students; with a SOPI, this can be done in less than one class period. Of course, all of those exams still need to be rated!
"In delivering an interview via computer, the SOPI may shift the center of power more toward the students; they may feel more ownership of their half of the (simulated) interaction, and thus feel more confident about speaking."
-Gabriela Sweet, assessment developer and lead trainer
Students are provided with a real-life context to speak in Spanish, recording their responses in the lab using DiLL. Student feedback was generally positive after the February pre-pilot. Students commented that the SOPI is more efficient and less stressful and that they enjoyed using the SOPI in lieu of the traditional face-to-face format.
The SOPI's delivery, in capturing extended student speech, reduces the possibility of a difficult-to-rate interview, which can occur in face-to-face interviews when learners may not get the opportunities they need to demonstrate the full range of their language abilities. Developers hope to use the SOPI assessment with College in the Schools (CiS) students in the future, and it is possible to adapt this assessment for other languages as the prompts are in English.
As with the OPI, SOPI tasks are designed to target a wide range of linguistic functions and real-world topics at the student's target level of proficiency. The SOPI is also more standardized compared to the OPI because it systematically facilitates a ratable sample of student speech. Students naturally tend to speak in complete sentences in the SOPI, which may not happen during the live interview when they can appropriately respond in sentence fragments. Since the SOPI response is at sentence-level, it may be easier for raters to analyze.
A six-person team carried out the SOPI pilot. This team included Spanish 1004 coordinator Sara Mack, Spanish 1004 instructors Marilena Mattos and Stephanie Hernández, Spanish Testing Coordinator Joanne Peltonen, Language Center Testing Development Coordinator Gabriela Sweet and Language Center Technical Coordinator Diane Rackowski. Additional input and support was provided by Language Center Director Dan Soneson.
The ILA
Did you know the Testing Program now provides assessment in languages from Amharic to Zulu?
CLA students who have achieved proficiency in a language not offered at the University of Minnesota have a new way to demonstrate their proficiency and complete the second language requirement. The ILA is a writing and speaking test that is adaptable for any modern language. It tests to the intermediate level, which is the level expected after two years of university language study. The test is rated by native or near-native speakers of the particular language who are trained and guided by Language Testing Program staff.
Since January 2013, a total of 32 students have been approved to take ILAs in 16 languages from around the world, including major languages spoken by hundreds of millions of people worldwide, but rare on our campus, and dialects specific to a particular village. The most common languages assessed are Vietnamese and Oromo.
For many languages, it can be challenging to find a qualified rater. Most raters are affiliated with the University as faculty, staff, graduate students, or former students. They are almost always native speakers with some prior linguistic or educational experience. Some of the raters have few other opportunities to use their language skills professionally, although they may use the language in their daily life.
Stephanie Treat, who has assisted with the hiring of most raters, said:
"We often get requests to rate languages that I've never heard of or know little about. We've had fun researching languages and scouring campus and beyond to find potential raters. The University of Minnesota is a global community, and with persistence, we can usually find someone currently connected to campus who is pleased to share his or her expertise."
In order to conform to the CLA mandate for proficiency in a second language, ILA raters are trained using guidelines that align with those used to evaluate student performance in languages that are taught at the University of Minnesota and that have an LPE. Before ILA raters begin the evaluation process, they are made familiar with both the instrument and the target level for production. The rater-training process is very hands-on; criteria are analyzed and then applied. The rater works closely with the rater trainer. Raters are guided through the process using a rater-training module.
The Language Testing Program plans to create a video to facilitate rater training, as well as refresher training, for raters who have completed evaluations at an earlier date but would need a quick refamiliarization to ensure that they apply the criteria in the same way to a new student's ILA. "We've been fortunate to work with some very talented people," assessment developer and lead trainer Gabriela Sweet said.
"It's fascinating to work with someone on a language with which you, the trainer, have very little experience, to see them point out clearly how students demonstrate the target levels. It has been a wonderful experience for us, in the Testing Program, to see how students in a variety of languages are able to show their proficiency in alignment with the College of Liberal Arts student learning objectives. I think it's also a privilege to be able to learn from colleagues in diverse languages; in hearing what students say and reading what they write in the ILAs we begin to see that the world is, in some ways, quite small... we're all working toward many of the same goals."
- Gabriela Sweet
Students interested in completing the CLA second language requirement via ILA exam should begin by contacting their advisor. CLA Student Services approves requests first, and then students contact the Language Testing Program to schedule an exam. There is a $30.00 fee assessed to assist with the cost of rater compensation. Students who pass both sections of the exam complete the CLA second language requirement.
Self-Assessment Instruments
Self-Assessment instruments help students become more aware of their own language development and describe their level of proficiency. This type of instrument requires little time to administer and students can take it from home at their convenience. Students can take the same assessment more than once and track their language development over time.
Two Self-Assessments for Spanish are currently in development: one intended for Spanish 1004 students and another which assesses higher-level proficiency for students pursuing the Certificate of Advanced-Level Proficiency. Developers are also creating new self-assessments for German, Italian, and French.
Students can use these self-assessments for feedback on performance in a language and to better prepare themselves before taking the LPE or other exam. In a pilot study in Fall 2013, many students found the self-assessments helpful and noted that routine self-assessment would be helpful in the future. The test developers will be very pleased to share more information about these instruments once they have been piloted and results have been analyzed.
SOPI, ILA, Self-Assessments
Developers in the Language Testing Program and the academic departments have been hard at work developing and improving language assessment instruments to create a more personalized, modern, and enjoyable testing experience for students. Undergraduate students themselves have played an important role by piloting these assessments and providing valuable feedback to developers. These new tests are designed to accurately reflect students' abilities and to provide them with information they can use as they continue to develop their proficiency.
| Type of Assessment | Benefits / Improvements | Development Timeframe |
|---|---|---|
| New Language Proficiency Exams (LPE's) | Korean and Somali were added as new LPE exam options. Adding these languages provides students with more possibilities in assessing language proficiency. | Korean: Will pilot Reading in Spring. Listening & Writing are ready to go! Somali: Will pilot Reading, Listening, and likely Writing this Spring |
| New German Reading Test | Developed another form in addition to Form A. Form B is up-to-date; the readings from Form A were 20 years old. Better quality photos were also added. | The Reading test will be piloted this Fall semester |
| Simulated Oral Proficiency Interview (SOPI) | Spanish 1004: Extended time limit for thinking and speaking. Bigger, better resolution photos. | Piloted in Spring 2013 |
| Individualized Language Assessment (ILA) | One format for all languages as a test of Speaking and Writing. Allows students flexibility in composing responses about real-world situations. | The test has been administered to 16 students with many more expected in the future. The long-term goal is to develop multiple versions of the test. |
| Self-Assessment Instruments | Spanish 1004: Students can demonstrate their second language proficiency before taking final proficiency tests Spanish Advanced Level: To help students determine their proficiency level. Intended for students pursuing the Certificate of Advanced-Level Proficiency. | Spanish 1004: Pilot was completed in Fall 2013 Spanish Advanced Level: Piloted with Spanish students December 11, 2013 German: Pilot 5 sections in Spring 2014 Italian: Pilot in two section Spring 2014 French: Pilot in eight sections Spring 2014 |
Formal and informal assessment of students' spoken language is used by most language departments, commonly as one-on-one interviews with the instructor. This can put a huge strain on instructors as they are responsible for not only proctoring and grading each interview, but also: coordinating interviews with each student (which usually takes up several class dates), finding equipment to record the interviews, and figuring out how to access the recordings. Fortunately, the Language Center computer classrooms offer a variety of alternatives to one-on-one oral skills assessment!