AP Physics Comparability Study.ppt

Helps set cut points for AP grades College students’ responses scored as AP students’ would be ETS statisticians analyze the data, to see how college grades of A, B etc, compare to AP grades of 5, 4, etc. AP grades are adjusted as appropriate, based on data and judgment of Chief Reader. Behind that judgment are curriculum surveys used to set course outline and exam specs, input of committee as exam questions are developed, and information about student performance from the Reading leadership. Goal is that 4/5 cut is at college mid-A’s, 3/4 cut at mid-B’s etc. Provides some information to colleges that relates their students’ performance to the AP students and other participating colleges College instructors receive data that allows them to compare their students as a group to the AP population and the total group of college students participating. Invitations sent to colleges and universities receiving 20 or more AP Physics B 2006 exam grades Letter sent to department chairs, college AP Physics Readers and present/past Development Committee members Instructors gave tests at appropriate time near the end of their semester courses Not all colleges gave every study test Physics B first semester had 11 colleges, Physics B second semester had 4 (all these also gave first semester) Physics C Mechanics and EM each had 8 Desire 20 colleges minimum – since study was added late in summer, many colleges couldn’t work it into their schedules 2007 exam analysis done as usual, using standard equating performed each year, yielding cut scores based on carryover of same score scale Data from colleges analyzed – through method similar to regular equating, a set of “new” cut scores for 2006 generated based on college standards Chief Reader, ETS staff, and CB staff review individual college data to identify any “suspect” data, e.g. Students not motivated Test content not good match to course content Low correlation between student course grade and stud