аЯрЁБс>ўџ =?ўџџџ>џџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџьЅС#` №ПŽ!bjbjЁЁ 4,УУŽџџџџџџЄЈЈЈЈЈЈЈМ    ДМx Жддддддддї љ љ љ љ љ љ $.h–А Ј' дд' '  ЈЈдд2 — — — ' ЈдЈдї — ' ї — — ЈЈ— дШ €ћLYШ 1 :— ї H 0x — Fk "F— FЈ— `диЌ š— F |Т eддд   дддx ' ' ' ' МММф МММ МММЈЈЈЈЈЈџџџџ City of Hope Graduate School of Biological Sciences The PhD program at City of Hope Graduate School of Biological Sciences trains students – a group of about 50 – to carry out highly specialized biomedical research. In 2005, the program underwent a Capacity and Preparatory Review. The action letter found much to commend, but it also called for significant progress on assessment and the demonstration of educational effectiveness by the time of the EER in fall 2007. City of Hope took that challenge seriously. The ALO and dean of the Graduate School sought out literature on assessment at the graduate level, engaged a consultant, refined a set of four program-level learning objectives that had been developed earlier for the CPR, created an assessment map, and drafted rubrics for the objectives. Working with faculty, they decided to use existing processes as the venues for assessment efforts. One of those venues was the Current Topics in Biology course. The learning objectives of the course are to develop 1) critical thinking regarding scientific literature, research data, and experimental design (learning objectives 1.2, 2.1); 2) critical scientific writing (objectives 2.3, 4.3); and 3) critical debating of scientific ideas (objective 4.4.). Coming at the end of the first year of doctoral study, the CTB course confronts students with a different topic each week. It requires them to read a set of research papers on the topic, propose a hypothesis in response to the work in the papers, and design an experiment to test the hypothesis. Presenting their ideas in written form gives students the opportunity to develop professional writing skills. The writing assignments are graded by the instructor responsible for that week’s topic. CTB is the last formal course students take before beginning full-time lab work and ultimately choosing a topic and advisor for their own research. Thus it plays a critical role in students’ training. Faculty had noted, however, that the course posed considerable challenges and students did not always emerge ready for the lab. As a first step, consultants were hired who observed several class meetings and then designed a student survey. The survey revealed problems with rotating instructors, lack of continuity, uneven grading practices, and excessive reading assignments. But the survey also showed that students valued the overarching purpose of the course, which is to teach them how to synthesize the work of others and think for themselves. A three-person CTB Assessment Committee was formed, and it worked with CTB faculty to study the course and find ways to strengthen it. Using a rubric to examine student work, the group discovered that there were several areas for improvement. The foremost weakness – flawed experimental approaches—was consistent with a finding coming out of a parallel study of qualifying examinations. The program concluded that they had been taking students’ knowledge of experimental design – and of the pros and cons of particular laboratory techniques – for granted, since more than half of entering student come with significant laboratory experience. Faculty recognized that this is one of the most difficult and sophisticated skills to master, and the one that is most alien to students at this early stage in their professional lives, so it was actually not surprising that this was an area that needed additional attention. In its analysis of findings, the CTB Assessment Committee noted the need to improve students’ overall content knowledge to a level sufficient to understand and reflect on the literature they were reading; to address the variability resulting from the diversity of students’ educational backgrounds; and to strengthen writing skills. They are still thinking about ways to respond to some of these findings, and they are considering additional ways to assess the course, perhaps through the use of different instruments. The committee did make two very specific recommendations related to experimental design, however. First, when their new teaching laboratory is completed in 2009, they recommend introduction of a course in laboratory techniques with a focus on understanding experimental approaches and their limitations. They anticipate that this will require faculty development, as well. Second, the learning aspects of first-year lab rotations should be enhanced. Beyond acquainting students with various projects and researchers, the rotations should become an opportunity to learn laboratory techniques and experimental design. Rotations can be structured to ensure that students are exposed to a full range of techniques and approaches before they make a decision about their own specialization. Students’ rotation reports and faculty evaluations of rotation performance can also be modified to include questions about students’ understanding of experimental design. For the EER, City of Hope submitted a report on the CTB course as one of its essays; the report mirrored the form of a scientific article, with multiple authors and sections titled “Background,” “Goals,” “Assessment Tools,” “Assessment,” “Recommendations,” and “Progress Toward Course Improvement.” There were also extensive appendices with data displays. In addition, City of Hope carried out assessment studies of two other milestones in the PhD program: qualifying examinations and thesis committee meetings. Finally, the self-review addressed concerns raised in the annual Student Survey and Faculty Survey. The conclusion of the EER report states that the Graduate School finds its training of graduate students is basically sound. Nevertheless, the assessment studies have surfaced areas needing improvement, particularly experimental design, data analysis, and writing. The self-studies have fostered faculty development and revealed the need for more. The assessment studies have made the lines of responsibility for learning clearer, engaged students, and produced new data. The school plans to enhance the role of the Assessment Committee, and the committee has designed a 5-year assessment plan that will continue to focus on the learning objectives and build on the findings of the current studies. Questions for discussion: 1. How does the City of Hope story align with your understanding of assessment? Not at all? Somewhat? Fully? Explain your reasons. 2. What could City of Hope have done differently or better? 3. Would this model work at your institution? Why or why not? (BDW, 1/15/08) 34|‚А е  Ь ј ˆ ‰ Ш № w | )/?Eiv„…†но*/дё (ˆUa~ЮЯЋВЎЗьѓRZdfgФгќ*0mqˆќ4S1й-.Z“ЬЮ/їяычыуыплпыпзпупчпзпзпупгзЯЫЯлЯлгчгчгчгЯгчгЧЫЧЫЧучугЫУЫУПУЫУЫУЫУЫУЫУзУПУчУh‰rЎhЁUХhХ ehтP]hъ]Fh3Uh˜6uhFQ3hѓ;&h3;Яh :hс5VhFQ3h=eі5hFQ3hс5V5J45з и ‰ Š я № опЯаghpq./”•Q R l m № ё ђ ѓ їѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕѕ$a$gdс5VŽ!ў/Баё§P Q R j  я № ђ є 4!5!t!Ž!ќјќјќє№шфрфрјрјрјhFQ3hАhFQ3hА>*h˜6uhЁUХh :h˜GRѓ є ѕ 1!2!3!4!5!6!t!u!Ž!§§§§§§§§§§№ „ф§„^„ф§`„gd : 21h:p :Аа/ Ар=!Аь"А„#а$ %ААаАа а†œ@@ёџ@ NormalCJ_HaJmH sH tH DAђџЁD Default Paragraph FontViѓџГV  Table Normal :V і4ж4ж laі (kєџС(No List Ž,џџџџ45зи‰Šя№опЯ а g h pq./”•QRlm№ёђѓєѕ123456tu˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€˜0€€/Ž!ѓ Ž!Ž!џџ ВѕЌG7 !ВѕьA7 "ВѕЌO7 #ВѕlN7 $ВѕlO7 %Вѕ,A7 &ВѕЌ@7 'Вѕ,O7 (Вѕь@7 )Вѕ,M7 *ВѕlH7 +Вѕ,K7 ,ВѕЌA7 -ВѕЌM7 .ВѕьL7  ррЈЈЦЦЯ……        ффЌЌЮе剉  8 *€urn:schemas-microsoft-com:office:smarttags€City€= *€urn:schemas-microsoft-com:office:smarttags €PlaceType€=*€urn:schemas-microsoft-com:office:smarttags €PlaceName€9*€urn:schemas-microsoft-com:office:smarttags€place€ Ьїn     НШЩвгйt333mНкх3UАѓ;&FQ3 :ъ]F˜GRЅNTс5VтP]Х e:Wj˜6u™Ž‰rЎЁUХ3;Я=eіџ@€шыnŽ@џџUnknownџџџџџџџџџџџџG‡z €џTimes New Roman5€Symbol3& ‡z €џArial"ёˆ№аhG‹С†G‹С†аО .аО .$№ьаДД42ƒ№мHX)№џ?фџџџџџџџџџџџџџџџџџџџџџс5V2џџ3City of Hope Graduate School of Biological Sciencesbwright jkotovskyўџр…ŸђљOhЋ‘+'Гй0œ˜др№ќ ,8 X d p|„Œ”ф4City of Hope Graduate School of Biological SciencesbwrightNormal jkotovsky2Microsoft Office Word@FУ#@2†цLYШ@2†цLYШаОўџеЭеœ.“—+,љЎ0  hp€ˆ˜  ЈАИ Р фWASC. Ф 4City of Hope Graduate School of Biological Sciences Title ўџџџўџџџ !"#$%&'ўџџџ)*+,-./ўџџџ1234567ўџџџ§џџџ:ўџџџўџџџўџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџRoot Entryџџџџџџџџ РFPNћLYШ<€Data џџџџџџџџџџџџ1TableџџџџFWordDocumentџџџџ4,SummaryInformation(џџџџџџџџџџџџ(DocumentSummaryInformation8џџџџџџџџ0CompObjџџџџџџџџџџџџqџџџџџџџџџџџџўџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџўџ џџџџ РFMicrosoft Office Word Document MSWordDocWord.Document.8є9ВqRoot Entryџџџџџџџџ РF@А\JК\ШB@Data џџџџџџџџџџџџ1TableџџџџFWordDocumentџџџџ4, ўџџџўџџџ !"#$%&'ўџџџ)*+,-./ўџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџA§џџџўџџџўџџџўџџџ@џџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџўџџџўџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџ,А SummaryInformation(џџџџџџџџџџџџ(DocumentSummaryInformation8џџџџџџџџ˜CompObjџџџџџџџџџџџџqџџџџџџџџџџџџўџ џџџџ РFMicrosoft Office Word Document MSWordDocWord.Document.8є9ВqўџеЭеœ.“—+,љЎDеЭеœ.“—+,љЎd  hp€ˆ˜  ЈАИ Р фWASC. Ф 4City of Hope Graduate School of Biological Sciences Title4 $€