Program Interpretations and Conclusions:

The Earth Science Teaching Program has a formative and summative assessment system comprised of several elements: 1) a performance- based, standards-aligned electronic portfolio, 2) entry and exit surveys, and 3) WEST-E content examination.

The Earth Science Teaching Portfolio is based on a common template collaboratively designed and constructed by all members of the Science Education Program, with additional insight provided by K-12 teachers. The conceptual framework is based on the latest scientific research on inquiry and how people learn (National Research Council, 2005). In particular, program assessment focuses on: 1) determining student preconceptions, 2) engaging candidates in authentic inquiry and investigation, 3) developing and applying robust content knowledge, and 4) promoting meta-cognitive awareness of learning process and critical thinking.

The main portfolio elements, or dimensions, were conceived through a consensus process by examining commonalities in three sets of professional standards; National Science Education Standards for Teaching, National Science Teacher Association Standards, and the Washington Competencies for Earth and Space Science. The common dimensions include: 1) Inquiry and Nature of Science; 2) Teaching; 3) Learning Environments; 4) Assessment and Evaluation, 5) Relevance, and 6) Professional Growth. A distinct dimension for Content in the Earth Science Teaching Portfolio is further subdivided into the major disciplinary themes in earth science, and include: a) Physical Geology, b) Historical Geology, c) Climate and Meteorology, d) Astronomy and Space Science, e) Environmental Geology, and f) Field Methods and Instrumentation.

Candidates must demonstrate knowledge, skills, and disposition proficiency by providing tangible, verifiable evidence chosen from coursework, research and field work, and related disciplinary and educational experiences. Each portfolio dimension contains several strands designed to focus student effort and evidential support. For example, for the Inquiry and Nature of Science dimension, students must provide separate evidence for inquiry, nature of science, and independent research. In an effort to promote higher order thinking and meta-cognitive awareness, for most of the portfolio artifacts students are required to supply evidence they deem suitable rather than evidence prescribed by faculty. Candidates must also justify their choice of evidence and connect these to progress in meeting professional standards through a reflection that accompanies each portfolio dimension. Candidates are required to demonstrate minimum proficiency for each portfolio dimension in order to be allowed to student teach. Compliance is accomplished via an advising hold that may only be removed by a Science Education faculty member.

When a candidate enters the Earth Science Teaching Program, they must complete an entry survey that includes basic demographics as well as written items that assess candidate disposition. These results are then embedded within the Earth Science Teaching Portfolio. At the completion of the Earth Science Teaching Program, each candidate must also complete an exit survey whereby they evaluate program effectiveness, including all education courses in the Professional Education Sequence as well as Science Education courses. Experiences that were particularly useful are described, as are met and unmet candidate expectations. Exit and entry survey results are subsequently compared and a comparative reflection completed.

Content knowledge is assessed via evidence provided by candidates in the Earth Science Teaching Portfolio as well a content examination. The WEST-E in Earth and Space Science is required of all teaching candidates. Total and component scores are posted within the Content dimension of the Earth Science Teaching Portfolio.

Collectively, these assessment procedures are used to evaluate candidate competency and to ensure knowledge, skill, and disposition proficiency for future earth science teachers.

Program Interpretations and Conclusions:

Overall I am still very pleased with the distribution and dispursion of the standards in the Earth Science Teaching Major Portfolio. The portfolio was very thoughtfully designed to give broad representation to the standards and I think it accomplished this well. Upon looking at this chart in detail, I see two issues. First, I realize that EAR 7 "Social Context" was inadvertantly left out of the portfolio Dimension 6: Relevance. This was an oversight that will be fixed. Second, I do not really understand the "frequencies" as they appear in the above chart. For example, EAR 1-3 and NSTA 2, 3 each appear one time in "Inquiry and Nature of Science". I am not sure why all but EAR 1 are shown as having been listed twice. Under "Content", EAR 1 and NSTA 1 are each listed once as a whole and 29 sub-headings for EAR 1 are also listed under different sub-sections. Therefore the "7s" are not quite accurate. The Science Teaching Minor is merely a link to a different portfolio and has not standards associated with it.

Program Interpretations and Conclusions:

I think the consistency and fairness of the data is high. I am the only person who assesses the Earth Science portfolios and I attempt to apply the rubric in the same way each time. Two topics of concern that are noticeable are that artifacts the students are left to select for themselves tend to be assessed at a lower quality than those specifically defined for the student from a specific class assignment - suggesting that the students do not always select well OR they work harder on assignments they know will end up in the portfolio. The reflections also tend to receive lower scores than the artifacts they refer to - suggesting that the students are not really aware of how to write the type of reflection required for this portfolio. As a Science Education Program, we hope to address both these issues through the development of a capstone course which all secondary science teaching majors are required to take. The seminar provides significantly more scaffolding for the students on artifact selection and reflection writing. The course was offerred as an elective for the first time in Winter 2008 and will be required for all students enrolling in the secondary science teaching programs in the future.

Program Interpretations and Conclusions:

These data show a high rate of success in completing the program and state standards. Teacher candidates are not allowed to progress to student teaching unless each of the portfolio dimensions is rated Proficient or higher. The portfolio assessment results from the portfolio are in agreement with the WEST-E results, which show 100% pass rate for the Earth and Space Science competency.

Please find below the West B data for the teacher residency program. Please use these data, the LiveText data, and the West E data found below to predict candidate success in your program. Given theses summaries, are there changes to your program or to the unit your program recommends the CTL consider?

• Between 2005-2007, 49% of the candidates passed all three sections of the exam their first attempt, 84% passed the reading portion in their first attempt, 82% math their first attempt, and 65% passed writing their first attempt.
• The mean number of candidates not passing reading portion is 11%, math 12%, and writing 25%.

CTL WEST B Data Summary 2002 to Present

Program Interpretations and Conclusions:

WEST-B scores are not a significant predictor of candidate success in science education. However, the distribution of first time pass rates matched what we see for basic skills in our courses and on our major artifacts. Being able to write effectively is the area of most acute need. We address this issue in our program by having numerous writing assignments, by having high expectations of excellent writing skills, and by explicitly evaluating the mechanics of writing on each assignment. According to the sample Livetext report shown above, writing mechanics typically receives the lowest score on a portfolio artifact. Our program stresses both technical (such as lesson plans) and reflective writing. Reflective writing is used to help students improve learning awareness and effectiveness (metacognition). Since the WEST-B summary data started being collected in 2002, the CWU General Education program has required students to take writing intensive courses. So far, no one has investigated the relationship between these courses and student success on the WEST-E.

Program Interpretations and Conclusions:

The pass rate for the Earth and Space Science WEST-E was 100% over the two years represented here.

Please find below the EBI teacher and principal data for all program completers. Discuss and report in the space provided what your program recommends the unit should accomplish to improve overall satisfaction, or what your program is doing to improve the trend.

• This survey is administered through OSPI and is contracted through Educational Benchmarking Inc. These data are collected for all new teachers in public schools by surveying new teachers and their principals.
• Response rate average over the seven years n=105
• The graph represents a seven year average satisfaction trend by category
• Highest satisfaction ratings are in the areas of:
  • Student learning
  • Instructional strategies
  • Management, control and environment
• Lowest satisfaction ratings are in the areas of:
  • Reading skills
• 5 year Principal responses followed similar patterns as teachers n=41

Program Interpretations and Conclusions:

Members of the Science Education Program are pleased to see that program satisfaction has generally improved over the last six years (EBI trend Data 2001-2007). However, the relative differences in satisfaction for the different categories has remained remarkably consistent with program effectiveness and reading skills have the lowest values throughout the six years. In the second graph, EBI Principal and Teacher Comparison Mean Satisfaction Score by category 2001-2006, the teachers are slightly less satisfied than the principals in all categories. The greatest difference in opinion lies in overall program effectiveness with the teachers more than half a point less satisfied. Everyone is in agreement that reading skills receive the lowest satisfaction rating.

The generally positive trend in teacher program satisfaction rating is something the Science Education Program will do everything possible to maintain. The implementation of our formative and summative assessment system is a key way we are pin pointing student dis/satisfaction within the program. Data acquired over time from several of these portfolio elements, 1) a performance-based, standards-aligned electronic portfolio, 2) entry and exit surveys, and 3) WEST-E and Major Field Test content examinations, will offer the Science Education Program team an opportunity to make adjustments.

Data from the Science Teaching Portfolios entry and exit assessments have been analyzed and used to address this unit’s overall program effectiveness. These entry and exit assessments not only promote students’ metacognitive awareness of the learning process and critical thinking, but offers our Science Education Program team important data that has been aggregated and analyzed to propose rationale programmatic changes. For example, aggregated exit surveys over the previous academic year, 2006-2007, from biology, chemistry, earth science, and physics teaching majors suggested that “EDCS 424: Reading in the Content Area” was least valued by students and seems to tie directly to the lower satisfaction seen the EBI survey. As a result of feedback from both sources, the Science Education Program has raised the question of how this course may either be improved or incorporated into existing science education courses. Currently, this specific discussion is ongoing while the Professional Education Program is being reexamined, but the topic has been raised due mainly to data that were acquired through Science Education Program’s entry and exit survey system.

Likewise, each student's choice of artifacts throughout the program portfolio, followed with written reflective summaries that defend their choice of artifact, help the Science Education Program to assess the students' proficiency of (a) pedagogical and content knowledge in the field of chemistry, (b) skills that are relative to inquiry teaching and learning, and (c) disposition as a teacher of chemistry in the context of each chosen artifact that aligns with national and state standards.

Please find below first year and third year teacher survey results summarized by graphing mean responses for each question.

• This survey is administered by CTL and data trend summary represents 2004-07
• The average response rate for 2004-2007 is 15%
• First year teacher N= 375, Third year teacher n =200
• The graph and subsequent ANOVA demonstrates a significantly higher average satisfaction rating from first year teachers when compared to third year teachers (p<.05)
• Highest satisfaction ratings are in the areas of:
  • Subject matter knowledge
  • Application of EALR's
• Lowest satisfaction ratings are in the areas of:
  • Classroom management
  • Involving and collaborating with parents

Program Interpretations and Conclusions:

Both classroom management and community relations (of which collaborating with parents is a subset) are topics that have been identified by other stakeholders, such as administrators and field observers, as areas of concern. Therefore, it is almost reassuring to see that new teachers also perceive these same topics to need a greater focus. Classroom management is a particularly challenging skill to become accomplished at outside of the classroom, so the science education program has for years focused on integrating as many practica experiences into the science education courses as possible. Furthermore, science education currently has a faculty representative on the Steering Committee to re-evaluate and possibly restructure the Professional Education Sequence, which is required of all education majors. One of the areas receiving particular attention during this processes is increasing the class time devoted to both theoretical classroom management and opportunities to implement this knowledge in teaching practica experiences. The Science Education program is also currently working to develop and integrate into existing secondary science programs a new course called "Science and Society". This course would be an ideal venue for presenting ways to involve and collaborate with parents.

Please find below a comparative analysis of candidate dispositions from beginning candidates to finishing candidates. Please comment on the changes you observe in your candidates over time and describe how and why you think this occurs. What does your program specifically do to engage candidates in developing professional teacher dispositions?

• This inventory is administered by the CTL at admissions (N=645), and again at the end of student teaching (N= 195). Some of the 645 candidates have not yet student taught, which is why the n's are different.
• There is a significant difference in 12 of 34 items (p<.05) between beginning candidates and candidates completing student teaching
• Change is in the preferred direction from agree to strongly agree
• This means somewhere between entry and before exit, the teacher program candidates are developing stronger professional beliefs and attitudes that reflect the underlying values and commitments of the unit's conceptual framework. Future work will include data that tells us where this change is occurring and if there are difference caused by demographic variables. If you want to read more about this disposition instrument, the validation study is published on the OREA web site under research.

Program Interpretations and Conclusions:

The Earth Science Teaching program promotes teacher dispositions by having clear professional expectations that are clearly and repeated communicated to candidates and validated by work with K-12 teachers. Professionalism is introduced at entry to program and reinforced repeatedly through classroom volunteer and teaching practica experiences in local K-12 schools, Science Education faculty panel interviews of candidates at the end of the secondary science methods course, advising, and entry and exit surveys. Advising and repeated observations of candidates in the field indicate that candidates develop a sincere appreciation for the field, an awareness of the responsibilities of a professional educator, and a willingness to meet the needs of all students. Candidate dispositions may change as a result of having clearly stated expectations, modeling of professionalism by Science Education faculty and K-12 teachers, and through increased awareness of the performance and financial benefits of high professionalism. The Earth Science Teaching portfolio also requires candidates to provide evidence of professionalism (Dimension 7) by becoming members of the National Science Teachers Association (CWU has an official NSTA chapter) or other professional teaching or science content discipline. Professionalism improves as candidates become immersed in regional and national conversations in NSTA journals and attend professional meetings.

Final Student Teaching Evaluation Report on LiveText

• The data report is too large to be placed in this document. Please access the data by going to this link on our assessment system web site http://www.cwu.edu/~ectl/ncate2.0/wastate/fsted-ltr.html
• The report reveals the final assessment of elements found in state standards IV and V
• Candidates are generally performing at a high level, although there are some candidates as depicted by the colors green and red who are not performing to standard.
• Examination of those elements indicates some agreement with results provided in the 1st and 3rd year teacher survey.

Please look at these data carefully and discuss with your program faculty some ways the teacher residency program can begin to address the few but common deficits occurring in candidate knowledge and skills relative to the State standard elements. If you need to refer to state standards please refer to this link in the assessment system website: http://www.cwu.edu/~ectl/ncate2.0/wastate/istandards.html

Program Interpretations and Conclusions:

These data show that most students are performing adequately during student teaching. The greatest deficits are in the Classroom Management and Discipline section. Our program feels deficits in this area can be addressed in the following ways.

1. Reconfigure the Professional Education Sequence (PES) to allow for more practical experience. Classroom management is a difficult topic to be taught in isolation. In the current professional sequence, classroom management is one part of one lecture-based course. Nearly every science content course requires practical experience in science (such as laboratories, field work, etc). Every science education course requires a practical experience in either science or science teaching. Thus, our candidates are set up to successfully meet science content and pedagogical content knowledge competencies. But, there is no room in our programs to better address classroom management competencies. The PES could be reconfigured in either one of two ways.

a) Cut down on the number of credits in the PES and allow individual programs such as science education to use those credits to design experiences to help candidates better meet all of the competencies including management. Safety is an important aspect of classroom management that is not adequately dealt with in the PES.

b) Cut down on the number of credits in the PES and develop a classroom management course with built-in opportunities to test management skills.

2. Develop a planning structure that allows discipline area faculty to observe student teachers so we can give feedback on management deficits early in the student teaching experience. This way, those weakness can be improved based on sound feedback.

Please examine these data and report any discussions your program has regarding the reported results.

• This survey is conducted by Career Services and reported to OSPI. The report, however, has been reanalyzed and the summary reflects the new analysis, which covers 2002-2006.
• Average response rate = 57%
• Of that 57%, the average percent of graduates who get jobs in state is 94%
• The average percent of graduate still seeking a position is 27%
• Two percent of the 57% have decided not to teach
• For 2005-2006; 35 % of the program graduates responded to questions regarding ethnicity and gender. Out of the 35% who responded, 90% were Caucasian, 5% were Hispanic, 3% were African-American, and 1.8% were Asian.

Program Interpretations and Conclusions:

Teacher placement data indicate most teaching graduates become employed in Washington and work in public schools, which is what they are trained to do. Some continue on to graduate work as well. Evidence from the Earth Science Teaching portfolio, entry and exit surveys, and WEST-E/MFT exams indicate that students are well prepared to teach biology in K-12 schools; the placement data verifies that students are being hired based on their training. Informal discussions with Earth Science Teaching graduates and in-service K-12 teachers indicate most enjoy their teaching positions and feel the Earth Science Teaching program prepared them to succeed in a K-12 environment. Portfolio data and survey results further pinpoint areas within the Earth Science Teaching and Professional Education programs that are particularly useful (and those less so). The placement data also show that a small number of graduates decide not to teach, which indicates that the Earth Science Teaching program selects those who really want to enter the teaching profession.