cwu home     site map         
        
    

CTL PROGRAM REPORT
Chemistry Program

  
 
1.   Please describe your program's assessment process and what standards you are measuring in relation to the NCATE and State standards of knowledge (content, pedagogy and professional), skills (professional and pedagogical) and dispositions. Is the system course based, end of program based, or other? Be sure to reference how the faculty in your program was involved in developing the assessment process. In addition, describe how the assessment of standards relates to the unit's and program's conceptual framework.
 

Program Interpretations and Conclusions:

The Chemistry Teaching Program has a formative and summative assessment system comprised of several elements: 1) a performance-based, standards-aligned electronic portfolio, 2) entry and exit surveys, and 3) WEST-E content examination.

The Chemistry Teaching Portfolio is based on a common template collaboratively designed and constructed by all members of the Science Education Program, with additional insight provided by K-12 teachers. The conceptual framework is based on the latest scientific research on inquiry and how people learn (National Research Council, 2005). In particular, program assessment focuses on: 1) determining student preconceptions, 2) engaging candidates in authentic inquiry and investigation, 3) developing and applying robust content knowledge, and 4) promoting meta-cognitive awareness of learning process and critical thinking.

The main portfolio elements, or dimensions, were conceived through a consensus process by examining commonalities in three sets of professional standards; National Science Education Standards for Teaching, National Science Teacher Association Standards, and the Washington Competencies for Chemistry. The common dimensions include: 1) Inquiry and Nature of Science; 2) Teaching; 3) Learning Environments; 4) Assessment and Evaluation, 5) Relevance, and 6) Professional Growth. A distinct dimension for Content in the Chemistry Teaching Portfolio is further subdivided into the major disciplinary themes in chemistry science, and include: a) Analytical/Instrumental Chemistry , b) Organic Chemistry , c) Biochemistry, d) Inorganic Chemistry, e) Physical Chemistry, e) Application of Mathematics and Physics to Chemistry.

Candidates must demonstrate knowledge, skills, and disposition proficiency by providing tangible, verifiable evidence chosen from coursework, research and field work, and related disciplinary and educational experiences. Each portfolio dimension contains several strands designed to focus student effort and evidential support. For example, for the Inquiry and Nature of Science dimension, students must provide separate evidence for inquiry, nature of science, and independent research. In an effort to promote higher order thinking and meta-cognitive awareness, for most of the portfolio artifacts students are required to supply evidence they deem suitable rather than evidence prescribed by faculty. Candidates must also justify their choice of evidence and connect these to progress in meeting professional standards through a reflection that accompanies each portfolio dimension. Candidates are required to demonstrate minimum proficiency for each portfolio dimension in order to be allowed to student teach. Compliance is accomplished via an advising hold that may only be removed by a Science Education faculty member.

When a candidate enters the Chemistry Teaching Program, they must complete an entry survey that includes basic demographics as well as written items that assess candidate disposition. These results are then embedded within the Chemistry Teaching Portfolio. At the completion of the Chemistry Teaching Program, each candidate must also complete an exit survey whereby they evaluate program effectiveness, including all education courses in the Professional Education Sequence as well as Science Education courses. Experiences that were particularly useful are described, as are met and unmet candidate expectations. Exit and entry survey results are subsequently compared and a comparative reflection completed.

2. Below is an analysis of the frequency with which your program cites CTL, WA State Standards/Competencies, and/or national standards within your LiveText artifacts, rubrics, and reports. Please examine the charts and write your program's interpretations and conclusions based on the information provided. (e.g., Are the standards dispersed appropriately in your program? Are all the standards represented as you wish them to be? After reviewing this analysis are there changes your program would recommend making to the way you cite standards or assess your candidates using LiveText?)
 
 

Program Interpretations and Conclusions:

All NSTA and nearly all WA COMP standards are represented and well dispersed across the Chemistry Teaching Portfolio. Content knowledge and learning environment are particularly emphasized for the WA COMP and NSTA standards; assessment and evaluation are also emphasized for NSTA standards. Greater emphasis on inquiry and nature of science standards, particularly in core content courses, is a goal. Discussions between the Chemistry Teaching advisor and other department faculty to include more investigative learning experiences in core content curriculum are ongoing. External evaluation via program review commends current integration between chemistry and science education and supports further collaboration. Introductory chemistry laboratory courses for science majors are currently undergoing revision and will provide opportunity for better integration of inquiry experiences across the disciplines (biology and physics) and a more developed system to assess all chemistry undergraduates, not only chemistry teacher candidates, will be in place. Science Education faculty will collaboratively discuss standards alignment and other portfolio revisions in a mini-retreat sometime prior to Fall 2008. Revisions will focus on realigning Biology, Chemistry, Earth Science, and Physics Teaching Portfolios to the new WA COMP standards. Re-evaluation to ensure adequate standards coverage will follow, such as addressing the lack of WA-COMP CHEM 7 "Social Context" and possibly integrating it into Dimension 6: Relevance.

3. Below you will find one sample of your Live Text Report that identifies an aggregation of candidate learning outcome data. Please examine all of your reports in the LiveText exhibit area and discuss the accuracy, consistency, and fairness of the data, as well as what improvements could be made in the program assessment rubrics, courses, artifacts, or reporting. Include your interpretations relative how well your candidates are meeting standards. After examining all of your report data, list any changes your program is considering.
 
 

Program Interpretations and Conclusions:

The formative and summative assessments described in Dimension 1 allows Chemistry Teaching faculty to collect robust data used to drive program decisions. Data is accurate and fair insofar as the Chemistry Teaching advisor(s) evaluate all students consistently relative to professional benchmarks. Since the two Chemistry Faculty evaluate the Chemistry Teaching portfolios in a group discussion and enter a value that reflects an agreed upon score, interrater reliability is not determined. Separate rubrics used for each portfolio dimension allow greater insight into candidate specific knowledge, skills, and dispositions, and help identify common areas of candidate strength and weakness. Reports of candidate performance (as of March 2008) indicate candidates are performing well overall, with approximately 75% demonstrating excellent or above average performance across all dimensions. As of March 28, 2008, an N=4 for Chemistry Teaching Majors and N=5 for Chemistry Teaching Minors has been recorded, therefore no statistical significance can be assigned to the data evaluated thus far and the relative percentage values may seem either overstated and/or understated in some instances. Having said this, areas that meet minimum proficiency could benefit from additional mentoring of choice in suitable artifacts, such as the four Chemistry Teaching Major's choice of assessment of student learning artifacts. Areas of the Chemistry Teaching portfolio that typically score the lowest (approx. 0-25% partially proficient for majors and 60-25% partially proficient level for minors) are the dimension reflections that require students to explicitly connect provided evidence to progress meeting professional standards. This may indicate a lack of experience connecting evidence to progress, undeveloped metacognitive awareness, or some other factor. Interestingly, a lack of assessment expertise (and classroom management) is a common complaint of candidates, particularly prior to taking science methods courses and having practica experiences.

Part of these issues are being addressed with a Science Education capstone course which all secondary science teaching majors are required to take. This course is designed to support students through the completion of their portfolio and mentor them through the process of thoughtful reflective practices. Lack of assessment expertise remains a concern and will be discussed in the upcoming mini-retreat for Science Education Faculty prior to Fall Quarter of 2008. Please note that the screen capture above is simply a random sample of assessment data from late January, 2008, and are prior to the evaluation of five other chemistry teaching minor candidates.

4. Below you will find a chart of the CTL Standards aggregated by course. Please examine the data results and discuss any improvements if any you might consider for your program. Using these data, please reflect upon your candidates' success in meeting standards. Compare these data to the data provided in the WEST B and E charts that follow. Is there consistency in the rates of success? What do these data tell you?
 
 

Program Interpretations and Conclusions:

Aggregated results of the chemistry minor portfolio dimensions above and alignment to Washington Competencies do not fully indicate candidates' success to date. Some of the individuals above did not complete the Chemistry Teaching Minor and left the program with their primary Science Education Teaching Major degree, which are reflected in the "incomplete" section of the content and teaching sections. Those individuals who have completed the program are in upwards of exemplary/proficient of performing at state benchmarks and have either graduated or have gone on to student teach. More recent data than supplied above suggests that 90% of graduating and student teaching candidates are currently meeting competency requirements as indicated by Chemistry Teaching portfolio results. These results are not entirely corroborated by the WEST-E pass rate for Chemistry Teaching candidates, since several minors and one major had some difficulty in passing the WEST-E several times over the past two years.

Teacher candidates are not allowed to progress to student teaching in Chemistry unless each of the portfolio dimensions is rated Proficient or higher and they pass thee WEST-E for Chemistry. The portfolio assessment results from the portfolio are in partial alignment with the WEST-E results, which show 50% pass rate for the Chemistry competency.

Portfolio results show content knowledge, for minors, and relevance, for majors, are more variable than for other dimensions.

Consistently generating results and reflecting on Chemistry Teaching candidate performance (as two Chemistry Teaching advisors and collectively as Science Education faculty) will continue to drive program revision and improvement.

5.

Please find below the West B data for the teacher residency program. Please use these data, the LiveText data, and the West E data found below to predict candidate success in your program. Given theses summaries, are there changes to your program or to the unit your program recommends the CTL consider?

  • Between 2005-2007, 49% of the candidates passed all three sections of the exam their first attempt, 84% passed the reading portion in their first attempt, 82% math their first attempt, and 65% passed writing their first attempt.
  • The mean number of candidates not passing reading portion is 11%, math 12%, and writing 25%.

CTL WEST B Data Summary 2002 to Present

 
 

Program Interpretations and Conclusions:

As eluded to in the previous section, WEST-B scores are not a significant predictor of candidate success in Science Education. However, the distribution of first time pass rates match what the faculty see for basic skills in our courses and on major course artifacts. Being able to write effectively is the area of most acute need. Science Education faculty address this issue in the Biology, Chemistry, Earth Science, and Physics Teaching programs by having numerous writing assignments, having high expectations of excellent writing and communication skill, and by explicitly evaluating writing mechanics on each assignment. For example, format, spelling, and grammar make up a small but significant part of each portfolio rubric. All the Science Education programs stress both technical (such as lesson plans) and reflective writing (connecting evidence to progress toward meeting professional standards). Reflective writing is used to help students improve how, not just what, they learn (metacognitive awareness).

Since WEST-B summary data started being collected in 2002, the CWU General Education program has required students to take writing intensive courses. So far, no one has investigated the relationship between these courses and student success on the WEST-E.

6. The WEST E is administered by ETS as a state requirement for program Exit, measuring content knowledge by endorsement area. ETS has not sent the final corrected data summary at the time of this report, however, the data we keep on a continuously updated basis is described below in the following graph. The graph compares 2005-2006 and 2006-2007 data by endorsement area. We suspect the 2006-2007 data will change after all scores are received from ETS. According to this set of data, 2005-06 pass rates were 90%. Remember all candidates must pass the test to be certified, so they take it multiple times. We are working on authenticating a different process that will show how many times candidate take the test and when. The 2006-07 data indicates pass rates of 87%. If your program is one of those with a pass rate below 80%; what program recommendations are you considering that will positively affect the rate of passing the WEST-E for 2007-2009?
 
 

Program Interpretations and Conclusions:

The pass rate for Chemistry Endorsement on the WEST-E is 50%. Such a low pass rate sent alarms to the Science Education Program and the Chemistry Teaching Program. We promptly asked for and quickly received data in the Winter Quarter of 2007 that elucidated the situation for us. The low pass rate was due mainly to a single person who was taking and retaking the WEST-E Chemistry exam multiple times, 5 times in total. This act alone, with such a small number of students taking the Chemistry WEST-E from CWU, decreased the pass rate significantly. Since this time, Chemistry Teaching Advisors have requested WEST-E information after every offering state-wide so that we can be proactive in advising failing students to either better prepare for the exam or advise them into an alternate career path.

Another reason for such a low pass rate is due, in part, to two or three individuals who were in progress of earning the chemistry teaching minor taking the exam and not retaking it a second time in order to pass.

In the case of WEST-E failure, since the Spring Quarter of 2007, the chemistry teaching advisors have followed up with each individual to discuss what portion of the WEST-E was most difficult for them. Most of the anecdotal evidence pointed to a particular portion of the exam that dealt with Nuclear Chemistry and Nuclear Decay. After reviewing the quantitative WEST-E data, we corroborated student perception, noticing that Category II: Atomic and Nuclear Structure was consistently low for all passing and non-passing chemistry teaching majors and minors. Probing further, we were able to pour over the chemistry curriculum and have actively participated with the Chemistry Department’s Curriculum Committee in order to integrate more nuclear chemistry into the general chemistry series and physical chemistry. It is our hope that this intervention will play a roll in supporting teacher candidate preparation for the WEST-E.

7.

Please find below the EBI teacher and principal data for all program completers. Discuss and report in the space provided what your program recommends the unit should accomplish to improve overall satisfaction, or what your program is doing to improve the trend.

  • This survey is administered through OSPI and is contracted through Educational Benchmarking Inc. These data are collected for all new teachers in public schools by surveying new teachers and their principals.
  • Response rate average over the seven years n=105
  • The graph represents a seven year average satisfaction trend by category
  • Highest satisfaction ratings are in the areas of:
    • Student learning
    • Instructional strategies
    • Management, control and environment
  • Lowest satisfaction ratings are in the areas of:
    • Reading skills
  • 5 year Principal responses followed similar patterns as teachers n=41

 

Program Interpretations and Conclusions:

Members of the Science Education Program are pleased to see that program satisfaction has generally improved over the last six years (EBI trend Data 2001-2007). However, the relative differences in satisfaction for the different categories has remained remarkably consistent with program effectiveness and reading skills have the lowest values throughout the six years. In the second graph, EBI Principal and Teacher Comparison Mean Satisfaction Score by category 2001-2006, the teachers are slightly less satisfied than the principals in all categories. The greatest difference in opinion lies in overall program effectiveness with the teachers more than half a point less satisfied. Everyone is in agreement that reading skills receive the lowest satisfaction rating.

The generally positive trend in teacher program satisfaction rating is something the Science Education Program will do everything possible to maintain. The implementation of our formative and summative assessment system is a key way we are pin pointing student dis/satisfaction within the program. Data acquired over time from several of these portfolio elements, 1) a performance-based, standards-aligned electronic portfolio, 2) entry and exit surveys, and 3) WEST-E and Major Field Test content examinations, will offer the Science Education Program team an opportunity to make adjustments.

Data from the Science Teaching Portfolios entry and exit assessments have been analyzed and used to address this unit’s overall program effectiveness. These entry and exit assessments not only promote students’ metacognitive awareness of the learning process and critical thinking, but offers our Science Education Program team important data that has been aggregated and analyzed to propose rationale programmatic changes. For example, aggregated exit surveys over the previous academic year, 2006-2007, from biology, chemistry, earth science, and physics teaching majors suggested that “EDCS 424: Reading in the Content Area” was least valued by students and seems to tie directly to the lower satisfaction seen the EBI survey. As a result of feedback from both sources, the Science Education Program has raised the question of how this course may either be improved or incorporated into existing science education courses. Currently, this specific discussion is ongoing while the Professional Education Program is being reexamined, but the topic has been raised due mainly to data that were acquired through Science Education Program’s entry and exit survey system.

8.

Please find below first year and third year teacher survey results summarized by graphing mean responses for each question.

  • This survey is administered by CTL and data trend summary represents 2004-07
  • The average response rate for 2004-2007 is 15%
  • First year teacher N= 375, Third year teacher n =200
  • The graph and subsequent ANOVA demonstrates a significantly higher average satisfaction rating from first year teachers when compared to third year teachers (p<.05)
  • Highest satisfaction ratings are in the areas of:
    • Subject matter knowledge
    • Application of EALR's
  • Lowest satisfaction ratings are in the areas of:
    • Classroom management
    • Involving and collaborating with parents
 

Program Interpretations and Conclusions:

Both classroom management and community relations (of which collaborating with parents is a subset) are topics that have been identified by other stakeholders, such as administrators and field observers, as areas of concern. Therefore, it is almost reassuring to see that new teachers also perceive these same topics to need a greater focus. Classroom management is a particularly challenging skill to become accomplished at outside of the classroom, so the science education program has for years focused on integrating as many practica experiences into the science education courses as possible. Furthermore, science education currently has a faculty representative on the Steering Committee to re-evaluate and possibly restructure the Professional Education Sequence, which is required of all education majors. One of the areas receiving particular attention during this processes is increasing the class time devoted to both theoretical classroom management and opportunities to implement this knowledge in teaching practica experiences. The Science Education program is also currently working to develop and integrate into existing secondary science programs a new course called "Science and Society". This course would be an ideal venue for presenting ways to involve and collaborate with parents.

9.

Please find below a comparative analysis of candidate dispositions from beginning candidates to finishing candidates. Please comment on the changes you observe in your candidates over time and describe how and why you think this occurs. What does your program specifically do to engage candidates in developing professional teacher dispositions?

  • This inventory is administered by the CTL at admissions (N=645), and again at the end of student teaching (N= 195). Some of the 645 candidates have not yet student taught, which is why the n's are different.
  • There is a significant difference in 12 of 34 items (p<.05) between beginning candidates and candidates completing student teaching
  • Change is in the preferred direction from agree to strongly agree
  • This means somewhere between entry and before exit, the teacher program candidates are developing stronger professional beliefs and attitudes that reflect the underlying values and commitments of the unit's conceptual framework. Future work will include data that tells us where this change is occurring and if there are difference caused by demographic variables. If you want to read more about this disposition instrument, the validation study is published on the OREA web site under research.
 

Program Interpretations and Conclusions:

The Chemistry Teaching program promotes teacher dispositions by having clear professional expectations that are communicated to candidates and validated by work with in-services K-12 teachers. Professionalism is introduced at entry to program and reinforced repeatedly through classroom volunteer and teaching practica experiences in local K-12 schools, Science Education faculty panel interviews of candidates at the end of the secondary science methods course, advising, and entry and exit surveys. Advising and repeated observations of candidates in the field indicate that candidates develop a sincere appreciation for the field, an awareness of the responsibilities of a professional educator, and a willingness to meet the needs of all students. Candidate dispositions may change as a result of having clearly stated expectations, modeling of professionalism by Science Education faculy and K-12 teachers, and through increased awareness of the performance and financial benefits of high professionalism. The Chemistry Teaching portfolio also requires candidates to provide evidence of professionalism (dimension 7) by becoming members of the National Science Teachers Association (CWU has an official NSTA chapter). Professionalism improves as candidates become immersed in regional and national conversations in NSTA journals and attend professional meetings.

 

10. 

Final Student Teaching Evaluation Report on LiveText

  • The data report is too large to be placed in this document. Please access the data by going to this link on our assessment system web site http://www.cwu.edu/~ectl/ncate2.0/wastate/fsted-ltr.html
  • The report reveals the final assessment of elements found in state standards IV and V
  • Candidates are generally performing at a high level, although there are some candidates as depicted by the colors green and red who are not performing to standard.
  • Examination of those elements indicates some agreement with results provided in the 1st and 3rd year teacher survey.

Please look at these data carefully and discuss with your program faculty some ways the teacher residency program can begin to address the few but common deficits occurring in candidate knowledge and skills relative to the State standard elements. If you need to refer to state standards please refer to this link in the assessment system website: http://www.cwu.edu/~ectl/ncate2.0/wastate/istandards.html

 

Program Interpretations and Conclusions:

These data show that most students are performing adequately during student teaching. The greatest deficits are in the Classroom Management and Discipline section. Our program feels deficits in this area can be addressed in the following ways.

1. Reconfigure the Professional Education Sequence (PES) to allow for more practical experience. Classroom management is a difficult topic to be taught in isolation. In the current professional sequence, classroom management is one part of one lecture-based course. Nearly every science content course requires practical experience in science (such as laboratories, field work, etc). Every science education course requires a practical experience in either science or science teaching. Thus, our candidates are set up to successfully meet science content and pedagogical content knowledge competencies. But, there is no room in our programs to better address classroom management competencies. The PES could be reconfigured in either one of two ways.

a) Cut down on the number of credits in the PES and allow individual programs such as science education to use those credits to design experiences to help candidates better meet all of the competencies including management. Safety is an important aspect of classroom management that is not adequately dealt with in the PES.

b) Cut down on the number of credits in the PES and develop a classroom management course with built-in opportunities to test management skills.

2. Develop a planning structure that allows discipline area faculty to observe student teachers so we can give feedback on management deficits early in the student teaching experience. In this way, we believe that said weakness may be addressed based on sound feedback.

11.

Please examine these data and report any discussions your program has regarding the reported results.

  • This survey is conducted by Career Services and reported to OSPI. The report, however, has been reanalyzed and the summary reflects the new analysis, which covers 2002-2006.
  • Average response rate = 57%
  • Of that 57%, the average percent of graduates who get jobs in state is 94%
  • The average percent of graduate still seeking a position is 27%
  • Two percent of the 57% have decided not to teach
  • For 2005-2006; 35 % of the program graduates responded to questions regarding ethnicity and gender. Out of the 35% who responded, 90% were Caucasian, 5% were Hispanic, 3% were African-American, and 1.8% were Asian.
 

Program Interpretations and Conclusions:

There is a high need for qualified science teachers in Washington. If our graduates are willing to teach anywhere in the state, we can almost guarantee them a job in their endorsement area or a similar area. We have developed a general science teaching program in response to the needs of districts that hire teachers for positions in two or more endorsement areas. Because science education faculty members work closely with our science teacher candidates and because we give them a variety of practical science and teaching experiences, our teacher candidates are generally satisfied with their career choice and actively seek teaching jobs.

Evidence from the Chemistry Teaching portfolio, entry and exit surveys, and WEST-E and ACS exams indicate that students who complete the program are well prepared to teach chemistry in K-12 schools; the placement data verifies that students are being hired based on their training. Informal discussions with Chemistry Teaching graduates indicate most enjoy their teaching positions and feel the Chemistry Teaching program prepared them to succeed in a K-12 environment. Portfolio data, survey results, and WEST-E and ACS exams further pinpoint areas within the Chemistry Teaching and Professional Education programs that are particularly useful (and those less so) and areas of the content curriculum in which students need more support.

As a team, Science Education is addressing the lack of ethnic diversity of CWU teacher candidates by creating program in more ethnically diverse areas such as the Lynnwood centers (General Science Teaching Program), procuring grants like WATERS to encourage science education interested in STEM graduate students, and applying for grants like Robert Noyce Teacher Scholarship Program which targets community college and 4 year college students to complete STEM related teaching programs at CWU and place them in high needs population high schools ( x>50% free and reduced lunch).

   

 

 

 

 

 
© Central Washington University   |   All Rights Reserved