Editor’s note: Senate Bill 1 (SB 1), enacted in the 2009 Kentucky General Assembly, requires a new public school assessment program beginning in the 2011-2012 school year. Kentucky Teacher is doing a series of stories explaining the Unbridled Learning: College/Career Readiness for All assessment and accountability system this month. This article focuses on middle school assessment. Future stories will focus on high school assessment and accountability. The system is subject to United States Education Department approval and may be changed prior to adoption.
Cole Kaminski reads “Hatchet” by Gary Paulsen during Julie Wadlington’s 6th-grade reading class at Lyon County Middle School. Wadlington tests the students after every two chapters for reading comprehension. Photo by Amy Wallot, Nov. 15, 2011
By Matthew Tungate
matthew.tungate@education.ky.gov
Kentucky’s previous assessment and accountability system was based on the idea of getting students, schools and districts to “proficiency.” But proficiency was an abstract concept, Kentucky Department of Education Associate Commissioner Larry Stinson said – and it left something to be desired in practicality.
“We have way too many students who would qualify to graduate, but they’re not ready to do anything,” he said. “So we were looking for a way to say, ‘If you are graduating from high school in Kentucky, you have some skills that you can use at that next level.’ And this is the way to go about it.”
So the state is defining proficiency based on college and career readiness, he said, and the ACT is the capstone measurement for determining it. The ACT provides an extremely strong research-based prediction of college readiness and plays a major part in Kentucky’s College/Career Readiness indicator. That is why all 11th-grade students take the ACT. The ACT PLAN test, given to all 10th-grade students in Kentucky, provides a direct connection from its scores to a predicted ACT score, thus linking early high school work to college readiness.
But college- and career-readiness assessment and accountability actually starts in middle school under Kentucky’s Unbridled Learning: College/Career Readiness for All assessment and accountability system. Until now, Stinson said, middle schools hadn’t had to show their students were on a college- or career-ready trajectory.
Under the new system, schools will receive a score based on how well students do on the EXPLORE test. And it will be up to schools to do something with those results, Stinson said.
“What are people doing with the results, or what are those scores telling us so that we can have more children achieve at higher levels?” he said.
But testing is only part of the Unbridled Learning: College/Career Readiness for All assessment and accountability system. The accountability model incorporates all aspects of school and district work. It includes student data from testing, gap, growth, college/career readiness and graduation rate (Next-Generation Learners); principal and teacher effectiveness (Next-Generation Professionals); and Program Reviews (Next-Generation Instructional Programs and Support).
This year, however, only the Next-Generation Learners component will be used for school and district accountability. For middle schools, that means results will be based on the results of the Kentucky Performance Rating for Educational Progress (K-PREP) tests and EXPLORE.
Schools must administer the K-PREP in the last 14 days of the school year.
What is the new test?
Just as it has for nearly the past decade, the new accountability system will continue to test every student in grades 3-8 in reading and mathematics every year.
“Reading and math are foundational skills,” said Rhonda Sims, a director in the Kentucky Department of Education’s Office of Assessment and Accountability. “They are certainly key foundational subjects for work in science, for work in social studies, for overall work on things like ACT, PLAN and EXPLORE.”
Students also will be tested in science and social studies once in elementary school and once in middle school, which meets federal requirements, Sims said.
“Kentucky, back from the beginning of reform in ’90, said all subjects were important,” she said. “There are states that have only tested reading and math for years.”
The new assessment for grades 3-8 is a built with norm-referenced test (NRT) and criterion-referenced test (CRT) items that consist of multiple-choice, extended-response and short answer items. The NRT is a purchased test with national norms and the CRT portion is customized for Kentucky.
The summative assessments in grades 3-8 are being developed based on Kentucky Core Academic Standards. The standards were written to have incrementally increasing levels of rigor and alignment with college readiness standards.
Students will take the EXPLORE test in the 8th grade.
What happens to the results once students take the test?
As in past years, students’ performance on the tests will be identified as novice, apprentice, proficient or distinguished based on the results, Sims said.
Middle school students’ results will be used in three of the Next-Generation Learner areas: achievement, gap and growth.
Achievement is similar to grades in that a yet-to-be-determined minimum score will determine whether a student is distinguished, proficient, novice or apprentice, Sims said.
“The standard setting of when a student is novice, apprentice, proficient or distinguished has to occur after we have test results, because otherwise we’re arbitrary in setting is the definition for novice performance when we don’t have the current test results,” Sims said. “We can’t assume that on the new test, where we say the standards are higher and tougher and harder, we will be at the same level of performance and the same distribution.”
The Kentucky Board of Education decided last year that schools would receive one point for every percentage of students scoring proficient or distinguished for each of the content areas (reading, writing, mathematics, science and social studies). Schools will receive half a point for each percent of students scoring apprentice, and no points for students scoring novice.
Schools also are eligible for bonus points. Each percentage of students scoring distinguished earns an additional half point, and the percent scoring novice loses a half point.
“If you have more distinguished kids than novice, then that difference becomes a bonus that’s added,” Sims said. “If the reverse is true and you have more low performers than high performers, nothing happens. You don’t lose points. It was important to the board, while recognizing the highest performance of our students, that we not lose our focus on improving our lowest performers.”
The second area on which schools will be judged is gap. Under the new accountability system, African-American, Hispanic, Native American, special education, low income and limited English proficiency students will be combined as one group for calculation of the achievement gap and school gap-reduction goal. However, scores for the various demographic groups would still be reported individually.
“It would hold all districts accountable for all children,” Commissioner Terry Holliday said.
The gap score is based on the percentage of students who score proficient and distinguished. Unlike the previous assessment and accountability system, however, the new system counts students’ scores only once, even if they fit into multiple gap groups.
The department is concerned about the potential masking of individual gap group scores even though all gap groups will be reported, so the accountability portion of Unbridled Learning takes that into consideration. All schools with gap groups underperforming by a certain percentage will face state consequences.
Sims said the consequences for underperforming individual gap groups are a new addition because of Kentucky’s request of a waiver from federal No Child Left Behind (NCLB) Act requirements.
The final K-PREP-related part of Unbridled Learning is growth in reading and mathematics. The growth calculation is designed to measure a student’s growth in learning each year as compared to the student’s academic peers. So, higher-performing students’ scores are compared against other high-scoring students across the state, while lower-performing students’ scores are compared against other lower-scoring students’ scores.
Points are awarded for the percentage of students who grow at typical or higher levels relative to their peers, with typical growth defined as at the 40th percentile or higher.
A computer program generates the bands based on the results of all Kentucky students who took the same test, Sims said, so schools won’t be able to calculate their own growth.
The results may surprise some schools, especially those with low expectations who think their students are growing until they are compared to students from schools with high expectations, she said.
“There you might be able to ask instructional questions about are you teaching the standards, are you linking back to the standards, are you at the level the standards expect, if you’re not seeing growth,” Sims said.
The growth model also will show high achievers who are just coasting, she said.
“Because when I look at him in comparison to all other kids scoring right up there at the top, he might not have shown growth,” Sims said.
Since 2011-12 is the first year of the K-PREP test, Kentucky Core Content Test (KCCT) scores from 2011 will be used as the basis for comparison, Sims said.
The growth part of the Unbridled Learning calculation is “the hardest thing to understand,” she said.
Middle schools’ college/career readiness scores will be based on the percentage of students meeting benchmarks in three content areas on EXPLORE at middle school. The percent of students meeting the ACT-established benchmarks for EXPLORE in reading (15), English (13) and mathematics (17) will be averaged to generate a middle school college readiness percentage
What happens to the test results?
Points earned from each of the four areas – achievement, gap, growth and college/career readiness – will be added to calculate a score, each with its own weight.
Achievement and gap will be calculated using reading, mathematics, science, social studies and writing results. Growth will be based on reading and mathematics results. College/career readiness will be factored based on EXPLORE results.
Until the other components are completed, only the Next-Generation Learners component will be used to generate an overall score for accountability. The following chart provides the overall score phase-in for the three components.
When will the other two areas of accountability be included?
Program Review, as part of Next-Generation Instructional Programs and Supports, will be added to the calculation in 2012-13. Rather than testing students to see what they have learned, Program Reviews require schools to gather evidence about how they integrate subjects across curricula and provide students with high-quality learning opportunities. The schools then use the information to improve programs.
Schools piloted Program Reviews during the 2010-11 school year in three areas: arts and humanities, practical living/career studies, and writing. Those Program Reviews are being field-tested and the results will be made public this school year. Full accountability for the three Program Reviews will begin in the 2012-13 school year.
Program Reviews in world language and K-3 will be added in future years.
In the 2013-14 school year, Kentucky will implement its teacher- and principal-evaluation system as part of the Next-Generation Professionals section.
During the December Kentucky Board of Education meeting, Holliday told the board that the system must have six components to meet federal waiver standards:
1. continuous improvement of instruction
2. meaningful differentiation of teacher/principal performance using at least three levels
3. multiple measures of effectiveness including the use of student growth data (both state standardized tests and formative growth measures that are rigorous and comparable across schools in a local district) as a significant factor
4. regular evaluation (most likely annual)
5. clear and timely feedback to include opportunities for professional development
6. use of the system to inform personnel decisions.
Kentucky’s plan meets these standards, including the multiple measures – and the use of student growth data, he said.
“If we get the waiver, there is no system that could ignore student growth on standardized tests as part of the evaluation system,” Holliday said.
The board’s most difficult discussion to come is the weighting of each of the multiple measures and how much flexibility local districts will have to use their own evaluation systems, he said.
“The neat thing about the new system is you’ve got all these different sort of looks or lenses that allow you to look at kid performance and school and district performance, whereas before it was all very focused just on achievement,” Sims said. “And are these things kind of correlated to each other? Well, of course they are.”
Leave A Comment