Stephen Pruitt
One year ago I began a listening tour of 11 town hall meetings throughout Kentucky. During those meetings, I asked those who attended what they valued in the Commonwealth’s public schools. This would serve as the basis for building our new accountability system under the Every Student Succeeds Act.
What we often heard was the new system should focus on the welfare of all students and promote good decision making for their benefit. We were told that the system should reflect the Kentucky Department of Education’s (KDE’s) guiding principles of equity, achievement and integrity. And we also heard that the data should be reported in a dashboard format to better illustrate how your child’s school or district was progressing and where there was still work to be done.
We heard and we listened.
Because education is too important to do in a vacuum, more than 300 people have served on committees during the past nine months that looked at every aspect of what is important in a new accountability model. Teachers, administrators, parents, legislators and representatives of higher-education, community groups and advocacy groups spent countless hours working to make sure the needs of all children in Kentucky were represented. It was a tremendously difficult and complicated job and I can’t sing their praises loud enough for the kind of dedication each one of them brought to the table.
These committees’ proposed accountability system – which was introduced to the Kentucky Board of Education last month – for the most part, hits the mark of what Kentuckians asked of us. At the heart of the new system is the intention to make sure each child is progressing and every child has the same opportunities to excel in the classroom.
Proficiency on state assessments will still remain a key indicator of how a school is performing per federal law. Schools will report the percentage of students measuring proficient or above in English/language arts, mathematics, science, social studies and writing on state tests. Schools also will be reporting the progress of students on English Language Proficiency assessments and the percentage of students who meet state-determined benchmarks or higher on college readiness tests.
At the elementary and middle school level, growth will be based on each individual student’s progress toward proficiency measured against their own personal target for improvement. This idea helps even the playing field for schools and students. Gifted students who may start off the school year meeting proficiency measures for their grade must continue to be challenged in order for them to meet their growth targets. Conversely, schools and districts will be given credit for the hard work they put into advancing students who start out behind their classmates, instead of getting penalized if those students still don’t meet proficiency standards at the end of the year.
Closing the gap – meaning the gap between how various student groups perform and between groups and proficiency goals – also is a key component in Kentucky’s proposed accountability system. The goal for the system is that while schools are expected to improve the academic performance of all students, those who fall into one of the lower-performing groups will need to make gains faster.
While our long-term goal is for 100 percent of students to achieve proficiency, we know that challenges in the Commonwealth – such as widespread poverty and the current allocation for education – mean that closing the gap will take time. The committee’s aim is for historically lower-performing student groups to close the achievement gap by 50 percent by 2030.
Schools will receive a performance level currently described as Very Strong, Strong, Moderate, Low or Critically Low to more easily identify how a particular school is doing in getting all students to higher levels of achievement. To emphasize how important gap closure is in the proposed accountability system, schools making strong progress will receive an additional Gap Closure Designation, while schools that have either a large gap or a student group that is underperforming and not making progress cannot earn the highest overall rating and will receive a Gap Issue Designation attached to their overall rating.
And because equity is so important to all of us at the Kentucky Department of Education and the Kentucky Board of Education, our students and our parents – as evidenced by many of the comments at last year’s Town Hall meetings – opportunity and access measures will be included in the accountability system for the first time. In education, equity equates to fairness, the idea that all children should have access to the high-quality programs and teachers that will help them become a vital and contributing member of Kentucky’s workforce.
Proposed items to include in the new opportunity and access indicator include measures such as the percentage of students – and it should be 100 percent – that have access to classes on the visual and performing arts, health and physical education, social studies, practical living and career studies, as well as global competency and/or world languages. It also will take into account whether schools have a librarian/media specialist during the school day, whether there are guidance counselors and the percentage of teacher turnover and the percentage of first-year teachers. And at the high school level, the accountability system will include whether all students have access to rigorous Advance Placement and dual credit classes, as well as specialized career pathways.
Please remember that development of the new accountability system is still in progress and not final. We want your input into what you think works and what doesn’t. I’ll be hosting 10 Town Hall meetings across the state again this year, beginning in mid-March and continuing throughout April, to get your input. The proposed system also will be posted on the KDE website for public comment. Here is the list of when and where the Town Halls will be held and how your voice can be heard.
Tell us what you think Kentucky. We’re listening again!
Proficiency/Growth:
I’d also like to request that KDE considers adding a provision to stabilize the measurement of student performance in policy and subsequent contracts with assessment vendors. Unless substantial changes are made to the content and performance standards, KDE should require the assessment vendors to use IRT true score equating given the existing item bank for the provision of anchoring items. This will allow the same scale to be generated across assessment vendors and minor shifts in the test blueprints/specifications.
If the content/performance standards do see a more substantive change, KDE should require the assessment vendors to develop a non-equivalent groups linking design to map the new scale onto comparable scores on the existing scales (e.g., equipercentile linking with/without pre/post smoothing). While not ideal, this would provide a more transparent way to ensure a greater degree of stability in the performance level categories (naturally this assumes that the test characteristic curves are relatively similar in terms of the test/item parameters).
KDE should also require vendors for their assessment program to use a test design that will yield a robust vertical scale. In addition to making the scaled scores more useful/meaningful to parents (with regards to being able to apply basic arithmetic to compute growth) it also provides a more robust scale for use in continuous improvement efforts (specifically program evaluation and/or any type of longitudinal analytics).
Thanks again,
Billy Buchanan
Director, Office of Data, Research, and Accountability
I’d like to advocate for a change in the methodology used by KDE to estimate the adjusted cohort graduation rates to provide a more precise measurement that is also inherently more equitable. The 2008 federal non-regulatory guidance regarding the adjusted cohort graduation rates provides a limited set of examples of the computation that focus exclusively on the last known location of students. Unfortunately, students who are more transient also have a higher probability of not successfully completing high school in four years. This ends up creating a “penalty” of sorts on the schools in which students are last enrolled. Similarly, if a student transfers at the end of their secondary schooling the new school is credited with graduating the student.
Instead, I would like to propose a proportionally weighted adjusted cohort graduation rate. Over the course of the 4 years, students could potentially be enrolled in KY schools for 36 months. If the students transfer between schools and/or districts during that time frame we could weight the contribution of each student to the school/district’s graduation rate based on the number of months they were enrolled at the location. For example, if student A is enrolled in school 1 for freshman year, school 2 for their sophomore year, school 1 for their junior year, and school 3 for their senior year they would contribute a weight of 0.5 to school 1, and 0.25 to schools 2 and 3 (50% of the time at school 1, and 25% of the time at schools 2 and 3).
This method will yield a more robust metric that is a better indicator of the schools’ performance by adjusting not only for the in/out flow of students in the membership of the cohort but also adjusting by the relative contribution of the respectively schools and districts that are investing the resources and effort to prepare these students.
I agree with Ms. Conley. As a school psychologist with 23 years of experience, I don’t think our current method for assessing students with disabilities reflects actual progress and may have a negative impact on educational decisions. For the 1% of students with the most severe disabilities who participate in alternate assessments, the assessment tasks are very time consuming (since most are completed individually) and don’t seem appropriate for many students. For example, in the area of physical science, students at the high school level complete tasks related to the periodic table. Spending hours per week on such skills does not seem to be a constructive use of time for a student with severe health, communication, motor, vision, and cognitive concerns (i.e., functioning at the 6 month old level, fed via tube, needs daily breathing treatments). When the students are also dependent on adult support for all self-care needs such as feeding and toileting, there is little time left in the student’s day to work on functional and social skills.
Since only 1% of students can qualify for alternate assessments, there are many additional students with cognitive deficits or severe learning disabilities who must take the regular KPREP with no hope of reaching the Proficient level. A student with a Mild Mental Disability might make tremendous progress during a school year, moving from 3rd grade to 4th grade level math skills, but not be able to answer any items correctly on the 8th grade test even when allowed all available accommodations. Students with learning disabilities may demonstrate proficiency on KPREP when given a reader and scribe, but may not make any meaningful progress in their independent reading and writing skills over the course of a school year. Demanding proficiency on traditional end-of-the-year tests for students with disabilities may encourage schools to focus solely on grade level standards (with accommodations) rather than remediating areas of weakness so the student can be more self-sufficient in the future.
A better way might be for students with cognitive or academic deficits to take reading, math, and writing tests with items ranging from infancy (for the most severely disabled) to the high school level. Readers, scribes, and calculators would not be allowed unless the student had vision or motor problems. Students would take the test each year to determine if progress is made. If the student makes enough progress to approach grade level standards, he or she could resume taking the traditional test.
Thank you for your comments, Ms. Dunham and Ms. Garris. We will make sure all of our readers’ comments are added to the feedback Commissioner Pruitt receives at his series of Town Hall meetings.
Ms Dunham,
As a quick correction, the 1% rule defined in the federal legislation does not place a restriction on the number of students who are assessed using an alternate assessment. It restricts the proportion of students allowed to be counted as proficient for federal accountability purposes. So, if your district had 2% of its students taking the alternate assessment and all 2% scored high enough to be classified as proficient/distinguished, only half of those scores would be allowed to count as proficient. There are many different models for how state’s handle these particular cases.
In the federal non-regulatory guidance on the “1% rule”, USED describes a few methods that I believe are not helpful for the purpose of measuring the quality of schools (e.g., either the SEA or the LEA would decide which scores did/didn’t count). Rather than creating a system that embeds gaming the system into it’s measurement, I’d suggest adopting the methodology developed by Steve Hebbler in Mississippi for handling this particular rule. In that particular instance, instead of treating some of the students’ scores as not meeting proficiency, the individual student scores are down weighted only when the number of students exceeds the 1% level at the district. In the case where 2% of the students were tested on an alternate assessment and all of those students were classified as proficient, all of the students would still be included in the denominator (no changes here) and instead of counting as a value of 1 in the numerator (proficient and above / tested students) the students would contribute 0.5 to the numerator. This way all of the students are still represented in the accountability model but are weighted to ensure that the percent of students classified as proficient or higher on the alternate assessment never exceeds 1%.
USED approved Mississippi using this method when I worked at the SEA there in 2013, so I wouldn’t imagine there being a problem with Kentucky using the same methodology if they wanted.
Hope this helps,
Billy Buchanan
Director, Office of Data, Research, and Accountability
Fayette County Public Schools
It seems like liberal arts are extremely devalued in the new accountability system. With no Program Reviews, accountability testing, nor “accountability points” (some kind of bonus for successes in these areas – much like KOSSA testing for vocations), the arts and humanities will get little attention/resource/staffing compared to other areas of study. We should never diminish the profound truth in Dr. Drew Gilpin Faust’s quote: “The ability to recognize opportunities and move in new – and sometimes unexpected – directions will benefit you no matter your interests or aspirations. A liberal arts education is designed to equip students for such flexibility and imagination.” Countless research studies support the importance of providing students a liberal arts education. Make schools accountable for providing enriching artistic experiences for our students!
As a special educator in a rural district in Kentucky, I oppose the new accountability system. In the above article it states, “The goal for the system is that while schools are expected to improve the academic performance of all students, those who fall into one of the lower-performing groups will need to make gains faster.” While I expect my students to make gains, I feel that it is wrong to hold them to the same standards as other children. If I have a student on my caseload who has an IQ of 62 which is considered a Mild Mental Disability, then I expect them to be able to complete work that is on their level. I understand that this specific child may not be able to complete grade level test questions. This child should not be punished because intellectually he or she can’t perform. I should not be punished as the teacher and my school certainly shouldn’t be punished with a label of low or critically low. We have special education in place for a reason. Going against what has worked for years is not the solution for the future.
Ms. Conley, thank you for your comment about the proposed accountability system. Your comments will be taken into consideration as we all move forward in crafting the Commonwealth’s new system.
I agree with you Ms. Conley. I taught for 41 years and feel that disadvantaged students are also punished because of the emphasis on testing. I also see so many students and educators stress over test scores that it is counterproductive to learning and teaching. Surely there is a better way to assess school performance than testing..