Study of the Turnaround Schoool Leaders Program

Contract Number: ED-PEP-11-O-0088/TO28

Task Order 28

Implementation Study of the
Turnaround School Leaders Program

Appendix E. District Respondent Interview

Interview Protocol

September 28, 2016

Prepared for:

U.S. Department of Education

Office of Planning, Evaluation and Policy Development

Policy and Program Studies Service

Prepared by:

Westat

1600 Research Boulevard

Rockville, MD 20850-3129

(301) 251-1500

and

Policy Studies Associates

1718 Connecticut Avenue, NW, Suite 400

Washington, DC 20009

(202) 939-9780

Interview Protocol

Introduction

Thank you for agreeing to talk to me. My name is [NAME], and I work for [ORGANIZATION], one of the partner organizations the U.S. Department of Education has contracted with to conduct this study.

This study is about how the Turnaround School Leaders Program (TSLP) is being implemented by [PROJECT NAME]. We are talking to a variety of project partners to gain a detailed picture of your project. We will be asking about partnerships, identification and recruitment of school leaders, pre-service training, ongoing support of school leaders, and program performance.

The interview should take about an hour.

[Have respondent read and sign consent form]

Do you have any questions before we begin?

[Start audio recorder]

Respondent information

1. Let’s just start with some background information. How long have you worked for [insert project name]? Would you give me a quick overview of your main responsibilities as [insert respondent position]? What other professional experiences have you had in either school leadership or school turnaround?

• Listen for and probe as necessary for connection to TSLP specifically and school leadership and school turnaround more generally.

Partnerships between LEAs and partner organizations

Topics for this section:

• Shared Vision (desired leadership standards/outcomes)

• Identification and division of roles and responsibilities

• Authority (memoranda of understanding, “final say”)

2. What leadership standards, competencies, or expected behaviors has your TSLP project identified as necessary for turnaround leaders? How did your project identify these standards and competencies?

3. Think about [the districts’ participating in this TSLP grant/your district’s] current leadership standards and job descriptions.

• How well aligned are current leadership standards and job descriptions to district school turnaround strategies and the TSLP project?

• What changes, if any, have been made to the leadership standards or job descriptions due to the TSLP project?

• What challenges have you experienced aligning your leadership standards or job descriptions with [the TSLP/those of participating districts]?

4. When you first heard about the TSLP grant, in what ways did you expect that it would support your district’s school leadership and school turnaround work?

• To help you recruit better leaders?

• To help you develop better leaders?

• To give you access to more resources for turnaround work?

• To support specific turnaround strategies, such as replacing existing principals?

• To build district capacity to develop and support school leaders?

5. How do you think the TSLP project fits or aligns with the district’s broader strategy for turning around its lowest performing schools?

• How does the project fit with or support other district initiatives?

6. Tell me about the process of working with your parners in the TSLP project.

• Do MOUs or other agreements exist that describe partners’ roles and responsibilities? How, if at all, have these MOUs changed since the project’s inception?

• [If MOU/agreement exists] What were the challenges associated with negotiating MOUs/agreements?

• How closely did you think your district’s, vision for the TSLP project align with those of the other partners?

• Why do you think the partners decided to join together? Why did your district become a partner?

7. Please describe how responsibilities of the partners are divided or shared for:

• project design and management,

• selection of participants,

• curriculum development,

• delivery of content,

• assessment of participants, and

• ongoing support after placement.

8. Overall, how is the partnership working out among the organizations involved in this TSLP project?

• What strengths do the different partners bring to the project?

• Could you provide an example of a situation that showed the strength of the partnership?

• What challenges has your project experienced in developing and maintaining partnerships? How have these been addressed?

Identification of leaders

Topics for this section:

• Recruitment (identification of talent pools of vetted, qualified aspirants)

• Selection (criteria, process)

• Hiring/placement (criteria, process)

9. For a typical participant please describe the entire process from recruitment to placement.

• Eligibility requirements, recruitment methods, and staff or organizations involved.

• Selection process.

• Selecting candidates from the applicant pool.

• Placing project participants in schools.

• How do these processes differ from those for leaders of non-turnaround schools?

Pre-service development and training of leaders

Topics for this section:

• Supervision

• Professional development (design, content, assessment)

10. Please describe the training and development your project provides to turnaround school leaders.

• What skills or competencies does the training emphasize?

• How did you decide on these particular skills or competencies?

• How do these competencies relate to district strategies for turning around schools?

• Were the specific strategies of the SIG turnaround or transformation models considered in deciding which skills or competencies to emphasize? Why or why not?

• To what extent does your project involve problem-based instruction?

• To what extent does your pre-service preparation involve a cohort model for aspiring leaders?

• To what extent is your pre-service preparation aligned with your district’s standards or strategy for school leadership and school turnaround?

• [If applicable] How did you align your project with district standards or strategies?

11. What are the arrangements for on-the-job learning? Is there a residency placement, or do participants carry out project-related assignments in their regular jobs?

12. To what extent does the time to complete your program vary for different aspiring leaders?

• [If time to complete varies] What are the main reasons that the time-to-complete varies?

13. What positions do participants tend to move into upon completion of the program (within the year after completion)?

What proportion are teachers, assistant principals, and principals ?

Ongoing support for leaders

Topics in this section:

• Mentoring/Coaching (design, content)

• In-service Professional Development (design, content)

• Placement and Retention Incentives (design, perceived utility)

• Leader Evaluation (criteria, procedures)

14. What supports did your district provide to school leaders before the TSLP grant?

15. In what ways, if at all, have your district’s systems of leader support, particularly for turnaround leaders, changed? What are the likely reasons for any changes?

• Have there been changes in mentoring and coaching provided by district staff, mentoring and coaching provided by outside providers, peer support networks, other professional development (i.e., not TSLP-supported), approaches to targeting support for individual leaders?

  • Probe for changes in intensity, duration, content, PD provider, release time, etc.

• What challenges, if any, were encountered in aligning existing supports for school leaders with those provided by the TSLP project? How were these addressed?

16. Do you provide any financial or non-financial incentives for leaders to take or stay in a leadership job in turnaround schools? Are any of these incentives performance-based? Please describe.

• [If provide incentives] Have these incentives affected recruitment and retention of effective turnaround leaders in the schools?

17. Are participants in the project evaluated differently from other school leaders? If so, how?

• [If participants are evaluated differently] Could you share an example of the performance measures (e.g., professional practice rubric)?

18. How often do participants get feedback on their performance?

• What kind of feedback do they get? Could you provide some examples?

Program performance

Topics for this section:

• Data collection (indicators, procedures, maintenance)

• Assessment metrics (design, use)

• Analysis (responsible parties, methods)

• Results (early themes, dissemination)

• Ongoing improvement

• Adaptations post-applications

• Challenges faced during implementation

19. How, if at all, are project, partner, and district data on candidates brought together into a single data system?

• If there is a single system, who administers it?

• What have been the challenges in integrating data systems and combining data?

• How are these data used in selecting participants for the TSLP project and placing candidates upon completion?

20. How would you describe the strengths and weaknesses of new turnaround leaders in [the districts participating in this TSLP grant/your district]?

21. What have been the biggest changes in your district in the development of turnaround leaders in the past three years?

• Has the quality of the pool of potential leaders improved?

• Has the quality of the pre-service professional development available for potential turnaround leaders improved?

• Has the support provided to newly placed turnaround leaders improved?

• Has retention of effective turnaround leaders improved?

22. How does this project fit into your district’s overall strategy for turning around low performing schools?

23. What are the biggest remaining challenges in your district’s development of turnaround leaders?

24. What changes have been made to your project since your application? Why were these changes made?

• What challenges have you faced in implementing your original plans?

• How have you made decisions about adjustments to the project?

• How have you used data and research evidence to make these changes?

• How has your thinking about the project overall changed as a result of needing to make these changes?

Lessons learned

25. We are interested in any lessons from your TSLP project that might be informative to other states and districts. Could you describe examples of strategies that worked well and examples of challenges related to the following:

• Essential features of a beneficial partnership seeking to prepare leaders?

• Recruitment, selection, and placement of leaders?

• Pre-service development and training of leaders?

• Ongoing support for leaders?

• Measuring program performance?

26. After the TSLP grant ends, what will the next steps be for you, the project, and your partners?

• What steps, if any, will you take to maintain the project work?

27. What steps, if any, has your district taken to develop a long-term pipeline for turnaround school leaders? Do you have a sustainability plan or goals?

• [If respondent describes steps taken] How has the TSLP project contributed to developing a long-term pipeline for turnaround school leaders?

• (If no) What could the project have done differently to help develop such a pipeline?