As a result of the evidence based decision making movement, the number of randomized trials evaluating educational programs and curricula has increased dramatically over the past twenty years. Policy makers and practitioners are encouraged to use the results of these trials to inform their decision making in schools and school districts. At the same time, however, little is known about the schools taking part in these randomized trials, both regarding how and why they were recruited and how they compare to populations in need of research. In this paper, we report on a study of 37 cluster randomized trials funded by the Institute of Education Sciences between 2011 - 2015. Principal Investigators of these grants were interviewed regarding the recruitment process and practices. Additionally, data on the schools included in 34 of these studies was analyzed to determine the general demographics of schools included in funded research, as well as how these samples compare to important policy relevant populations. We show that the types of schools included in research differ in a variety of ways from these populations. Large schools from large school districts in urban areas were over-represented, while schools from small school districts in rural areas and towns are under-represented. The paper concludes with a discussion of how recruitment practices might be improved in order to meet the goals of the evidence-based decision making movement.