Appendix
As part of this survey, staff at the programs were asked to provide evaluation materials. Their studies were compiled for different purposes: to tell a program's story and share its experience with a wider community; to evaluate the strengths and weaknesses of the program for mid-course improvements; to assess the progress of participants individually or as a group; to compare current program practices with program goals.
Most of the assessments were conducted by program staff, rather than outside evaluators, using a wide range of tools, including site visits; case studies; journals; focus groups; rating scales and surveys; school records; media coverage; portfolio reviews and performance evaluations, including video documentation. The most common formats were program-to-date summaries and one-time post-program surveys of participants, teachers/ artists/scholars or parents.
The findings were primarily short term and qualitative. They focused on program dimensions and accomplishments, or participant behavior and self-perception. Few provided quantifiable information on changes in the status of children and youth. Testimonial evidence, however, should not be undervalued as prelude and complementary to more quantitative approaches. At present, the cost and time involved
in quantitative research
is often prohibitive for these programs.
A handful of studies have begun to document the positive relationship between program participation and cognitive development, motivation to learn, organization, self-perception and resiliency.
For instance, the Wolf Trap Institute for Early Learning Through the Arts evaluated children in Head Start preschool programs with 7-week arts residencies in Arizona, the District of Columbia and Tennessee. It reports that preschool children in these residencies, when compared with children in "regular" programs, progressed in two known antecedents to learning-student engagement (the ability to focus attention and participate in ongoing activities) and social participation (the ability to be sensitive to others and to work in group settings).1
The Kaleidoscope Preschool Arts Enrichment Program at the Settlement Music School in Philadelphia found that their at-risk preschool children made gains in cognitive development, language development and achievement above and beyond what was seen in a control group of preschoolers not in an arts-enriched program. It reports the gains were maintained as continued time was spent in the Program and, in all cases, remained above the level that was achieved by the control group
in 2 years.2
Project Self Discovery at The Cleo Parker Robinson Dance Theatre in Denver is a national demonstration program using artistic engagement as a platform for transforming the lives of youth in trouble from drugs, alcohol and crime. In this after-school program, creative self-expression, artistic mentoring, clinical case management and rigorous recreation are combined to enhance self-esteem, stability and socialization. Using adolescent self-reporting and observation as evaluation tools, the program documented improvements in young people's resiliency skills for managing problems and situations. When compared with program dropouts and a comparison youth group, the participants maintained a higher level of involvement in artistic activities, functioned better in school and used a higher level of cognitive processing in thinking about and dealing with
life experiences.3
The MOTHEREAD, Inc. Program for Incarcerated Mothers in North Carolina, one component of the North Carolina MOTHEREAD/FATHEREAD literacy program, uses children's literature to enhance mothers' literacy and parenting skills. Participants learn to read effectively to their children, identify child development themes underlying the stories and relate those themes to their own lives. The researchers have documented improve-ments in the emotional health of the participating mothers. At the end of the program, the mothers reported fewer symptoms of depression and negative self-esteem and more control over their lives. These participants also became more understanding of their children's needs and less likely to expect their own needs to be completely satisfied by their children.4
Staff at these four programs would be the first to point out the methodological limits of their work, including one or more
of the following: small sample size; no comparison group or one that was not comparable to the test group in all respects; an inability to separate the impact of the arts or humanities component
of the program from
the impact of other factors; little or no follow-up over time.
1 Torff, Bruce, 1994, "Evaluation of Wolf Trap Institute for Early Learning Through the Arts: Annual Reports l991-92, l992-93," Harvard Project Zero, Cambridge, MA, as summarized in National Endowment for the Arts and Morrison Institute for Public Policy, l995, Schools,
Communities, and the Arts: A Research Compendium, National Endowment for the Arts, Washington, DC, pp. 38-39.
2 Coakley, Mary C., l995, "Making a Difference in the Lives of Children: Final Report of a Four-Year Evaluation of an Arts-Based Early Intervention Program,"
Settlement Music School, Philadelphia, PA.
Although encouraged by the findings, the researcher added the following caveats: (a) The number of subjects was fairly small, and the group sizes were unequal, which may have weakened the ability to detect differences in the control group; (b) the teachers at the Title XX-funded center had less formal education than the teachers at Kaleidoscope, which may have produced some differences in the two groups; (c) the Kaleidoscope Program had a more generous funding base, which may have provided children and staff greater opportunities.
3 Milkman, Harvey, Kenneth Wanberg, and Cleo Parker
Robinson, 1995, "Project Self
Discovery: Artistic Alternatives for High Risk Youth," The Cleo Parker Robinson Dance Theatre, Denver, CO.
4 Martin, Sandra L., Niki Cotten, Dorothy Browne, Lawrence Kupper, Brenda Kurz, and Elizabeth Robertson, l993, "Evaluation of the MOTHEREAD Program,"
University of North Carolina at Chapel Hill, Chapel Hill, NC.
The researchers noted a positive impact on the participants, but noted the following limits of the study: (a) There lacked a "no treatment" comparison group, making it difficult to separate the impact of the program from other factors; (b) attrition over the course of the evaluation was considerable, leaving unanswered its effect on the findings; (c) the sample size was small, thereby limiting the statistical analysis that could be applied to the data.
Footnotes