Traditional Evaluation Tools Limit Arts-In-Ed Assessment

Patti Chilsen, CWP’s Assistant Program Director: Training + Evaluation, facilitates pre- and post-arts residency surveys of CWP classroom students. At one of our partner schools she encountered a resistant student which brought up important issues regarding evaluation.

The complex and relational nature of the work we do is very difficult to assess, and often the tools we are given to measure our progress do not suit our circumstances. – John Bare: Philanthropy, Evaluation, Accountability, and Social Change

John Bare beautifully describes the traditional logic model as a string of dominoes, where he states that when we knock one over, we will get a chain reaction that knocks over every other perfectly lined-up domino in our “If! Then!” theory of change. However, we all know that we do not live in a linear and simple world. Bare challenges us to think of our world rather like the game of pick-up sticks, where our interconnections are messy and where it is virtually impossible to touch one item without all other aspects being affected. … We work in complex worlds, and what we attempt to see and evaluate is often the “in-betweenness” of things. … As a field we are now challenged to build better tools that can give us better feedback to match the complexity of our systems and the issues at hand.

I was facilitating CWP post-evaluations in a third grade class when a young boy quietly sidled up beside me. Shyly, and somewhat glumly, he handed me his survey. I said thank you. Something about this boy’s survey caught my eye. I noticed that he had written “nothing” in all the qualitative response slots. He had also circled all zeros in the quantitative responses, and written only one sentence for the essay response.

I knelt down by his desk and asked if he wanted to review his responses. He shook his head no. I asked if he was upset. He nodded his head yes. I asked if he was upset over Community-Word Project and he shook his head no. I asked if it was something outside of school. He nodded his head.

I told him that art can give us a way to show our emotions, even ones we feel when we are upset. When I had arrived earlier, I noticed he had been coloring a sketch of the class mural they were creating, so I mentioned how coloring is one way to express emotions. Brimming with tears moments before, now his dark eyes sparkled. “Can I color the picture on the survey?” he asked eagerly. I automatically replied, “Yes!”

Neither words nor numbers define the limits of our cognition; we know more than we can tell! We need art forms to say what literal language cannot say.

I felt sure I would get a picture colored black or deep red or purple from that young boy. When I picked up his paper, he smiled at me. He had colored the picture one vibrant shade of sunlight yellow.

Educators are consumed with measuring student success and student growth. We are compelled to somehow grasp on paper the significance of our impact in the classroom. At the same time trapped by traditional routes of measurement for what often evades typical assessment.

An evaluation that focuses exclusively on the mechanics of an arts program may leave a significant gap in our understanding of its full value and impact.

Administrations and funders clamor for data that proves we are reaching and teaching our students. I get it. We have to know we are doing what we say we do. Assessing our impact can also show how to improve our effectiveness. We must be accountable and able to reveal that our work provides focused instruction with lessons targeted on the growth and development of the children.

What we need is a movement that challenges the orthodoxy of “domino” views of the world and more realistically sees the “pick-up sticks” world in which we live. We need tools that can better help us to view our worlds and we need database systems that are up to the challenge of recording and reporting on the things that matter. We need support for these efforts. We need to creatively design tools and systems that can dynamically inform our efforts, ones that are committed to the outcomes but also have the flexibility to help us “adapt and adjust in terms of what it will take to produce the desired impact. – John Bare

Many educators are working to develop systems that show how deeply the arts build not only the ability to express but to think creatively, and that art-making involves myriad critical thinking skills. Quality arts integration reaches essential aspects of the learner, providing opportunities to:

• express, and convey complex, intangible ideas
• analyze and “read” not only what is expressed but how it is conveyed
• compare and contrast
• make choices and work to implement them
• alter the course when things go wrong, or not quite as you had planned
• rethink, revise, and edit
• explore and push beyond the edges of what is known at the moment
• assess and self-evaluate in the process, which impacts end-product
• reflect and grow, respond, communicate and create

This is how arts in education intertwines with college and career readiness.

A measurable effect on instruction: When teachers and teaching artists collaborate in the delivery of Right Brain residencies, students engage in using 21st century skills (e.g., creative and critical thinking, collaboration, etc.) nearly twice as often compared to ongoing classroom instruction.  In addition, during residencies, children learn to think and act like artists (e.g., using materials and techniques carefully and in new ways, expressing their personal ideas and experiences, developing a unique style, etc.) throughout the sessions – strategies that are largely missing in academic instruction.

Traveling from the classroom back to my office, I thought about how those zeros and nothings were going to affect my averages. I thought about how this system did not reflect that young boy’s experience with Community-Word Project in his classroom. I thought about his sad face and brooding eyes. And how they brightened at the prospect of coloring the image on the survey. How was I to record what that beaming yellow color he used meant to this student? How was I to record what he felt about his experience? His love for arts in his classroom? His ability in a dark moment to express a joyful feeling?

Can we measure joy? Yes we can. And by knowing where and how to look for it, how to document what we see and hear, and how to communicate that, (we) can create opportunities for more joy more often. – Measuring Joy, by Deborah Bedwell, Executive Director, Baltimore Clayworks

“The only man who behaves sensibly is my tailor; he takes my measurements anew every time he sees me, while all the rest go on with their old measurements and expect me to fit them.” – George Bernard Shaw