Wicked Problem Project
Posted in CEP 812, Education, student achievement, technology, Uncategorized on March 6, 2012 by Bill SwartzFor our final project for CEP 812 I created 2 parts: Part 1, a fun YouTube video and script that can be considered the movie version of the book and Part 2, the War and Peace novel of Wicked Problem Projects. It is probably not a good practice to begin with an apology but, fair warning, the text portion will require at least a refill on your favorite beverage.
To view the You Tube video of Quest to Assess, Please click on the link below:
Part I
Act I
The Quest to Assess
Once upon a time there was a preschool administrator, Sir William, who had a Wicked Educational Problem. The Valiant Teachers in his kingdom used an online assessment program that had become a Monster. It demanded that teachers feed it data about 38 Learning Objectives from their curriculum, which was a good thing, but the Monster made life difficult for Sir William and the Valiant Teachers in several ways.
First, the Monster made the Valiant Teachers collect their data in a really old fashioned way that was slow and tedious and like using stone tablets. The Valiant Teachers were not having fun, did not like the Monster and wanted it to go away.

In addition, the Monster was an expensive pet. It required a high annual ransom fee that never went away and went up a few potatoes every year. The Monster was eating up resources that were desperately needed in other parts of the kingdom.
Also, the Monster was very difficult to get along with. If Sir William brought a problem to the attention of the Monster, the Monster would look at the problem and then make it worse. When Sir William called the Monster’s Mother on the phone, she seemed to delight in passing his call from one friend to another. The friends would all seem to want to help but often they didn’t seem to know what to do and would tell Sir William that they would “check into it” and “get back to him.” Sir William was very frustrated.
Worst of all, once teachers had fed the Monster what it wanted, Sir William and the Valiant Teachers did not make the Monster do any work for them. The Monster could have told them all kinds of interesting things about their students, their classes and their preschool program as a whole but it didn’t. The Monster was silent. In truth, neither the Valiant Teachers nor Sir William asked much of the Monster–but they were busy people and thought that by paying the Monster, their problems would be solved and that the Monster would do all of the work for them.
Finally, Sir William cried out, “Enough. We can do better on our own without the Monster!” and, with a bold stroke of his fiscal sword, cut off the head of the Monster.

This was the beginning of Sir William’s Quest to Assess.
Act II
The Templates of Excelsior, Ipads, & PLC
For many a cold, sleepless and long wintery night. No
For many a cold, sleepless night. No
For many a cold night. No.
Alright, Sir William wandered in the wastelands searching for the Holy Grail of Assessment programs for about 2 minutes. He looked neither to the right nor to the left he did not look, for all along the way he had decided to call upon the powers of Prince Excelsior to carry the weight of assessment for his Valiant Teachers. He knew that Prince Excelsior was a spreadsheet of great renown and had powers to create magical formulas that would make data entry a breeze. Sir William would ask Prince Excelsior to use his magic powers to create a new order that he would call, “The Round Table of Templates Excelsior”.
As he hurdled headlong in haste on his journey to hire the Prince, Sir William came upon a curious place. It was the Great Hall of Learning and sitting on the steps was the wize Wizard/Witch White.
“That which you seek is a good thing,” she said, “but know that before you choose Prince Excelsior, he must pass the Terrible Test of TPACK or you and all your Valiant Teachers will be stranded in the Land of Mediocrity for all time.”
Prince William was puzzled and admitted that he had need to relearn the ancient language taught to him so many semesters ago. He learned that the test was more about the knowledge he and his Valiant Teachers had than that possessed by any off the shelf assessment program or spreadsheet, like Prince Excelsior. Sir William discovered that he and The Valiant Teachers must know knowledge in 3 areas for Prince Excelsior to be of any assistance.
First, they must learn the technology of Excel and be comfortable with using it for data entry. The formulas they did not need to know, nor the inner computations. They would, however, have to be comfortable enough to enter data, modify it and do simple operations like cutting and pasting.
In addition, Valiant Teachers would need to have a deep understanding of the vast accumulation of knowledge about all things Early Childhood. They must know about science, math, language, and other subject matter, themes, vocabulary, and other wisdom that was used in the preschool classroom. This was no small matter as the Valiant Teachers must be able to explain important things to 4 year olds. Things like why an “M” looks like a “W” and why Criss-Cross Apple Sauce was the best way to sit at Circle Time. They would have need of these things in order to assess their students. This knowledge is known as the Great Content.
Finally, The Valiant Teachers must have knowledge about Pedagogy, that mysterious word that means:
ped·a·go·gy
[ped-uh-goh-jee, -goj-ee] Show IPA
noun, plural -gies.
1. the function or work of a teacher; teaching.
2. the art or science of teaching; education; instructional methods.
Origin:
1575–85; < Greek paidagōgía office of a child’s tutor. See pedagogue, -y3
When Sir William translated this word into meaning for this case, he came to understand that the use of the Templars of Excelsior would have the potential to make changes in what The Valiant Teachers did in the classroom.
Sir William was thrilled to discover that Prince Excelsior was indeed a worthy part of the solution to solving his Wicked Educational Problem and so he ran home to get busy creating the Round Table of Templates Excelsior.
The implementation of Sir William’s solution would involve 3 phases. First, after The Valiant Teachers were comfortable with the services of Prince Excelsior, they would need a new way to enter their data in a way that would not require them to enter it twice (the stone tablet way) as they had in the past. As it happens, Sir William, was an Apple Fan Boy of many years and knew just the tool. Armed with an Ipad in each classroom, the Valiant Teachers would become mobile assessment machines no longer tethered to their desktop PCs.
Second, although the Ipad was a great tool for playing Words with Friends, it would be very difficult to collect data directly into one of the Templates of Excelsior. Nevertheless, Sir William had heard a growing cry in the land, “There’s an app for that!” and decided that was just the thing he needed. With an Ipad and an app to collect the data with, The Valiant Teachers would once again call Sir William – The Most Wonderful, Magnificent & Noble.
The final piece of Sir William’s solution to the Wicked Educational Problem will be to use the data collected to make progress with students. The vehicle to do this will be through the practice of a Professional Learning Community (PLC). When all these things have been completed, there will, no doubt, be joy in the land and . . . world peace.
Read Sir William’s blog entry below to see how The Quest to Assess has turned out so far.
Part II
POSITION STATEMENT
Early Childhood Curriculum, Assessment, and Program Evaluation
Building an Effective, Accountable System in Programs for Children Birth through Age 8
A Joint Position Statement of the National Association for the Education of Young Children (NAEYC) and the National Association of Early Childhood Specialists in State Departments of Education (NAECS/SDE)
Adopted November 2003
The National Association for the Education of Young Children and the National Association of Early Childhood Specialists in State Departments of Education take the position that policy makers, the early childhood profession, and other stakeholders in young children’s lives have a shared responsibility to:
. . . make ethical, appropriate, valid, and reliable assessment a central part of all early childhood programs. To assess young children’s strengths, progress, and needs, use assessment methods that are developmentally appropriate, culturally and linguistically responsive, tied to children’s daily activities, supported by professional development, inclusive of families, and connected to specific, beneficial purposes: (1) making sound decisions about teaching and learning, (2) identifying significant concerns that may require focused intervention for individual children, and (3) helping programs improve their educational and developmental interventions. (DAP, p.1)
Rational:
Assessment is one of the pillars of student achievement and an important part of any educational setting. If you do not measure progress, growth, and student achievement in some way, then it is difficult to say whether your educational program is effective. The preschool program I oversee assesses student progress through the use of 38 Objectives for Development. In the past, the teachers used a paper and pencil system that observed growth in these objectives. Although it was an effective tool for collecting data, teachers and administration were not using the data effectively for making adjustments for instruction, student learning, or identification of professional development needs. Nor was it used to provide any concrete evidence of program effectiveness or quality. In addition, the data was not easy to share with parents or caregivers. When the company developed an online system for assessment, we were excited at the idea of being able to use technology to manage our data collection and the resulting reports we would have access to. Unfortunately, it was a frustrating experience providing more evidence that the use of technology, in and of itself, is not always the answer. Support for the online access was extremely poor and hours of time on and off the phone did little to improve either the problems we were having or limitations we felt the product had. For example, reports for parents and caregivers were difficult to explain and used an excessive amount of paper when printing. In addition, the cost of the subscription for the online assessment system became increasingly prohibitive as our financial situation was stretched. Last spring we decided not to continue the online assessment system and our teachers currently gather data using their paper and pencil methods.
Technology Solution Proposal:
My proposal is to use technology to improve the way we do assessment in the preschool setting. Specifically, I would like to be able to provide a method for teachers to collect and use the data from our assessments to impact student learning. By providing a better way of collecting information, teachers can use the data for their own instructional purposes and it can also be useful for staff, administration, parents and caregivers. In addition, it will add to the increasing demand for evidence that our program is effective. This proposal is not, however, about just using a nifty technology tool to collect data. As I have hinted here, the big picture view is to make an impact in student learning.
The problem is that assessment information is of limited value unless teachers understand what it means and know how to use it to guide children’s learning. Too often assessment results are seen as an end product rather than as knowledge that opens the door to learning about each child and to planning meaningful curriculum. (Beyond Outcomes, p.10)
The overall and long term goal, then, of this proposal is to change the way we use assessment in our program.
My technology plan is to create a series of template spreadsheets using Excel and/or the spreadsheet in Google Docs. The first template will copy the format the teachers currently use on paper but can be changed if additions or subtractions are desired. These templates would allow data collection for individual students and classes and are the foundation piece for teachers to use in evaluating their instruction and student learning. Additional templates can be created from this form that would be in the form of graphs and charts to better illustrate the data.
In addition, a family friendly report card using data from the Objectives will be created. This would allow teachers to more easily explain their assessments using some of the graphs and charts. Anyone who has worked with a standards or objectives based report card knows that the language can sometimes be daunting for the uninitiated and, for many of our parents, this is their first exposure to school as a parent.
Another master template would also allow the aggregation of all the data into a program overview. From this program overview template, various graphs and charts could be created to give useful ways of looking at the data as a way to provide evidence of program effectiveness.
Finally, a future step would be to make the data collection a one step process. I believe our teachers will all have Ipads by next Fall, so I would like to explore creating an Ipad app that would allow teachers to use a tablet to enter their data into these templates.
Benefits:
- Free. There are no upfront or ongoing charges or costs. No ongoing subscription or maintenance fees. The only investment is the time and energy to develop the templates.
- Creation of reports and data sharing will be easier. Teachers need to spend their time working with students while they are in the classroom. Data collection needs to be easy and seamless. The less time it takes to enter and collect data, the more time teachers will have to spend on other things.
- Powerful. By using a spreadsheet we will be able to use the data we collect in a variety of ways. Templates for charts and graphs can be created for both teacher and administrator use. Family friendly templates for reports are possible.
- Collaborative. By making the program overview template available in Google Docs, it can become a document that is accessed by anyone who is invited.
- Accessible. Because the template will be a Google Doc, it will be accessible from anywhere there is an Internet connection.
- Downloadable data. Since the data will be in digital form, it has the potential to be used in a variety of ways that paper and pencil forms cannot.
In summary, there are three compelling reasons why I believe this technology proposal would be important to undertake. First, “the most powerful outcome of ongoing assessment is the positive relationships teachers can build with each child. (Beyond Outcomes, p.15) Knowing the individual strengths and weaknesses of their students will make teachers better. Helping my teachers collect and use the data from their assessments in an easier more effective way will have a lasting impact.
Second, “A system of ongoing assessment also helps teachers build a relationship with each child’s family. (Beyond Outcomes, p.16) Being able to talk with parents about their students’ progress in a form that is understandable lays the foundation to develop a solid home/school partnership.
Finally, as a very last step in the assessment process, the information teachers have gathered and used to plan curriculum and to support each child’s learning can be used to report on group progress. As advocates for developmentally appropriate practice, it is important for early childhood professionals to share what children are learning with those outside their programs. This information may be shared with many stakeholders: administrators, government officials, funding organizations, families, and the general public. (Beyond Outcomes, p.16)
For a number of years now, accountability for student, teacher and school has been increasing in education. Teacher evaluation is now being tied to student achievement. The trend in student achievement testing has been slowly moving downward to lower grades. MEAP testing now begins in the third grade and there are calls to test every student every year. Early childhood educators and organizations have not been ignorant of the need to account for program efficacy and have been on the forefront of developing ways to do so. By being proactive in creating a better assessment system for my preschool, I believe we will also have a tool that will help with our stakeholders.
This is definitely an ambitious proposal and goes way beyond the scope of anything that may be accomplished during the short period of this course. However, it will be of lasting impact as it is fully implemented. I envision being able to create several of the templates that will be needed and perhaps begin data collection using them. In addition, I will research the feasability of an Ipad app. The real work will be transitioning to using the data to impact student learning, teacher instruction and program efficacy but cannot really be started until the systems are in place and teachers are comfortable using them.
Application of TPACK
My proposal to use technology to improve the way we do assessment in the preschool will produce a positive impact by changing what content teachers teach and how they teach it. By using Excel to move our assessment data to a digital form, it will speed up the process of analyzing where a student or class or program is at in each of 38 Objectives for Growth.
I see three phases of TPACK application for this project. First, teachers will need to be technologically knowledgeable in how to use Excel. Although they will have templates to enter their data into, they will still need to understand how the electronic spreadsheet works and the kinds of things they can do with it. The move from a paper and pencil form to an electronic spreadsheet is the integration process of technology: using technology to speed up and enhance what we are already doing. Although this step would be an improvement, it is not where the real power of technology lies for teachers.
Teachers will also need to be knowledgeable in their content and pedagogy and how the technology can make an impact in those areas. Having digital data will allow teachers to view the data in ways that they have not been able to up to this point. By understanding more quickly where students are not progressing, teachers can focus on providing better instruction for a particular content area. This might mean some individual work with a student or as a group. It could mean a change in how a teacher provides instruction for a content area. It will also help identify areas of strength. For example, a teacher who is very effective with students in a particular objective can share what she is doing instructionally with her colleagues. This begins to move into the innovative stage where teachers are intentionally changing the way they teach in order to produce better results when students are not making progress.
Also, as a program we will be able to see areas of strength and weakness. If we have low scores in a particular area, we can address it. It may be that professional development as a group or for individual teachers is in order. It can have implications for professional development. Finally, being able to report these results in a better more efficient way can have an impact on national reports of effectiveness.
Implementation
The ultimate goal of My Wicked Problem Project is to change the way we do assessment in the preschool I work in. The first step of creating a template spreadsheet for data collection, although labor intensive, may actually be the easy part of the project. I created a spreadsheet template with 38 Objectives for Development to enter and record the data that teachers use for our curriculum. I was able to create the entire spreadsheet with formulas to do the calculations and also to format it so it fits nicely on 8.5 x 11 inch paper. That was important to minimize the number of pages required for our reports. The main page of the spreadsheet is the template so there is always a master to use that doesn’t have data in it. On page 2, I copied the template again with the addition of the teachers names and am/pm classes on it to use as a data collection tool. Page three is another copy of the template where the accumulated totals can be entered for all 5 teachers. In addition, I am working on creating the form that we will share with parents to give them some feedback on where their student is.
In our program, we collect data 3 times during the year so we already had data from the Fall. I was able to enter the data into the spreadsheet and I’m pretty happy with the way it looks. The data entry is smooth and it’s nice to have the spreadsheet do the calculations without having to add up rows of numbers.
One surprise was in how long it took to create this template. I spent a lot more time than I thought I would on things like making the text fit into boxes, formatting text to run vertically instead of horizontally for the headers, and creating the formulas for calculating.
There were a couple of unexpected bumps in the road and issues that will still need to be resolved. First, I need to address how to enter and calculate a number for a student who is not scored for a particular objective. As we entered the data, we saw that there were some students who were absent or for other reasons did not receive a score for a particular objective. I need to figure out how to enter a null value for those scores so the total calculation takes this into account.
Also, the assessment numbers for each objective are on a continuum and scores range from 0-9. The ideal number for each objective, however, is different. For example, a student might be expected to be at a 9 level in a physical objective like walking where as in another objective like counting, a 5 or 6 would be age appropriate. We’ll still need to come up with a formula that figures out the ideal totals for the overall student score.
Findings and Implications
I knew when I began my Wicked Problem Project to change the way we do assessment in the preschool I work in that it would be too large in scope to be able to implement completely before the end of class. One of the things I have enjoyed about the Educational Technology classes is that what I learn in class relates directly to my work setting and makes a difference in the real world, not just the classroom.
At this point, I have completed the spreadsheet template and just have a few formulas to tweak. Data from this past Fall has been entered and I have had some time to look at some of the results. We have another data collection point coming in mid March, so there will soon be two sets of data to make some comparisons with. I anticipate that the real work of implementing change will begin next fall as I work with teachers to use the data in our classrooms. We have really been working hard in our District to make Professional Learning Communities (PLC) effective and this will be a much needed tool to to help with the process.
One thing I will do differently as this project progresses is delegate responsibility for parts of the project to others. Although I think it was important for me to enter the data from last Fall so that I could have a good understanding of the template and the work it takes to do the data entry, I will have teachers or their paraprofessionals do the data entry in the future.
One lesson I think that is important is that effective delegation can make the project even better than what I would be able to accomplish on my own. I found it helpful to get some feedback from teachers about the template for reporting to parents. Their input resulted in a number of changes to the design of the report and made it a lot better. One of the next things I would like to do is have an Ipad application for data collection. I have researched building apps enough to know that I will pursue having someone who does it for a living create the actual app. I also have the benefit of having some human resources that can make this project even better. Our office has a Director of Data, who will be able to help with the formulas I need help with and suggestions for the various reports we would like to have from the data.
I think this project has given me confidence to apply technology to other Wicked Problems I may encounter in my job in the future and I doubt that this will be the last Wicked Problem Project I work on.
References
Beyond Outcomes. How Ongoing Assessment Supports Children’s Learning and Leads to Meaningful Curriculum. Spotlight on Young Children and Assessment, 2004. Diane Trister Dodge, Cate Heroman, Julia Charles, and Jessica Maiorca.
Copple,Carol and Bredekamp, Sue, editors. 2009. Developmentally Appropriate Practice in Early Childhood Programs Serving Children from Birth through Age 8. 3rd ed. Washington, DC: NAEYC.
Dodge, D.T., L.J. Colker, & C. Heroman. 2002. The Creative Curriculum for Preschool. 4th ed. Washington, DC: Teaching Strategies.
Epstein, Ann S. 2007. The Intentional Teacher. Washington, DC: NAEYC.
Early Childhood Curriculum, Assessment, and Program Evaluation: Building an Effective, Accountable System in Programs for Children Birth Through Age 8 (Position Statement). 2003. National Association for the Education of Young Children (NAEYC).




