The DISCOVER Assessment—What Is It?

     The DISCOVER Assessment is a measurement of abilities unlike any other, a research based instrument founded upon multiple intelligence, problem solving and diversity (click to see details of how these three components have shaped the Assessment).  Participants are guided through active, hands-on problem solving exercises (using toys and other age-appropriate materials) designed to measure problem-solving abilities.  Problems & challenges presented range from simple and closed—requiring convergent thinking—to open-ended and complex—requiring divergent thinking.  As participants solve the problems, certified Observers take careful notes, recording which of approximately 120 “superior problem solving skills” are observed.  This information is later compiled according to the respective intelligences and is used to create “Strength Profiles”, reports that show the levels of strength for all the intelligences. Strength Profiles explain and expand upon the Assessment results, and can be used to improve learning or (for older students) guide selection of higher education topics and careers.

     Important characteristics of the DISCOVER Assessment:

• Non-Biased Format—The Assessment was developed from research involving numerous age groups, cultures, languages, geographic locations, and ethnic identities.  Its corresponding design is applicable and accurate in all languages and cultures, virtually eliminating the ethnic, cultural, and linguistic bias prevalent in many current instruments.

• Performance-Based—Rather than relying on paper-and-pencil tests, the Assessment measures performance over a broad perspective of potential abilities, using a range of materials.

• Intelligence-Fair (Gardner, 1992)—In the DISCOVER Assessment, each Intelligence is measured in a way appropriate to its own characteristics, rather than being filtered through another intelligence.  This component is critical to an accurate assessment of abilities and unfortunately is not present in many traditional instruments.  Consider, for example, a math exercise that is presented as a word problem in English.  A child with a dominant language other than English might struggle with the language component, thus masking his or her true ability to solve the problem.  In this case the ability actually being assessed is linguistic, not mathematical, producing potentially skewed or erroneous results.  For accuracy, each intelligence must be assessed (as much as possible) in a manner free from dependency on any other intelligence.

• Criterion-Referenced vs. Norm Referenced—Traditional assessment and testing instruments, generally, are Norm-Referenced, meaning that participants are compared to a large sampling of other participants and are rated on a percentile of right answers (e.g. if a participant tests at the 60^th percentile, he or she answered more right answers than 59% of other participants who have taken the same test.)  The main disadvantage of this approach is that the results are too general.  Simple comparisons of one student to all other students who have taken the test obscure valuable information about individuals.   By only measuring right answers, much useful information is lost, such as what strategies were used and what was the reasoning behind choosing one answer over another.  When testing a student’s knowledge, often we are assessing exposure, not the ability to learn information.  The ability to learn the information is the key.  The DISCOVER Assessment, on the other hand, is Criterion-Referenced, meaning that participants are “scored” according to the number of superior problems-solving skills they exhibit (while pursuing the “right” answer or while solving a problem that has no clearly defined “right” answer).  This difference is important because real-life situations seldom reduce to clear wrong or right answers.  Rather, such situations are usually a matter of finding the “best” or “most appropriate” answer.  Focusing on effective problem solving tends to “level the playing field” and displays a more accurate picture of innate strengths beneath the levels of knowledge exposure.

• Standardized—The Assessment is standardized in its implementation.  Specifically, the directions and procedures always follow the same format, regardless of where implementation occurs.  However, standardization does not equate to a lack of flexibility or adaptability.  Within the standardized framework, many components can be customized to better integrate cultural applications (e.g. local or regional toys can be substituted for the toys normally used in the Oral Linguistic component, providing a more comfortable and applicable experience for young participants).

• Future Oriented—Unlike most tests, the DISCOVER Assessment results are useful in both the long and short term.  For example, with a student it can provide a framework for improving the effectiveness of future school curricula and the home learning environment.  By measuring open-ended problem solving skills and creativity, the Assessment is measuring more than just knowledge, or the ability to “pass tests”.  It measures the capacities needed for success in life.  Developing these problem-solving abilities leads to life-long enrichment.

Why is the DISCOVER Assessment needed?

     One of the most troubling issues confronting educators today, especially teachers of the gifted, is the growing realization that traditional testing methods (e.g. I.Q. Test, Raven, DCAT) often fail to properly identify children’s potential and fail to accurately predict success in life.  These methods, created decades ago in absence of modern theories on learning and brain functioning, do not differentiate between capacities we now recognize as being important.  This is especially true for children whose dominant language is something other than English, or for children from a disadvantaged or culturally different background.

     A case in point is that Native American populations tend to show significantly higher Spatial Intelligence than do Anglos; yet it is not uncommon for schools with predominantly Native American populations to have no gifted program because none of the children score anywhere close to “gifted” levels on traditional tests.  It likewise is not unheard of for a school containing 90% African Americans and 10% Anglos to have the opposite ratio in its gifted program.  Many years of DISCOVER research have shown that these situations, and others like them, are unwarranted and that, sadly, the traditional educational structure is squandering a great deal of natural ability and potential, especially among “at risk” populations.   

     The DISCOVER philosophy incorporates the belief that overall ability is distributed relatively equally across diverse groups.  This is not to say that all groups have the same abilities.  On the contrary, differing ethnic and cultural groups do tend to show consistently different patterns of strengths, involving different intelligences.  But when all the categories of intelligence are considered together, the numbers of individuals with, for example, “high ability” is practically the same across diverse groups.  The importance of this fact cannot be over emphasized.  Our research has shown that if the ethnic percentages in a school’s program for the gifted do not reasonably reflect the ethnic percentages of the school-wide population, something is very wrong—and some high ability students are falling through the cracks.

     The DISCOVER Assessment was created (and is being continually modified) using tens of thousands of participants from diverse cultural, ethnic, and language backgrounds, in an attempt to eliminate, as much as possible, any barriers to accurate identification.  By cutting through linguistic and “exposure” filters, DISCOVER instruments work equally well in very different environments, measuring innate abilities that can then be further developed.  We believe schools have an imperative to seriously question the usage of traditional paper-and-pencil tests that provide little useful information and often discriminate against individuals who do not fit the typical “dominant culture” mold.

How Was the DISCOVER Assessment Developed?

     The framework of DISCOVER was designed to create a better alignment between the definition of problem solving, its assessment, and its development in an educational context.  In the early research of psychologist Mihalyi Csikszentmihalyi (Getzels & Csikszentmihalyi, 1976; 1967), the ability (and willingness) to structure an open-ended or ill-structured problem, or “problem-finding”, as it was later labeled, was the single trait that most accurately predicted the later creative achievements of artists. This research has had a significant effect on the field of education for gifted students, leading to the development of numerous teaching models in which problem-finding is valued over the solving of already-defined problems or problems with known solutions (Gallagher, Sepien, & Rosenthal, 1992; Maker & Nielson, 1995).

     This framework was used by the DISCOVER research teams to create “intelligence-fair” (Gardner, 1992), developmentally-appropriate problem solving tasks that were engaging to individuals of varied ages, abilities, languages, ethnicities, and environmental backgrounds.  Observers (also from varied ages, abilities, languages, ethnicities, experiential and theoretical perspectives) watched and documented the problem solving strategies and characteristics of products created by the students.  After each observation, Observers were asked to tell which students were “effective, efficient, economical, or elegant” problem solvers,” and to provide detailed descriptions of the superior behaviors they observed that led them to nominate each student.  These behavior descriptions were recorded and maintained.

    As a result of the process described above, data from observations of over 5,000 children (Maker, 1996, 1997) and extensive case studies of competent and highly competent children and adults with varied ability patterns (Maker, 1993) were used to develop a checklist of observable behaviors that could be used to guide decisions made about problem solving behaviors in students, grades K-8.  A similar process has been followed with a smaller (but increasing) sample of high school students (Maker, 1994).  Approximately 120 problem-solving behaviors have been documented at present.

     Repeated assessments, revisions, feedback, and on-going data collection have resulted in a set of activities for each of five grade levels (Pre K, K-2, 3-5, 6-8, 9-12). DISCOVER staff members also have developed standardized procedures and directions, a behavior checklist to provide consistency in evaluations, and a “debriefing” process for increasing interrater reliability.

What Are Its Components & How Is It Done?

     The DISCOVER Assessment has several possible formats.  Pre-school Assessments (ages 3-5) are always conducted one-on-one, with the Observer leading the child through various exercises, usually in the child’s home or in another familiar environment.  Assessments of children six and older (as well as of adults) can be conducted via a one-on-one format, or in a group setting for increased cost effectiveness.  Group Assessments still provide results for each participating individual, but in the context of an Observer watching a group of 4 or 5 individuals simultaneously performing the same exercises.  Group Assessments of students typically are conducted in the normal classroom but can be conducted in any familiar environment.  Different Assessment formats exist, for pre-school, grades K-2, grades 3-5, grades 6-8, high school, and adult.  The adult Assessment has no upper age limit.  A proposed future project will develop a version of the adult Assessment specifically for professionals, designed to measure work-related abilities.

     In some cases, DISCOVER staff conduct the Assessments (either in Tucson or on-site in the respective community).  In other cases, Assessments are conducted by local school personnel or community members who have completed a training and certification course offered through the DISCOVER Projects Assessment Training Program.  Implementation using local personnel usually is more cost effective, especially when the number of students to be assessed is greater than 50.  For large-scale, ongoing assessment needs, DISCOVER staff personnel can train and certify experienced local school or community Observers to be trainers of other local Observers, increasing cost effectiveness even further.

     A large majority of Assessments are conducted in a school-based setting.  These Assessments occur in the familiar classroom environment, with one of the regular teachers acting as facilitator and classroom organizer, in cooperation with trained DISCOVER Observers.  The teacher helps maintain discipline and normal classroom routine, allowing the Observers to focus on their prescribed activities.  The Observers function as a team and often operate at a district (or regional) level, traveling from school to school throughout the school year.  The background of these Observers varies.  Some are local classroom teachers, while others are specialists in education of the gifted, bilingual or special education experts, pre-service educators, counselors, community members, retired teachers, administrators, or other experts—as needed.  All Observers must be thoroughly trained and must be certified yearly by DISCOVER personnel before being granted permission to conduct Assessments.  In cases where dominant languages other than English exist, the team includes bilingual Observers and a teacher or spokesman who can present instructions in the dominant language(s) of the students.

     Procedures and components vary according to which type of Assessment is being conducted.  Children participate in five activities: Spatial Artistic, Spatial Analytical, Oral and Written Linguistic, and Mathematics.  The result is the determination of strengths for each child in the areas of Linguistic, Spatial, Logical-Mathematical,

     Interpersonal and Intrapersonal Intelligence; additionally, creativity traits are delineated (e.g. fluency, flexibility, elaboration, and originality).  In general, children participate in five activities, resulting ultimately in the measurement of Linguistic, Spatial, Logical-Mathematical, Interpersonal and Intrapersonal Intelligence, as well as traits of creativity (e.g. fluency, flexibility, elaboration, and originality).  The regular classroom teacher administers two of the activities, usually a week or two in advance of the observation team’s arrival.  The observation team members analyze the resulting products at a later date.  One of these activities is a math worksheet that measures certain components of Logical-Mathematical Intelligence.  The other is a story-writing activity that measures Written Linguistic Intelligence.  The remaining three activities often are run back-to-back on a single day, in the presence of the observation team.  (Some schools for upper grades spread the activities over three days).  The first of these measures Spatial Artistic ability.  Students receive bright-colored, multi-shaped, pieces of cardboard that they use in construction exercises, progressing from simple problem solving up to open and complex thinking.  Observation team members look for many different qualities, such as enjoyment, complexity, and clear resemblance.  In the high school Assessment, a parallel activity uses clay, pastels, and mechanical materials.  The second activity conducted in the presence of the observation team measures Spatial Analytical ability.  Students use Tangrams to complete exercises and solve puzzles that vary in complexity.  Observation team members watch for various problem solving skills associated with the different exercises and puzzles.  For older students, the puzzles are correspondingly more difficult.  The third activity involving the observation team is a story-telling exercise using toys or interesting items.  The stories are recorded, to be later transcribed and analyzed.  Older students watch slides and create poems, stories, or other descriptions as they imagine themselves in one of the slides.

     After all Assessment exercises are complete, the observation team organizes the data and holds a debriefing meeting.  The teamwork component of this meeting is critical.  Team members analyze the data and eventually come to agreement on which ratings will be assigned to which students.  These ratings, and other data components, form the basis for Strength Profiles that are later given to the students, parents, teachers, and administrators.  The debriefing process is a unique component of the DISCOVER group Assessment that distinguishes it from any other performance-based assessment and contributes to a higher degree of reliability and validity.

     A typical Assessment (not including the Logical Mathematical or Written Linguistic components) usually is conducted in a single day and lasts approximately 8 hours (for the three observed activities and debriefing sessions).