The Who, Where, Why, How, and What of Being a Critical Consumer of Research

Erin Morris Miller, PhD

As schools and education agencies work to become more data-informed and use research-based practices, it is vital that, as the gifted education community, we become responsible consumers of the research in our field.

The nature of our work, and the close relationships with our children, students, and schools, at times leads us to overgeneralize personal stories, anecdotes, and emotional responses. However, now is the time for us to become critical consumers of the research in our field, to question, analyze, and report back on the published works, so that we can provide the very best for our students, children, and teachers. This can only make us stronger, better able to advocate, serve, and innovate in gifted education.

The following sections will outline the critical areas in which to begin analyzing a piece of gifted education scholarship. Each area can help readers to become critical consumers of the research, making evaluations on the reliability and veracity of the claims that are made in a given article.

Who conducted the study? What kind of experience do they have in working with gifted students? What qualifications do they have to do research?

Drawing of a man with brain exposed.

The qualification of the authors is an important consideration, but name or renown doesn’t automatically translate to quality science. There are different levels of experience and expertise both in working with gifted students and research abilities. Authors who are affiliated with a college or university are held to a high standard of conduct by their organizations. They were usually hired based on both their teaching and research potential, and most often their work must be reviewed by internal ethics boards. Although tenure and promotion standards can sometimes incentivize quantity over quality, these authors are also evaluated by how often their work is cited or discussed by other researchers. Thus, they are highly motivated to make sure that their work is consistent with past findings and predicts future work.

Articles written by teachers, parents, and other practitioners can provide insights about a particular school, family, or child. It can be helpful to obtain perspectives from the individuals who are working directly with gifted children. However, information derived from a single source or one person’s experience should never be used to make generalizations about a broad group of individuals.

Where was the article published? Was it in a peer reviewed journal? On a Professional Association website? Or One Person’s Blog?

There is a hierarchy of quality of publications. Peer review journal articles have been evaluated by multiple other researchers to ensure some level of quality control. There are also differences in the quality of journals. The premier journal in the field of gifted education is Gifted Child Quarterly. Other quality journals specific to gifted education are the Journal for the Education of the Gifted, the Journal of Advanced Academics, and Roeper Review.

You may want to also consider the audience for a given piece. For example, journals that are peer-reviewed are written for other academics. They typically have a higher level of academic rigor, because other researchers read them and provided reviews and recommended publications. On the other hand, articles in publications such as Gifted Child Today or Teaching for High Potential, were written for practitioners (teachers, parents, school administrators). They typically have less academic rigor and discuss the research findings to a lesser degree. Articles in these types of journals have also undergone editorial and/or peer-review, and the findings in the articles may be research translated for practitioners. However, it can be difficult to determine the quality of the research that is backing the claims in these articles. It is helpful to consider the expertise of the authors in these publications.

Why was the research conducted? Does the author have biases or a commercial interest in the results? Was it shared on a commercial website? Did you have to pay for it?

Authors have different motivations for conducting research. Many times, the research is fueled by simple curiosity and most researchers in the area of gifted education have a sincere desire to make the world better for children with high intellectual ability. However, it is important to consider if the author is making money based on promoting a certain viewpoint. Commercial organizations may be motivated to share only those studies that are consistent with their message. Similarly, online and internet sources may also have bias in the findings from the research that they would be willing to publish. Dig carefully and use your critical eye when considering articles that come from internet sources.

How has the research been conducted?

There are many issues with “the how” of research that are essential factors to consider. First, by research we are referring to empirical science, not book chapters or essays based on experiences. A common motto among researchers is, “In God we trust; all others bring evidence.” Good research is incremental, with each study building on the ones before. It is important that the study considers the work that has already been completed. The hypotheses should arise from the previous research. Next you need to look at the number of participants and how they were selected. Small samples are less reliable. This is of particular importance when considering the results of studies from the field of neuroscience that often have very small samples. Although the results may seem exciting, you should wait until the study is replicated with many different groups of participants before applying the results.

Evaluating the design of the study and the statistics can be challenging if you do not have a background in research. It is the job of professional researchers to critique research and their livelihoods and prestige are based on doing this well. Do not be afraid to seek their expertise. But also, visualize what it would be like to be a participant in the study and consider all the possible factors that might influence the results. When considering research results, you should be a skeptic.

Finally, you should consider how the conclusions fit with other scientific evidence. You should be more skeptical of research that inconsistent with past work and does not present a way to reconcile this difference. The strongest research will connect findings from other fields, such as educational psychology, general education, or special education.

What are the particular issues you should look out for when evaluating gifted education research?

Getting a representative sample of gifted children and/or adults is very difficult. By definition, the population is smaller. It is often impossible to get a random sample. Common samples include: participants in talent search programs (i.e. Duke TIP, or CTY), members of organizations such as Mensa, undergraduates, clinical populations, and student volunteers. You should make sure that the sample is representative of the population for which you are interested. For example, a sample of individuals who have sought help with mental health challenges would not be representative of all gifted students, particularly in a study investigating the prevalence of gifted children who struggle.

The most trustworthy researchers are careful not to apply findings beyond the original research. Be careful with interpreting correlations as indicating a causal link. For example, there seems to be a positive relationship between anorexia and intelligence, although the effect is very small and there is a large degree of variability. But this does not mean that higher cognitive ability causes anorexia or even that intelligence is necessarily a risk factor for developing an eating disorder. The relationship could also be explained by environmental factors such as family structures and level of education.

When evaluating a summary of research, you should be skeptical of authors who only present findings that support the author’s position. There are almost always contrary findings and the most trustworthy researchers consider the whole picture. When you hear contradictory findings from different experts, you should be more careful in applying the research, even if one side or the other better matches your individual experiences.