Cognitive Reflection Test and the Polarizing Force-Identification Questions in the FCI

Abstract: The set of polarizing force-identification (PFI) questions in the FCI consists of six items all basically asking only one question: the set of forces acting on a given body. Although it may sound trivial, these questions are among the most challenging in the FCI. In this work involving 163 students, we investigate the correlation between student performance on the set of PFI questions and the Cognitive Reflection Test. We find that for both scores in the FCI as a whole and in the PFI questions, the range of values of the Pearson coefficient at 95\% confidence interval, is suggestive that cognitive reflection may be one of the contributing factors in the student performance in the FCI. This is consistent with the idea that high level of cognitive reflection may help in eliminating seemingly valid choices (misconceptions) in the FCI that are intuitive from everyday experience or "common sense" but otherwise misleading. The ability to activate System 2 in Dual Process Theory, whether from System 1 or right after reading a physics problem, may contribute in narrowing down the set of prospective valid answers in a given physics problem. Complementary to cognitive reflection are other factors associated with deep understanding of physics whose effects are expected to become more evident with the level of difficulty of a set of physics problems. Given two students with the same level of cognitive reflection, the one with deeper understanding of physics is more likely to get the correct answer. In our analysis, the range of correlation coefficient for the set of PFI questions is downshifted with respect to that for the FCI as a whole. This may be attributed to the more challenging nature of the latter compared to a significant fraction of the remaining questions in the former.