Another recent paper, The camel has two humps, draws a more general conclusion. In this one, two British researchers correlated responses on an exam used to filter incoming students aiming to study CS. They boldly claim to be able to predict a student’s success in CS courses before students have had any contact with any programming language.
Their exam included several questions about how assignments in computer languages will change variables, with multiple-choice answers implying different theories about how assignment works. Theories included the correct one (a = b copies the value of b to a) and several incorrect ones (a = b moves the value of b to a, zeroing b; or a = b swaps the values of a and b).
The result: The test-takers who went on to fail in the CS courses were the ones who applied different theories of assignment to different questions. The ones who succeeded applied the same theory to each question; whether it was the correct theory did not matter!
” - Armed and Dangerous » Generative models and programming talent