In Stu­art Reges’ paper The Mys­tery of “b := (b = false)”. He analy­ses the cor­re­la­tions between the ques­tions on the 1988 Advanced Place­ment exam in Com­puter Sci­ence. He found that there were sev­eral prob­lems that had a high cor­re­la­tion with all of the other prob­lems. In other words, if stu­dents suc­cess­fully solved these ques­tions they were more likely to solve the other prob­lems. Stu­art refers to these ques­tions as pow­er­house ques­tions.
We dis­cussed the pos­si­bil­ity that the dif­fi­culty of these prob­lems was with the assign­ment oper­a­tor. Stu­dents are first intro­duced to the = sym­bol in math­e­mat­ics. Then, when they are rein­tro­duced in pro­gram­ming, the stu­dents must cre­ate a new men­tal model of what = does. It requires think­ing about vari­ables val­ues before the assign­ment state­ment as well as after. These prob­lems require think­ing about the dynamic state of the pro­gram. This is a con­cept which many stu­dents have not been intro­duced to. The stu­dents who do under­stand these prob­lems thus have a bet­ter under­stand­ing of how pro­grams work.
While this idea was por­trayed in the paper as well as our dis­cus­sions, there was some dis­agree­ment with it. For exam­ple, Grace sug­gested that it was more a prob­lem with the con­cept of Boolean val­ues. Many stu­dents are able use con­di­tional state­ments but do not grasp the con­cept that the con­di­tion sim­pli­fies to a Boolean value. So with a prob­lem such as b := (b = false), they do not see (b = false) as hav­ing the value true or false. They can­not deter­mine what b is being set to because of this mis­un­der­stand­ing of Boolean. This idea is backed up by the numer­ous stu­dents in 142 who have con­sis­tent trou­ble with Boolean val­ues and Boolean Zen.
Both of these ideas, of why pow­er­house prob­lems are such a good indi­ca­tor, reflect a prob­lem with the abstract under­stand­ing of pro­gram­ming rather than the com­plex­ity of the prob­lems. Ben brought up a quite inter­est­ing point about the dif­fer­ence between pat­tern match­ing and under­stand­ing. There are many stu­dents who can excel at math and pro­gram­ming but have lit­tle under­stand­ing of the abstract con­cepts of what they are doing. Rather, they are really good at pat­tern match­ing. They can tell what they need to do and how to do it, but have lit­tle con­cept of the over­all pic­ture. The exam­ple that Ben gave us was some of the stu­dents he works with are able to manip­u­late equa­tions to change the phys­i­cal appear­ance of the graph. They know where and what to change to get the graph to change but they don’t grasp why it changes.
This prob­lem of not under­stand­ing the why behind things could very well be why these pow­er­house prob­lems are such a good indi­ca­tor. They require that the per­son solv­ing them have a strong under­stand­ing of why and how pro­gram­ming state­ments have the effects they do. This would imply that the stu­dents who were suc­cess­ful in pat­tern match­ing would have a more dif­fi­cult time with prob­lems like b := (b = false). In his paper, Stu­art Reges, states that test mak­ers should not pack their tests full of pow­er­house prob­lems. He says that these prob­lems might be more use­ful as prac­tice prob­lems because they will illu­mi­nate what the stu­dents don’t under­stand fully.