I just fin­ished read­ing Jonah Lehrer’s How We Decide, a fas­ci­nat­ing sur­vey of recent (and not so recent) schol­arly lit­er­a­ture on deci­sion mak­ing, behav­ioral eco­nom­ics, and neu­ro­science. The cen­tral the­sis, or per­haps, the cen­tral extract from the large body of work on the sub­ject, is that while we often think of the ratio­nal parts of our minds as cen­tral to effec­tive deci­sion mak­ing, the emo­tional parts of our minds are in fact often more objec­tive. Jonah argues, through the exten­sive overview of sev­eral hun­dreds of stud­ies span­ning eco­nom­ics, psy­chol­ogy, mar­ket­ing, and med­i­cine, that this is because our brains actu­ally take in much more infor­ma­tion that the work­ing mem­ory our ratio­nal mind depends on could ever hope to process. There­fore, when our instincts (assum­ing our instincts come from prac­ticed, expert behav­ior) are often the bet­ter informed of the two. There are obvi­ously may sub­tleties to this point (and Lehrer does a great job explain­ing them), but the bot­tom line comes down to a fairly straight­for­ward to under­stand (though dif­fi­cult to exe­cute) rubric for deci­sion making:

• If the prob­lem is novel (to you), instincts will not suf­fice. This is a job not only for the ratio­nal part of your mind, but for our cre­ativ­ity. Our instincts can often help with the sub­prob­lems of novel prob­lems, but the ratio­nal mind must inte­grate them.
• If the prob­lem is rou­tine and can be fully char­ac­ter­ized with a few well defined vari­ables (or sim­pli­fied in this way because the decision’s con­se­quence mat­ters lit­tle), let rea­son care­fully assess and ana­lyze the options.
• If the prob­lem is rou­tine, but can­not be sim­pli­fied to a few well defined vari­ables, use your ratio­nale mind to iden­tify what infor­ma­tion is and is not avail­able, but let the emo­tional part of your mind process and ana­lyze it. The instinct result­ing from this is your mind’s expert judgement.

These ideas about human deci­sion mak­ing have many fas­ci­nat­ing impli­ca­tions for soft­ware engi­neer­ing. For one, the rubric above sug­gests that soft­ware engi­neers need a keen abil­ity to know when a prob­lem is new to them, so that they may apply dif­fer­ent strate­gies to solv­ing it. I’ve seen in many of the courses I’ve taught involv­ing soft­ware engi­neer­ing deci­sions that novice engi­neers often view every prob­lem as rou­tine, where some prior solu­tion is likely to solve the new prob­lem. Rec­og­niz­ing when this is not the case is a cru­cial skill. This might involve, for exam­ple, step­ping away from a prob­lem after try­ing to apply some known solu­tions, and using the ratio­nal mind to judge whether the prob­lem has novel char­ac­ter­is­tics that deserve cre­ative solutions.

Not only should soft­ware engi­neers be able to rec­og­nize a prob­lem as novel, but they must be able to judge whether it is novel to them or novel to every­one. Novice soft­ware devel­op­ers quickly real­ize that most prob­lems they encounter have been solved already; the chal­lenge thus is not to cre­ate solu­tions, but to find exist­ing solu­tions whose assump­tions fit the prob­lem at hand. The same is true in solu­tion spaces of a smaller scope; for exam­ple, some prob­lems may be new to an indi­vid­ual, but old hat to an orga­ni­za­tion. Novice soft­ware engi­neers should know when to con­sult cowork­ers for this expertise.

Both of the above abil­i­ties prob­a­bly come with expe­ri­ence; one issue that may not, how­ever, is know­ing what you don’t know. For exam­ple, I rou­tinely see experts strug­gle with bug triage deci­sions, mak­ing quick emo­tional judge­ments about a bug report’s legit­i­macy, fix­a­bil­ity, or impact, and then ignore infor­ma­tion in the report that would dis­con­firm their judge­ment. What’s miss­ing from these deci­sions is a process by which soft­ware engi­neers use their ratio­nal mind to care­fully enu­mer­ate what infor­ma­tion they don’t have, or what infor­ma­tion is sus­pect. Fight­ing this con­fir­ma­tion bias and get­ting com­fort­able with uncer­tainty is a fun­da­men­tal part of mak­ing effec­tive deci­sions about com­plex problems.

In his dis­cus­sion of avi­a­tion, Lehrer makes an inter­est­ing point about how com­put­ers can help pilots with such biases:

The reason planes are so safe, even though both the pilot and autopilot are fallible, is that both systems are constantly working to correct each other. Mistakes are fixed before they spiral out of control.


What would bug triage look like if humans and com­put­ers col­lab­o­rated together to judge and ana­lyze infor­ma­tion about bugs? Would it help if bug reports high­lighted what infor­ma­tion was miss­ing from bug reports, such as details about the defect’s impact on users, details about who those users are, and infor­ma­tion about what com­po­nents a bug concerns?

Of course, bug triage is just one of many deci­sion mak­ing processes in soft­ware engi­neer­ing. Require­ments engi­neer­ing involves a great deal of trade­off and analy­sis and knowl­edge about human deci­sion mak­ing might improve design­ers’ con­fi­dence that what they are spec­i­fy­ing will sat­isfy user needs. Debug­ging and other types of diag­nos­tic activ­ity share the same diver­sity of novel and rou­tine prob­lems and may ben­e­fit from the same kind of metacog­ni­tive strate­gies. Bug fix­ing is also rife with choices about the scope of a change and the impli­ca­tions of a change on both users, inter­act­ing sys­tems, and other com­po­nents. In all of these, it may be of the utmost impor­tance to pro­vide novice soft­ware devel­op­ers prac­tice in rec­og­niz­ing and char­ac­ter­iz­ing the prob­lems encounter so that they may choose effec­tive strate­gies to solve them.