As orig­i­nally con­ceived, empiri­cism was a way to test your the­o­ries and hypothe­ses against obser­va­tions of the nat­ural world. This approach to under­stand­ing real­ity was a rev­o­lu­tion in the sci­ences, bring­ing a flood of sci­en­tific knowl­edge and tech­no­log­i­cal advance­ments.
It’s not sur­pris­ing then that in mod­ern applied sci­ences, such as human-computer inter­ac­tion and soft­ware engi­neer­ing, empiri­cism plays a sim­i­lar role in help­ing researchers under­stand the phe­nom­ena they work with. One major dif­fer­ence between “soft” sci­ences such as HCI and soft­ware engi­neer­ing and “hard” sci­ences such as biol­ogy, chem­istry, and physics, is that the phe­nom­ena they study are quite dif­fer­ent in their per­ma­nence. Physi­cists study the inner work­ings of the laws of the uni­verse; biol­o­gists study the form and func­tion of processes in life. These are phe­nom­ena that change very slowly over time, which gives “hard” sci­en­tists time to hypoth­e­size, the­o­rize, reject, and synthesize.

In con­trast, “soft” sci­ences study phe­nom­ena that are extremely quick to change, rel­a­tive to the expan­sion of the uni­verse and evo­lu­tion on Earth. I might sug­gest some rela­tion­ships between pro­gram­ming envi­ron­ment design and group work, only to have yet undis­cov­ered and fun­da­men­tal dimen­sions of these pro­gram­ming envi­ron­ments change under my nose. I might pro­pose a the­ory about the causal rela­tion­ships between pro­gram­ming par­a­digms and pro­duc­tiv­ity, but in 20 years, some­one could rein­vent the pro­gram­ming lan­guage, pos­si­bly mak­ing my the­o­ries irrel­e­vant. Worse yet, by rely­ing on empir­i­cal data, which is extremely fickle and context-sensitive, I may not even con­firm my hypothe­ses or the­o­ries before they have to be thrown out.

What’s the value of empiri­cism then, if it’s not fast enough to keep up with the pace of tech­no­log­i­cal change? One argu­ment might be that the objec­tiv­ity that empiri­cism pro­vides has a slight edge on our best intu­itions, even in the short term. If I’m try­ing to decide what to name a par­tic­u­lar field on a web form, am I bet­ter off with my per­sonal biases, or small sam­ple of data, how­ever small or skewed? Prob­a­bly with data. At least then, the design choice is defensible.

But I think there’s a bet­ter rea­son. What under­lies these trade­offs is the fact that many “soft” sci­ences are used more directly for design than the harder sci­ences. In all design, there is never enough data to pro­vide an objec­tive rec­om­men­da­tion to every design deci­sion. At best, the research might offer a clean delin­eation of the trade­offs involved in gen­eral cat­e­gories of deci­sions, but ulti­mately, it is the designer who must make the final deci­sion, and they must make that deci­sion with their intu­ition. When I design tools and nota­tions, I might make a hun­dred design deci­sions a day, with only one backed up by empir­i­cal data.
Empiri­cism then, is best suited at arm­ing the designer with the most objec­tive and reli­able intu­ition that data can pro­vide. And that means that the most impor­tant part of empir­i­cal research to get right in the soft sci­ences is to have the designer get the data. That’s right: your pro­gram­mers must observe the phe­nom­ena they will sup­port. They must become experts in the domain. They must under­stand it with such detail that when faced with the small­est of pro­gram­ming deci­sions, they have an empir­i­cally grounded intu­ition about what will work and what will not, and a deep sense of the trade­offs of their choice.

If we really need empiri­cism to cul­ti­vate intu­ition, what is the value of report­ing empir­i­cal data? I don’t per­son­ally believe that read­ing a report of another researcher’s obser­va­tions is nearly as enlight­en­ing. But what it can do is change what researchers look for and how they inter­pret their own obser­va­tions. The role of the research com­mu­nity is to tem­per indi­vid­ual obser­va­tions with a broader col­lec­tion of data. This is how we gen­er­al­ize and val­i­date our expe­ri­ences, in the search for truth. We just have to make sure that in the process of seek­ing truth, we expose the design­ers and engi­neers who will be build­ing our world to as much of real­ity as we can.