My sabbatical research pivot

Since I started research back in 1999 as an undergraduate, I’ve always been intrigued by the goal of helping people write code more productively. Sometimes I ran studies that tried to identify barriers to their productivity. Sometimes I made tools that helped them navigate faster, debug faster, test faster, and communicate faster. Every one of my research efforts was aimed explicitly at speed: the more a developer can do per unit time, the better off the world is, right?

Then something changed. I founded a software startup, and led it’s technical and product endeavors. And I watched: how much did developer productivity matter? What influenced the quality of the software? What was the real value of faster developers?

In my experience, faster development didn’t matter much. Developers’ speed mattered somewhat, but only to the extent that we made effective product design choices based on the valid understanding of customer, user, and stakeholder needs. It didn’t matter how fast they were moving if they were moving in the wrong direction. And developer tools—whether a language, API, framework, IDE, process, or practice—mattered only to the extent that developers could learn to use these tools and techniques effectively. Many times they could not. Rather than faster developers, I needed better developers, and better developers came through better and faster learning.

Furthermore, I couldn’t help but wonder: what part of this job is fulfilling to them? It’s certainly wasn’t writing a lot of code. There was always more code to write. In fact, it was the moments they weren’t coding—when they were reading about a new framework, picking up a new language, trying a new process—that they enjoyed most. These were moments of empowerment and moments of discovery. These were moments of learning. Around the same time, my student Paul Li was beginning to investigating software engineering expertise, and finding that, much as I had experienced, it was the quality of a developer’s code, and their ability to learn new coding skills, that were critical facets of great engineers. Better learning allows developers to not only be more productive and more effective, but also more fulfilled, Better learning makes better developers, who envision better tools, better processes, and better ideas. As obvious as it should have been to someone with a Ph.D. in HCI, it was the human in this equation that was the source of productivity, not the computer. Like most things computational, developer tools are garbage in, garbage out.

After I stepped down as AnswerDash CTO and begin my post-tenure sabbatical, it became clear I had to pivot my research focus. No more developer tools. No more studies of productivity. I’m now much less interested in accelerating developers’ work, and much more interested shaping how developers (and developers-in-training) learn and shape their behavior.

This pivot from productivity to learning has already had profound consequences to my research career. For a long time, I’ve published in software engineering venues that are much more concerned with productivity than learning. That might mean I have less to say to that community, or that I start contributing discoveries that they’re not used to reading about, evaluating, or prioritizing. It means that I’ll be publishing more in computing education conferences (like ACM’s International Computing Education Research conference). It means I’ll be looking for students that are less interested in designing tools that help them code faster, and more interested in designing tools to help developers of all skill levels code better. And it means that my measures of success will no longer be about the time it takes to code, but how long it takes to learn to code and how well someone codes.

This pivot wasn’t an easy choice. Computing education research is a much smaller, much less mature, and much less prestigious research community in computing research. There’s less funding, fewer students, and honestly, the research is much more difficult than HCI and software engineering research, because measuring learning and shaping how people think and behave is more difficult than creating tools. Making this pivot means making real sacrifices in my own professional productivity. It means seeing the friends I made in the software engineering research community less often. It means tackling much trickier, more nuanced problems, and having to educate my doctoral students in a broader range of disciplines (computer science, social science, and learning science).

But here’s the upside: I believe my work will be vastly more important and impactful in the arc of my career. I won’t just be making an engineer at Google ship product faster, I’ll be inventing learning technologies and techniques that make the next 10,000 Google engineers more effective at their job. I’ll be helping to eliminate the hundreds of thousands of horrific experiences that people have learning to code into more fulfilling and empowering experiences, potentially giving the world an order of magnitude more capable engineers. Creating a massive increase in the supply of well-educated engineers might even slow down some of the unsustainable growth of software engineering salaries, which are at least part of the unsustainable gentrification of many of our great American cities. And most importantly, I’ll be helping to give everyone that learns to code the belief that they can succeed at learning something that is shaping the foundational infrastructures of our societies.

I’ll continue to be part of the software engineering research community. But don’t be surprised if my work begins to focus on making helping better developers write better code than simply writing code faster. I’ll continue to be part of the HCI research community, but you’ll see my work focus on interactive learning technologies that accelerate learning, promote transfer, and shape identity. And for now, you’ll see me invest much more in building the nascent community of computing education researchers, helping it blossom into the field it needs to become to transform society’s ability to use and reason about code as it weaves itself deeper into our world.

I’m so excited to engage in this new trajectory, and hope to see many of you join me!

12 thoughts on “My sabbatical research pivot

  1. I think this is a wonderful research direction, and I think you’ll ultimately have more impact on the state-of-the-practice than any tool-related work.

    Back when I was in academia, I had always wanted to do more to study skill transfer: in particular, to understand how we could transfer the skills of expert developers to competent ones. I’ve aways been inspired by the work that Patricia Benner did studying expertise in nursing. Her book “From Novice to Expert: Excellence and Power in Clinical Nursing Practice” is fantastic and I highly recommend it.

  2. Andy,
    I read about your pivot with great interest.So glad to have you on board! I made this “pivot” early in my career, all the while selling myself as an HCI researcher so that I could get a job at a research university.

    It’s indeed unfortunate that computing education researchers are still sometimes regarded as second class citizens at R1 universities. I think CS faculty, by virtue of their being teachers, sometimes have the view that computing education can’t be a”real ” research field. I’ve always countered that view by going out and winning great NSF grants. My colleagues can’t argue with that, even if they somehow hold a lower view of the kind of work I do.

    This leads me to your perception that “making this pivot means making real sacrifices in my own professional productivity.” I’m curious what you mean by that. How do you define “professional productivity,” and what sacrifices do you anticipate making? My impression is that there is TONS of funding in educational research, including computing education research. If you define productivity in terms of funding, I am skeptical you’ll have to make sacrifices. You are a talented grant writer and researcher, and I know you will continue to do just fine. Perhaps the sacrifices have more to do with how you think you will be perceived by your colleagues and institution?

    • Thanks for sharing Chris. The point about professional productivity is important for me to clarify. I wasn’t speaking about grants or even students, really: I was actually referring to the difficulty of the research. I think it’s a lot easier and a lot faster to build a system for a UIST or CHI paper than it is to contribute a solid study of learning, because learning is simply more difficulty to impact and measure. More difficult work means slower work, which means a slower rate of contributions. That’s fine by me, since I think the work is more important!

  3. Hear hear! I believe that tenure time should always be a time of change — can be changing interests, changing something in how you’re going about things, moving to something that’s somehow bigger, or more visionary, or bigger scale, or whatever.

  4. I admire both your willingness to take a risk and your particular choice of direction. This bit struck me as particularly brave and honest:

    “Computing education research is a much smaller, much less mature, and much less prestigious research community in computing research.”

    This may be the case now, but your leadership will change that. Computing Education Research is a fundamentally valuable field. It will be big, you and others will help it mature, and prestige will follow from that.

    • Thanks Lydia. It’s long road, but I think the world is ready for it and needs it. It’s especially exciting to join a field full of passionate researchers and teachers who’ve done the hard work of creating the field from scratch over the past several decades, and help establish it as a core part of Computer Science, Education, and other disciplines.

      • I’d like to second the statement that CS Ed researchers as a bunch are passionate. I am an early-stage researcher and I attended my first SIGCSE in Memphis back in March. That week any doubts I had about CS Ed as a career focus were quashed. The level of passion was palpable. Every person I met was so genuinely welcoming and ardent that at times I questioned if it was all real or if I had been transported into an alternate universe. I am proud to count myself as a new member of this community.

        • The more the better! It’s an exciting time to join the field and help it grow.

  5. Thank you for taking this on. I have been writing software since 1972 and while the tools have changed the thing that slows me down the most is learning the new things. There are always new languages, new APIs, even new concepts – there was no object oriented design in the 1970s – that have to be learned to remain productive. They often involved new mindsets and ways of looking at computing problems. Learning those things is fun, frustrating, exciting, exhausting, and why I am still in computing even if I have moved out of development. Most of all now that I am teaching computing I really want to understand how to do it better.

    • Thank you for sharing! I hope that as the computing education research community matures, we see work on learning spanning the lifespan, from K-99+. Some of my students are beginning to look at things like documentation as learning resources and identifying some exciting opportunities to improve their capacity to teach.

  6. Great post Andy. I’ve been on a parallel path for the last few years where I find myself caring less about tools that generate visualizations efficiently and more about how visual and other representations can provide people with better ways to reason with data. Its not exactly education, but learning and reasoning are now themes in almost all of my projects.

    • I think that’s a fascinating perspective on infoviz (and one of the reasons we were so excited to have you join!). Literacy, learning, and education are all tied together, but different things. In many ways, literacy and learning are core topics in information schools, which have always focused on empowering people access information, develop knowledge, and learn skills. Educational institutions are one way at that problem; informal access to resources, search, and information are another. You and I should should talk more about these connections!

Leave a Reply

Your email address will not be published. Required fields are marked *