Using Rubrics For More Objective Hiring, With Christine Spang

Today on Scaling Software Teams, we’re joined by Christine Spang, Co-Founder and CTO at Nylas – an API built to integrate clients’ email, calendar, and contact applications in one seamless interface.

Besides being a brilliant developer and tech leader, one of Christine’s foremost passions has been balancing her career with a focused eye on her own self-care, something, she points out, that often gets overlooked in the tech community.

Dating back to her time as an undergrad at MIT and throughout her career at Nylas, Oracle, and Ksplice, Christine has made sure not to sacrifice her own sense of balance, healthy living, and virtue. She believes that this commitment has not only helped her avoid burnout – it’s helped make her an even better tech leader, too.

We talked to Christine about her self-care philosophy – as well as her thoughts on the current state of diversity in tech, and why hiring rubrics need to confront many of the biases present in the humans who construct them.

LISTEN TO THE FULL EPISODE HERE

‍

Topics We Covered:

• Getting Enough Sleep
• A Google Study on the Gender Pay Gap

Full Episode Transcript:

Christine thanks so much for joining us today on scaling software teams.

Thanks Wes. It’s really great to be having this conversation today.

So what got you into building software?

I really got into building software when I was in high school. On an emotional level, I was trying to find activities that kind of let me escape a bit, and one thing that I found that was really interesting in that vein was, first, books and then later I got into computer games – especially like role playing and strategy games starting with kind of graphical things like Age of Empires and Starcraft. Over time that branched more and more into games that just involve creating characters and worlds.

Eventually that led me to this class of games called MUDs. These are like text based games you play online. And I got really into this this Lord Of The Rings-themed MUD called The Shadows Of The Silver. And after playing the game for probably less than a year, I got so into it that the people that were running it asked me to help them.

So I started being a role-play administrator on that game – making up storylines and plots and making things happen in the world, which made things really interesting for all players. Eventually, I got to a point where they have these in-game tools that you could use to build new rooms, and different areas and objects – things like that. But if you wanted to be able to alter the very fabric of the world and add new capabilities you had to know how to code.

The game engine was written in C based on this really old mode code base called Deacon Mud, and I started teaching myself C in high school because I really wanted to work on this game… and then also the game only ran on Linux, so I had to install Linux… and through a series of “hops of curiosity” I went from installing Linux, to being curious about how his operating system come to be?

The operating system that I installed was this flavor Linux called Debian, and one of the interesting things about Debian is that it’s completely volunteer run so there’s like a thousand developers all around the world who work completely asynchronously and collaborate through mailing lists, and somehow every two years they release a fully functional operating system. That kind of blew my mind.

I started following the blog aggregator for all the developers, and one day there was a developer who made a post about this group called Debian Women that was an affinity group for encouraging women to get involved in working on Debian. And that really spoke to me because I felt like I was the target audience, so I joined their IRC channel and eventually I started to teach myself Python so I could help work on some Debian-related stuff. So it was this very curiosity-driven path throughout high school in my free time that led me to actually building things. And I always knew that I wanted to be an engineer because I was really into science and math in school, and I have a fair number of family members who are in some flavor of engineering; my dad was an electrical engineer, and my grandpa was a chemical engineer who actually eventually got into computers because he thought they were more interesting than chemical engineering. That was like way in the early days in like the 70s or 60s or whatever and computers were huge then.

So I always had this idea that I wanted to be an engineer, but that’s such a broad category – and through this path that I took in high school I came out of that being like I want to be a software engineer

So I played MMOs and some MUDs early on, and doing that  – leading voluntary groups where you have to motivate people and everyone can leave whenever they want – has made me a better leader. Do you think that your MUD days being a role play administrator helped you at your job now?

That’s a great question. I think there’s some crossovers, but then there’s also just some things about being in a group of humans that is largely in-person – but also kind of a commercial setting – that’s also different in a number ways. Learning stuff about how people interact is always helpful to you in a professional setting as your career, so it’s like a little hard for me to separate that from all of the lessons of just like… living.

So you got into Debian’s women’s affinity group. You want to be an engineer. When did you transition from building software to leading teams that build software?

Basically the path was… I decided in high school that I want to be a software engineer, and that led me to apply to a number of different schools that were really good for ZFS. And my top choice was to go at M.I.T. but I didn’t think I would ever get in… it was kind of my secret application that I made, but never told anyone about. And then when I got in, I was kind of this like, “Oh cool!”

I end up going to M.I.T. and then through that I got into startups in this weird, tangential way. Really the defining or pivotal moment that set off this chain of events in my career was not anything that happened after my career started, it was literally getting into open source software in high school – because I can trace a direct path from that to where I am today.

So being involved in Debian basically got me my first first couple jobs – like my freshman year, before I actually went to M.I.T. I ended up working in the M.I.T. media lab because when I was visiting M.I.T. as a pre-freshman, I hung out with some Debian people, toured the lab where they worked and some guy was like, “Oh someone who knows how to code who’s coming M.I.T. I need like more minions in my lab.”

So I end up working there, which was pretty cool because I literally graduate from high school on a Saturday and then on Monday moved to Cambridge Massachusetts and started working in this lab.

That continued, and I met a bunch of folks through this organization M.I.T. called the Student Information Processing Board, or “SIPB” which is basically the M.I.T. computer club That’s what led me to my first job after college, which was at this tiny startup called Ksplice, which was building a rebootless kernel technology for Linux.

And I got that job because I knew the folks who started it through SIPB and I was actually sponsoring Debian package uploads for one of the founders who was working on this math software called Sage and I had at that point become an official Debian developer contributor and was able to make new uploads of different software to the archive and I was doing that for Tim Abbott, who’s one of the founders of Ksplice. So it’s like literally all through free and open source software that I ended up at this tiny bootstrapped startup, and that was my first real work experience.

I never really thought that much about startups as a concept before that. I know that there’s like a group of people that like gets into entrepreneurship as a concept, and then decides that they want to be an entrepreneur but that’s not really the path that led me to startups.

It was always just like technical, open source software path. And so eventually after Ksplice I started my own company called Nylus, and we are basically building a platform that makes it much easier for developers to integrate with unstructured data like email context and calendar, and basically transitioned to leadership as a part of the complete firehose of starting this company. So like the first year after starting was really focused on building you know we were like less than six people the whole time.

I started out by building a lot of our original back-end infrastructure, our like iMap sync engine for connecting to iMap. I figured out how to get our stuff running on AWS and stuff like that. And I was really just kind of like a mid-level engineer when I started the company. I had three years of experience at a previous company and it was definitely like flying by the seat of my pants in that we were trying to build the product, and then we hired a few people. The first people we hired were like acquaintances from college and then it kind of branched out into, like, this one guy that we met in a café who just happened to be a completely badass front-end engineer. But it was a very spontaneous way that we met each other because we were working in cafés at that time – we didn’t have an office.

That’s a recruiting strategy – just to have everyone work in cafés and serendipity will hit and you’ll find bad ass front-end engineers.

Yeah I don’t recommend that as the way to bootstrap your company, but sometimes random stuff happens and that’s just the way it works.

So basically as the company grew over time, as a founder you have to do whatever needs to be done to keep the company going, and at some point the thing that needed to be done was someone needed to manage the other engineers. And so I basically went out and talked to all the people I knew that had management and leadership experience and it was like, “Help. What do I do? How do I do this?”

And they gave me some book recommendations – I’ve always been a really big reader so reading books helps me.

What are some of the books that you got the most value out of during that transition?

I think the first book that a friend recommended to me actually was this book by this guy named Jerry Weinberg called “Becoming a Technical Leader” and the things that it focuses on is really this transition from being primarily hands-on and building things, to moving into leadership skills, and how to lead an organized team – and also just talking about how important people skills are in the craft of building software.

So I don’t remember specific lessons that I took away from it but it made a big impression on me in terms of really solidly outlining why leadership is important, and why you should spend a lot of time working on it because it’s so much easier to focus on technical skills. But that’s probably not what’s holding you back.

So thinking about that transition and thinking where you are now and what you’ve learned, what are some things that younger Christine maybe missed? If you could give your past self advice? I think the fact that you went out and asked, and realized how important that people side was… that’s like step zero for success. But it’s still pretty hard.

I think one lesson that I would have liked to have internalized earlier is that it’s really important to me to have friends and mentors that you can be really open with about your feelings, and that are really supportive. I think it’s really common, especially as a founder, to like hold things in and try to be really stoic.

But vulnerability is a huge part of connection, and you’re gonna feel way better about everything you’re going through if you find the right folks to be really open with and share your struggles. Because even if those people don’t have solutions, they may have gone through similar things, and just having this connection and support from other people makes things infinitely easier to handle.

And how did you find those folks?

It was kind of organically over time – some of them were through meet-ups. One person I really lean on a lot is my friend, Charity Majors who actually has another company – Honeycomb. They’re great. And we kind of met each other at this ops meetup, and just like in a time of crisis for me she was the person who was there and I will never forget that.

I’ve heard emotion described as like as a pipe. So if you don’t let some of it out it out, it forces itself out sometimes in surprising ways. It ends up just squirting out in the wrong place. That’s that’s been true in my life.

I think it’s also just really bad for you physically, like you can get all sorts of you get sick from holding things in like physically.

 

Wes: You’ve talked about how sleep is really important to you, speaking of other things it might be bad for you physically, how do you incorporate getting enough sleep into making yourself effective at your job?

Christine: Yeah. This is something that’s really evolved for me over time. When I was at M.I.T. I learned a lot of really important life lessons, and the thing I learned in my freshman year was that if I did not get enough sleep, I was literally going to fail out of college.

There’s this tradition at M.I.T. of students basically staying up late into the night and going exploring around campus, and climbing on top of buildings and playing pranks. And it’s a really cool and interesting tradition, but I found that if I stayed up too late and I didn’t get enough sleep, I just was not [going to do] well as a student.

Luckily, when you’re a freshman at M.I.T. there’s this thing called pass/no record, which essentially means that if you fail any classes your freshman year, they don’t go on your transcripts. It’s a very great feature because M.I.T. is a really intense place, and that intensity combines with it being a typical college experience, where all of these young folks are on their own for the first time – they don’t have parents who can tell them what to do.

If I did not get enough sleep, I was literally going to fail out of college.

— Christine Spang

And so you want to break all the rules for a while – like maybe I can stay up late and eat ice cream for dinner or whatever.

But I think that maybe learned the lesson faster than most people, because I’ve learned over time that I’m a really sensitive person, and my feedback mechanisms for my body are very powerful, so I literally just can’t handle not taking care of myself for long periods of time.

So with this like rapid feedback loop where I completely fall apart if I’m not taking good care of myself, I learned that I just can’t let that suffer. So I managed to graduate from M.I.T. largely with the help of not staying up all night, and that really led me to have this very internalized sense of the line at which I will not let my self-care suffer.

I remember coming out of M.I.T. starting my first job at Ksplice, when chatting with the founders about taking the job, I asked them if I would have to work more than 40 hours a week for this job, because honestly it takes awhile to recover from M.I.T., and I felt like I needed time to recover.

Plus, I was figuring stuff out for myself and I was worried about overloading myself, and wanted to make sure that like we were on the same page from the very beginning about what my commitment level was. It was a great job and I was really into it, and when Jeff told me yes on that first day I was like OK I feel good about this.

 

That you that you would have to work more than 40 hours a week?

Oh, no I got that flipped in my head. [Laugh] He said no.

So you had like you had a firm commitment for how much you would work, about how much you were going to need to work, and you felt good about that?

Yeah. So that was an experience of like laying out boundaries for myself and it was a good experience. And, I’ve just taken away from that this very strong internalized sense of what is worth it and what’s not worth it. And even when starting this company I just decided from the get-go that I wasn’t going to sacrifice my physical or emotional well-being in order to get the company off the ground, because it wasn’t worth it to me.

So with starting the company I think it was really interesting, because I was kind of worried about the workload and maybe stress level, but I think that there’s this interesting interplay where if you find the right thing, and you’re really excited about it, you actually can work more.

And I didn’t go into the company being like I’m going to work my ass off`… there are times at this company where I do work a lot. But I feel like because I came into it with this internalized sense of what is OK and what’s not… I still prioritize self-care in a way that is maybe abnormal or unusual or an outlier. But I can do that while also putting a lot of energy into things that I’m excited about.

I guess tactically… I have discovered over the past five years that I’m definitely a morning person, and I allow myself to just not stay up late most of the time. So if I’m at a work related dinner, I leave by 9:30. And sometimes it’s still hard to leave an event before it’s done, but I just like think about the next day and what I would sacrifice from losing sleep. So it’s a motivator to go home go to bed.

And I’ve developed a bunch of like rituals for winding down, because sometimes being a startup is really stressful and you have to be able to disconnect and transition into a state of relaxation in order to actually sleep. So for example, one thing that I do is I really like taking hot baths before bed. I do that like once, maybe twice a week sometimes, and I find that helps me sleep.

Recently in the last couple months I’ve actually started using the sleep tracking device called the Oura Ring which is kind of like a Fitbit but it’s actually a ring… it’s sleep tracking feature is more accurate than the Fitbit, and having that feedback loop of telling me how well I slept immediately afterwards also helps motivate me to do the right things to get good sleep.

Yeah I think anything we can do is tighten feedback loops… I’m wearing my Fitbit and Oura Ring now.

I’ve always been into kind of these like “quantified self” type things. I just love having the data. It’s really fun.

Do you use the HRV readiness information at all?

I do actually! So HRV stands for “heart rate variability” and there’s a couple of features on the Oura that are kind of related. There’s your age variability and then it also measures your heart rate while you’re sleeping. It can determine your resting heart rate really accurately and it can also give you a graph of your resting heart rate and HRV over the course of the night while you’re sleeping.

So those two things together. There’s like a little bit of mixed science behind this but they can help you determine your recovery state. So I’m really excited and really into these kinds of things because I’m also really into rock climbing – I actually train to perform better at rock climbing – I have been pleasantly surprised at how useful it is for determining which days in which I should train hard and which I should back off. So there’s this like interplay between the data and how you feel. And I find it helps paint a more accurate picture of how to alter what I’m doing; when I should prioritize like going to bed or when I should skip doing something hard because my body is just not going to be able to handle it well.

So when you see your readiness level is lower, you’re going to skip that hard work out and go to bed earlier, and if it’s higher maybe if you push through?

Yeah definitely. Like yesterday, for example, my clinical readiness was a little bit lower and so when I got home I decided to take a bath before bed because I know it helps me sleep better. Then, also, funny story, before this podcast I had originally set my alarm clock this morning to get up a little bit earlier – I’m trying to readjust after daylight savings time because it’s really annoying. I hate it so much. But I’ve been trying to get back on my normal, early schedule because the clocks went forward and I don’t like being in a rush in the morning.

And so I set my alarm clock but then before bed I was like, “Oh. You know actually maybe I’m not going to set my alarm clock because I think it’s more important to get a little bit more sleep because I clearly didn’t sleep great the night before.” So I decided that shifting my schedule was not as important as making sure I was super ready to be chatting with you right now.

Well we certainly appreciate it!

 

Wes: Google was in the news recently for announcing that they’ve leveled their gender pay. What’s the problem with using within-level pay as a measure of pay equality?

Christine: I think there’s a lot of problems with that particular article. One is that the journalists who wrote that was just picking a sensationalist headline to get clicks, and that makes me angry because it’s not contributing in a useful way to the conversations happening right now.

But in terms of like what it was actually talking about, the article talked about how the metric that they were looking at was whether folks in the same level at Google were getting paid equitably. But the problem is that when you look at men and women, and you look at racial minorities, the fact of the matter is that those folks are very often under-leveled.

So they’ll be hired into a position where they have more years of experience than a typical white man does in that level, and if you’re only looking at the same [position] level then you’re losing a ton of nuance as to how people are actually being paid or underpaid. I’ve never seen a good study at a big company where they actually managed to correct for that kind of thing, and I don’t think that the data is meaningful until you’re actually able to incorporate those aspects because the leveling ladder can be just as biased as the pay within a band.

If you’re only looking at the same level then you’re losing a ton of nuance as to how people are actually being paid or underpaid.

— Christine Spang

Gotcha. Yeah I saw several folks kind of point out on Twitter and I think it’s totally right. If I were trying to put myself in their shoes I’m trying to think, how do you fix it? [In your opinion]… if you’re the CEO of Google… what’s the thing that you might do? This is a hard question…

Yeah… I think that there’s this common thread when talking about diversity inclusion at tech companies is a people being like, Well the problem is really hard, so maybe we’ll never solve it. But like we’ve built a global distributed system that connects people all around the world and allows you to look up – in milliseconds – any piece of information you want. We’re really smart people – surely you can put a little more effort into it than that. I think that perhaps the piece that is missing is that I think it all starts with listening to the folks who are not like you, and who are being affected the most, because it’s hard to solve a problem when the problem doesn’t affect you.

I actually have to give a lot of credit to my team for instilling this perspective in me. They’re very vocal about their feedback, and at some point our leadership team was talking about diversity and someone pointed out that like our team as a whole is more diverse than our leadership team.

Which is very common…

Yeah it is very common for a variety of reasons. And I spent a lot of time thinking about that for like days afterwards and I just keep coming back to it whenever I think about this stuff because it just makes so much sense to look to the people who are affected most, and make sure that you’re starting with their perspective and maybe those ideas will help us make more progress than just trying to like sit in a room by ourselves and reason from our outlook.

 

In a Reddit AMA you said the hiring a more diverse team requires buying into the concept, examining your own biases, and implementing solutions to address those biases. What are some of the solutions we’ve implemented to help Nylas?

So there’s a few different things. In terms of like kind of like tactical things like hiring more diversely we do a few different things. One is, obviously, it starts with the top of your funnel and working on sourcing a variety of different people and starting conversations with them. That’s step one. You can’t do anything else if you’re not even initiating conversations with diverse set of people.

And then we’ve focused a lot on trying to bring a rigor of standardization to how we evaluate folks, and I think there’s always more room for improvement here. Obviously you’re never gonna get to a point where like you’re able to like objectively evaluate a human being on all axes. But as we’ve grown as a team, we’ve more and more actually tried to create rubrics for the questions that we ask – [in terms of establishing], “What does a good answer look like?” And where’s the line between like hire and no hire – which I think is really useful and interesting.

There’s always going to be cases where you talk to someone and what they say falls completely outside of like anything you’ve thought of before, and you have to like make a judgment call as to where that falls.

But rubrics don’t have to be perfect to be better than not.

Yeah. Giving you a structured way to think through, like, what are the traits that we’re looking for and what is good and what is not good. I think that’s really valuable.

I also think it’s important to just like have a culture of talking about biases, and what we should correct for what we shouldn’t. For example in our hiring wrap up meetings after an interview, things will come up and someone will say, “You know, my gut feeling while going through this was was X but after examining that gut feeling for a while I think that I shouldn’t pay too much attention to that because there is this bias that can cause that feeling.” And I think it’s important to be able to have those kinds of conversations. If you’ve read this book called “Thinking Fast and Slow” by Daniel Kahneman.

That’s one of my favorite books!

Yeah so gut feelings are information, and they’re very useful information – they’re intuitive information. But gut feelings can also be full of bias, so I think that especially in the context of hiring, you want to use that information and not discount it completely.

But it’s also useful to go down the road of slow thinking and analyze that somewhat, because there are going to be cases where you’ll be feeling that gut feeling because of a very deeply ingrained bias. And you’ll have a better outcome if you question that a bit.

I think in the book for his Israeli army hiring he had people actually write down just their gut feeling and that added to the algorithm and made it more predictable.

Yeah. But like, especially in the tech world, intuitive pattern matching is literally what has gotten us into this situation we’re in today. So you can’t just leave that by itself and say we’re just going to keep doing that.

Absolutely, that has to part of the system. Yeah. As engineers we have to we build systems so we can improve them.

It can feel like really tedious and annoying until I go through the process of writing down exactly what you’re looking for, and what’s good and what’s bad. It takes time but I think when it comes to diversity and inclusion it will lead to better outcomes.

‍