Welcome to Scaling Software Teams, a weekly podcast that helps software leaders navigate fast growth without losing the magic that made that growth possible. I'm your host Wes Winham. Today we're joined by Skyler Nesheim, the VP of engineering at the Dwolla, a fast growth FinTech scaler.
They've raised over $50 million to connect enterprises to the US banking system. The Dwolla team has built one of the most disruptive products out there in the enterprise FinTech space, and they've done it all from Des Moines, Iowa. In this episode, we talk about how to transition to a microservices architecture, how to incorporate customer feedback into our product vision, and why it's actually a good thing that we're not Google.
I really loved the concrete conversation about transitioning to a microservice architecture. That's something that's on the roadmap for lots of startups, but there aren't a lot of "hard won" lessons learned out there.
The point in the "you are not Google" is not an insult, I don't think, and it's not something that you should feel down about, but I think it's meant to serve as a reminder. The context is very important.
Now here's my conversation with Skyler Nesheim.
Skyler Nesheim, thank you for joining us today on Scaling Software Teams.
Thanks. It's an honor to be here.
What is the most important inflection point in your career that got you to where you are today?
I was thinking about this and I have a little bit of a different angle.
For me, I think back to the time when I met my partner actually, because there's this thing where you meet someone and you form a connection with them, and they change your perspective on a number of things. Maybe they're a balance or a counter-example to who you are and they have the ability to really get to know you and push you and drive you.
When I met Jessica, I was in college and I was doing okay in college but I think, as we got to know each other and talk, I think she really pushed me like, "You don't challenge yourself. You don't work hard enough. You're getting a B when you could easily get an A. You're sloughing off on your study a little bit."
She, not in straight up words but in nonspecific words, set the tone of the type of person that [she would want to partner with]. That relationship changed things for me and pushed me in new ways. I think a lot of that stuff I have brought into my career, and it has very much influenced the types of tasks that I've taken on and the types of challenges I've placed for myself. That partnership I think was really important.
That's awesome. So you changed your trajectory as what you aspire to do in work. I know it's not a romantic partnership, but I think we have partnerships with our teams. How do you think you've been able to take that push and challenge and trajectory, and put it into running your teams?
This is talked about a lot, but… it's the actual process of writing code and building systems and deploying things, and we spend a lot of time talking about that stuff.
Another very important piece of this is building relationships and figuring out how we work together to best solve problems. When you think about maybe a romantic relationship or a relationship with a partner, it's not the exact same way that you apply it at work, obviously. But there are certain traits that you do bring into your work life.
Another aspect of this that is not work related but definitely influences the way that I talk to people in the way I worked through problems, is some of my experiences with parenting and having young kids and talking to them and working through things with them.
These experiences outside of work give me an opportunity to work through conflict and think about how we resolve conflict internally, how we communicate conflict, and how we raise conflict. Ultimately, getting through those things gives us an opportunity to build trust so that we can better work together going forward.
You're coming up on seven years at Dwolla so you've been there since Series B. Lots of change, lots of growth. I know you went to the process of splitting your application out into multiple services so that you could better support multiple teams. When did you realize you had to do that, and how did that project go?
I think this is a journey that a lot of companies go through - you have an application, and you're focused on iterating on that application or that set of that technology stack and you're moving very quickly so you're keeping it small, you're keeping it tight, you're trying to reduce complexity.
At some point, as your business becomes successful, you start to find a market fit. You realize the different ways that you need to grow and build your company, and you might see that you need to expand teams. Maybe you've been focused on one team, and it's four or five people and they're all masters of this technology stack and they're all working together and collaborating on it.
You might see that, hey, in order to actually make progress on our company's mission and help our customers, we've got to split what we're trying to do into multiple goals and we actually need to run multiple things at once. We've got to grow our team. As you look at that, for us, one of the things that I think we realized is that it's not just about adding more engineers or booting up the way that we do work into multiple threads or parallel tracks.
The architecture of the applications and the way that we deploy our applications also needs to evolve to support this. So at Dwolla when we started to realize that, we started looking at how to take something that may be mostly one technology stack, and split that up so that multiple teams can work on it simultaneously.
At that time we were really looking at a microservices architecture, where we could have services that were completely their own stack and were responsible for their own problem domain, so that we had the ability to work on multiple things at once and deploy them with tight interfaces between these things.
[In other words,] we would trust if we deploy service A, it's not going to break service B. We had to have a tight interface there, but as long as we kept to those contracts and tight interfaces and tried to keep those technology stacks separate, we were able to work on and evolve service A and service B simultaneously.
On an abstract level that sounds awesome... I think that story seems pretty succinct that I hear it a lot, but I'd love to hear... You actually did it, you went through this process, you thought through what's the right domains.
What are the right services, what are some things that you wish you could tell past Skyler that you could avoid some missteps and some rework?
I mentioned tight interfaces and that is an abstract concept and one that's hard to attain. If there's a contract between service A and service B, that contract needs to be maintained and the power in maintaining that contract means that you can ultimately build trust and a relationship between those two things.
Past Skyler and past engineering teams would have really liked to hone in on that more quickly, that if there is a contract between two services, we need to in some way, deeply consider how we prevent breaking that relationship.
And were most of these contracts APIs, is that what the contract looks like?
Yes, API contracts. Microservices. It's systems talking to other systems. That's the innards of what's going on in the application - in our platform stack - but that same thing applies back to our customers. We've done I think, a really good job there. That's maybe something that we worked through that maybe wasn't as intentional as it happened, but internally we discovered a lot about how to run a service oriented, or a microservice, architecture.
That influenced a lot of the ways that we approach the API that we provide to our customers, and we really do everything in our power to prevent breaking changes within that API. In fact, I'm not aware of anything that we've ever done that would be considered a breaking change to our API that would cause massive re-versioning or massive rewrite for our customers. It's a challenge.
Internally, it sounds maybe you didn't have that discipline early on or you weighted that relatively lower in importance and now you've learned that having really strict API guarantees is important internally. Is that what I'm hearing?
Yes, we definitely learned that lesson. I think a counterexample to this that I'm thinking about now is also, to have the knowledge that you're going to make a few mistakes as you try to expand and do this. That's okay if you approach it from the aspect that, “We are going to make mistakes, we're going to do things wrong.”
If you remove that constraint from yourself to say, "Hey, we're trying to evolve this." If you remove the constraint of "we can't make a mistake" and say, instead, "we plan to make a mistake" then what things would your team do or would the technology need to do to support that kind of evolution?
Making it resilient to mistakes.
Then I think that opens up the types of questions that you want to ask: “How could we be resilient to this type of change or how could we make us take small steps so that we can evaluate if this is going to cause problems?” Things like that.
That was another lesson learned in the process of evolving this architecture; taking very small steps. Not trying to take a big bang or holistic approaches to problems, but [instead], "What is the first step we can take in an iteration or half an iteration or in a day that allows us to evaluate this approach and gain feedback?"
Whenever you were making those... you had your existing teams, you're splitting things out. Did these teams split come first or did the service split come first? Did you split a team and say goes with this thing out or did you split the thing out and then the team followed?
We focused on the architecture first and then that allowed us to later split out teams that could work on those things. I think that you could do it either way.
The other approach that we took that was pretty successful, is we had a part of our application that would be maybe more monolithic. Instead of sitting down and saying, all right, we need to break this whole thing up and it's a large project in order to do that, we instead tried to be opportunistic looking at the roadmap for our product and knowing the things that we needed to do and thinking about how we could utilize our product roadmap as opportunities to evaluate ways to evolve our architecture at the same time.
That allowed us to chip away maybe some of that core thing that was originally built and start to break it out as our teams evolve for need.
The other side effect of that was that we didn't have to work on things that weren't necessary for our business to evolve. We were able to prioritize the right things that enabled the business to evolve... we took both a business and a technology focus at the same time instead of taking just a technology view of it or just a business view of it. Those two things live together as we evolved the product.
Do you remember any of the features or enhancements you're working at the time that spurred, "I work on this part of the system so let's split this system out because we're already enhancing it."
Yes, a big one [where business and technology grew side by side] was that Dwolla's original project was very consumer to consumer focused. It allowed business to business transfers of money and things like that but it was very focused on consumer to consumer.
As we started opening up an API to developers, we started to hear feedback like, "Hey we like the API, we love the ability to work with this product that you've developed, but it still has a lot of Dwolla's name to it when we do it. When we have to accept an OAuth flow, Dwolla's name is part of that. When we go to transfer money, sometimes Dwolla's name as part of that. We would like the ability to [have] more control over our users experience."
Some people would maybe refer to it as a white label way. We started thinking about the next application of our API or our platform for moving money, and what would it look like to run that in an experience where it's more controlled by our customers, and less controlled and less part of our name? That was the business thing.
At the same time we had the technology side, where our original API was a part of a core or a legacy part of the product. Those two things paired together ended up in this evolution of our API where we now offer a white label API for money movement; that stack is itself a separate microservice that evolved out of that thing.
It was both a conscious business decision to think about a new vertical and at the same time applying new technology to that new vertical.
That's a great example: you were going to build a white label and instead of building it within the white label API, instead of building with the existing monolith, use as an opportunity to build a new service that serves that new use case.
There were a bunch of other fringe things that we needed back from that core product. As we built out the new API and realized that we needed access to those features, we built services that would give us access to those features and the core product instead of other ways that you might integrate with the thing. That built upon itself in terms of evolving the architecture.
I've just been asking you for some advice that has a lot of contexts around it - I'm curious to go meta on that. How do you find advice? We're going to build microservice, we're going to go service oriented architectures. It's really nice in the abstract, but how do you look for advice as a leader that has the appropriate context?
I was actually just thinking a lot about this myself. I think in this time that we're in now, there's so much information out there. You can find a number of articles online, a number of blogs, a number of technology leaders that will give you different kinds of input and different ideas on how you might build systems or how you might evolve architecture.
To your point, context is very important there because the context of what strategy you employ on a five person team, in a consumer focused product is perhaps a lot different than the context you would you would need in a 10,000 or 1,000 person development team in a business focus context; those are two very different problems and a lot of the foundation that we start upon is similar but the approaches that we take are very context dependent.
A good starting point for me is, who are other leaders that are in businesses like myself? In a technology company, if you've received venture funding or you have other partner companies that you work with, one thing that has been successful for me is to look for opportunities to find partnerships or mentors within your network.
If you can connect with those folks, and they have a similar context to you, they can be a great sounding board for advice on an approach that you're considering. That's one strategy that I've used that's pretty successful.
Finding more long lived relationships with folks that might be in your industry that are already in your network?
Yes. One thing that I've heard recently is... I think I saw an article about this recently. The title might even been this, “You're Not Google”.
You'll see these things that Google does and engineers are always fascinated [by them]. I myself definitely have a background in engineering and I am also fascinated by super challenging problems at a very large scale. Those are a lot of the types of things that Google or other large technology companies have to solve.
The point in the you are not Google is not an insult I don't think, and it's not something that you should feel down about, but I think it's meant to serve as a reminder of the context is very important.
I read the White Paper on, I think it's called Zanzibar. They're globally distributed authentication system that takes trillions of requests per second, has to respond in 10 [milliseconds]. That is really cool, but that is not the problem that we have with authentication at all. If we go and try to build that, we are wasting a lot of time. I think that same pattern applies for hiring, applies for teams microservices.
In the early days of building a company or getting started and even as you keep expanding, I think it's important to keep in mind what are core competencies that you want to focus on. Sometimes your team can surprise you. Core competencies are actually really important - they’re differentiators for you that may be directly related to your business model or just supporting of your business model.
There are other times where you might be working on a problem and your team may be giving you this feedback or this underlying tone that it's sometimes a challenge to pick up on. It's like, "Hey Skyler, this doesn't feel like a core competency for us." To me as a leader that's a signal like, "Hey, can we start looking at how we might buy this product or how we might partner to get this product?" Instead of it being something that we do ourselves.
Have you made one of those decisions recently where you decided that you should buy or partner with a better choice than developing that core competency?
In this case I would actually use working with Woven as an example - when we go to find candidates for engineering positions. We can spend a lot of time trying to dig in on the data about the candidates and trying to find the right technical assessment for candidates and things like that. With Woven, we saw an opportunity, where that is a core competency for your team and it's something that I think your team is doing very well, Wes.
When I first chatted with you - I didn't actually mean for this to evolve in a sales pitch, but here we are - I saw that as a core competency and something that I think that your team gets and they're evolving and I saw that as an opportunity to partner with someone. To outsource that capability for our team.
Is it something that we could solve? I think so. I think we have some smart people but I also recognized on the other side that you've got a bunch of great smart people on your staff, as well, and I saw an opportunity to work with you. There's an example right there of partnering on something that might not be a core competency so you can focus back on your core product.
Makes sense. I promise I wasn't fishing for that, but I appreciate the example. I think there's lots of areas where, with a scaling engineering team, there are so many areas to focus.
I've seen performance management tools, [for example]. Could you build the best performance management system from scratch? Probably, you got smart engineers, you could do it. The best hiring system, the best... The monitoring system. I saw a tweet from a medium size startup that they built their own columnar data store for their monitoring system.
That is really cool engineering, but it is not obvious to me that that is the best use of that particular team's time. Which is challenging because good engineering team's really like building stuff... Have you answered any heuristics on what should be a core competency versus what shouldn't be a core competency for you?
I think that you have to keep your business model in mind and know who your customer is. And keeping contact with that customer is probably the best way.
As long as you have that view or that feedback mechanism in place, I think you will naturally gravitate towards the right things that are core competencies for you because you're constantly getting customer feedback and hearing what they need and hearing how your product needs to evolve.
That feedback will be a very strong voice in your organization and you won't be able to ignore that feedback or that advice to go work on exciting problem like you building your own columnar data store.
How have you been able to keep that feedback loop? One challenge is we need engineers in focus spots of time to be successful customer, just tickets going all the time is interrupting. Are there any processes that you've been able to balance that interrupt driven versus direct customer feedback at Dwolla?
One really cool thing that we've been working on, is the UX team that's part of our product organization has actually created a mechanism for customer feedback. We have a customer feedback program and we've openly invited customers to contact us and reach out if they'd like to be part of that problem.
The team literally has calls with them where they ask them questions about the problems they're facing, interfacing with our product... or things that their business is facing and how we could expand to support them even further. That has been a great mechanism for getting feedback on our product - something that I've been really excited about seeing evolve.
The UX team is getting customers enrolled, they’re scheduling meetings and then the engineering team… are they watching recordings or debriefs or are they actually on those calls?
It depends. A lot of times we know the topic or what customers want to talk about… Sometimes it makes sense for an engineer to be on that call and sometimes it doesn't. It actually varies what kind of engineer makes sense. Sometimes it may make more sense that someone that has front end experience or front end focus, and sometimes it might make more sense from someone who's thinking more about the API product and its accessibility and how easy it is to call and how easy it is to work with and modify for your purposes.
It really varies call to call, which is, again, honing in on what feedback we are seeking and what's important to this particular customer.
I love to hear about engineer's on those actual calls. That's a... I think there's no replacement for that human interaction. Hearing some of the motion why they struggle with the problem is maybe... At least for me that is the most motivating thing to go build a solution.
Having a team [that is] making that happen [to say], "Oh I'll just jump on a conversation." That sounds amazing. Have you seen any decisions that the engineering decisions made differently as a result of engineers actually being in those conversations?
Absolutely. Since we are an API driven, or an API first company, sometimes that point of contact or that point of feedback is almost engineer to engineer and so, then the feedback multiplies in terms of the impact that it can have because not only can I hear what you're saying, but we can talk about it maybe with the exact same terminology or very similar terminology.
Then that impacts... I was thinking about making this change to the API. Now that I've heard this feedback, I'm thinking about whether or not that's actually the most efficient or most simplistic way to make the change that I'm considering. That's been really valuable for our engineers to consider.
I didn't consider that, that you might have developers as your users there. That is an amazingly valuable conversation.
You've grown a great engineering team at Dwolla in Des Moines, Iowa. Not a city that typically comes to mind immediately when you think about fast growing fintech companies, can you talk about the challenges associated with recruiting and hiring in small to mid markets?
For me, it has been a lot of fun to be part of Dwolla's journey in Des Moines. I know that prior to being at Dwolla, I've been here for about seven years. I was working in what would more classically be called an enterprise company. It was a great opportunity. I learned a lot about building software, about delivering software, about the impact that software can have.
But when it came time for me to start thinking about my career and where I wanted to go and what I wanted to do, I was really looking for a place that I could make an impact. At that time I saw Dwolla as an opportunity to join a company early on. Yes, I was taking some risk, but also an opportunity to make an impact - that was something that was very exciting for me.
As I think about being a small company or a company that is starting in maybe a small city or a smaller community, a big opportunity that you can provide potential employees, or a thing that you can talk to them about or make them aware of, is their opportunity to make an impact at your company.
That opportunity is something [that you can’t find everywhere], especially in a community like Des Moines. We have a bunch of financial insurance-focused businesses in Des Moines that are at a large scale and they're doing very well. They're great places to work. They're great places to learn, to actually start a career or join and build a career from.
The value prop that I think we've tried to highlight at Dwolla is some of those companies... they definitely have big problems to solve. Dwolla has big problems to solve as well, but [we have] fewer people and less time to do it with.
What that means is if you thought about our floor here and you thought about problems just laying on the floor, there's a number of problems that you can help solve, that would be a once in a career type problems in a larger company. It's really fun for me to present those types of challenges to our team and give them the opportunity to work on them.
I love that. I saw some blog posts recently about how you can recruit in San Francisco and headlines [said] "just promising impact doesn't work anymore". Everyone's got that. To me that was shocking because I'm in Indianapolis so I think to me promising impact is what most of the people I talk to get excited about.
I love that you highlighted “impact” as a thing that you can offer and it's differentiated in Des Moines. Maybe that's less of San Francisco, which would be... that sounds really hard. I'm glad I'm not there sometimes.
I'm not an expert on this, but there's this discussion that's happening about, where is the right place to build a company? Because the coast definitely offers you the proximity to VCs and the proximity to great engineers that have worked on problems like this before.
But the Midwest and this area, it also offers opportunity for people who want to work on problems like that, but probably or possibly aren't interested in relocating or moving to this coastal areas. So there are, I think, companies that could find value in playing into that niche market. Again, I'm not an expert on this or know how to advise when you should or shouldn't apply that strategy, but it's definitely been part of Dwolla's strategy that I think has worked pretty well.
The other thing that I've noticed about our team here, that makes our team great is when they join the company and they build it, they become part of the mission and they start solving hard problems. Our engineers have tended to stick around and wanted to be a part of our mission and see things through.
Anecdotally, again, I don't have data to back this up, but I don't know if that's always the exact same case for people who are joining companies on the coast. Anecdotally I've heard there can tend to be a little bit more jumping around or moving between positions and I think that might be evidence that it's not just opportunity like you were saying it's table stakes. In the Midwest for us, opportunity has been a way to encourage people to stick around for the mission.
Yes, it's a differentiator in Des Moines, [and Indianapolis]. When you think about those folks that have stuck around to see it to the end, have you done anything to cultivate that? Is there any practice you have internally that you think has improved the likelihood that those folks are going to stick around?
I am constantly thinking about the people on my team and the challenges that they're currently working on and what might be the next great challenge for them. I think as long as you have that in mind and you're trying to constantly present people with new opportunity, it continues to reinforce that the place that you're at is a great place to be.
My guest today has been Skyler Nesheim. Skyler, thanks for joining us on Scaling Software Teams.
Thanks again to Skyler for sharing his experience with us. Here are a few takeaways I had from our chat.
Number one, as we scale, we'll probably need to consider a microservices architecture. Skyler said it's not just about adding engineers, our technology needs to evolve to make it possible for our teams to work on multiple tracks at once.
This underlies the idea that microservices are a team scaling strategy more than they are a technology scaling strategy. One thing Skyler learned from his transition is that tight interfaces are a key to a successful microservices transition from a monolithic architecture.
There needs to be tight contracts between internal service A and internal service B, otherwise problems become evident. We need to deeply consider any way that we might break that contract. We need to version our internal APIs just like we might with an external API. Having a really strict API guarantee is very important for our microservices transition.
Number two, the best advice is highly contextual... AKA, you're not Google. There are a lot of ideas out there and some of them are very exciting. They're crazy scale challenges, trillions of transactions a day, those sort of problems, but we aren't Google.
A five person team is different than a 1,000 person team. There's also B2B versus B2C, SMB versus enterprise. That context matters.
What we should ask ourselves is, who is in a situation similar to me? Who has the same context? Then we can find partnerships with our investor network to find other folks like me and we can lean on their experience rather than trying to copy Google and a 1,000 person architecture.
Number three, our engineers should talk to actual customers. We need to keep our business model in mind and the best way to do that is to get engineers in front of customers, especially on already scheduled calls that they can just drop in. That allows them to hear how our product needs to evolve, which creates opportunity for innovation.
Engineers know what's possible. They can also know what the customer is experiencing, [and] that opens up lots of lane for innovation. Skyler has a customer feedback program which actively reaches out to customers to see if they want to help craft the solution. This reduces the friction between an engineer wanting to speak to a customer and being immediately on a call with a customer who is excited to talk.
Thanks again to Skyler for coming on the show. Now I want to hear from you. Please leave us a review on iTunes, Google play, or wherever you get your podcasts. This will help others find our community. Until next week, keep on scaling.