Cracking the Code: Discovering Technical Skills for Commercial SaaS positions

I was recently asked by one of my cross-functional stakeholder peers in the commercial organization, how they should evaluate a candidate for their technical prowess. The general brief was that

it was a candidate for the commercial part of the business.
this candidate had to be able to speak to the capabilities of the platform to have some conversations with the clients.
they would need to know when to pull other folks in if the technical nuances got too complicated for them to handle.
this was a role that past employees struggled to adapt, where they were good at the commercial aspect of the role, but struggled to skill up technically.

Starting with the basics

Let us gameplay this. I won’t speak to the commercial needs of this role, but for the technical component I see two primary instincts that are crucial for this candidate —

How do they learn? How curious are they about wanting to learn more about a platform or product(s) they are not aware of?
How is their problem solving ability? If they encounter a roadblock or an ambiguity, how are they able to create clarity, or know what questions to ask so that they can move forward?

One could argue that this is the basis for any candidate interview, but parsing out these features for this type of role is not always apparent.

Taking a leaf from Engineering interviews

In Engineering interviews, we broadly have 2 rounds of interviews

Coding/Programming — where we evaluate the deep technical abilities of an engineer, with regards to how they think of algorithms and problem solving. Their ability to zoom in and attention to detail for things like performance, testing, observability, code quality and ultimately successful outcomes are part of the test.
System design/Architecture — where we evaluate wide technical abilities, such as their ability to zoom out. They are expected to ask enough questions so as to propose components that can interconnect and move towards the solution. We then assess their ability to state assumptions and make tradeoffs.

Getting to the right candidate

Using the principles of the architecture round, there are a few paths you could go down —

Give candidates parts of the platform API specs ahead of time and ask them to describe their understanding of how things work. This could be used to see how keen their observations are. You could change a few scenarios around and see how they adapt to the changes.
You could use an open interface, like OpenRTB and ask candidates to describe parts of how it works — specifically the parts that are relevant to your business.
A summary of how a specific feature on the platform functions could be provided. Following that, candidates could be prompted to articulate this feature to a client. Before doing so, they may need to inquire about certain aspects to ensure their comprehension before engaging with others. This process would demonstrate their curiosity and understanding.
Extending from the previous point, interviewers can also role-play as the customer. By asking candidates questions, they can assess their problem-solving abilities. This evaluates whether candidates can effectively use the information provided earlier to address the customer’s needs.
What problems are you facing TODAY? Trying to see how someone would work through a real-world current problem is a great way to see how they might contribute to your team. Proprietary details can be obfuscated but there should be enough to be able to describe the problem sufficiently.
Were people hired before who weren’t able to do the job? What exactly did they struggle to do? Use those as case studies to see if this candidate does better or worse.
Ask the candidates what kind of technical tools they use and LOVE. Ask them to describe how they would change change one or 2 things about the product. It shows an attention to detail and an ability to form opinions.

Hiring is a tough exercise. The cost of hiring the wrong person can often be quite high. You need to know what you are looking for a candidate and make sure you run a fair and objective interview process. Hopefully, that results in hiring the right person for this role!