How to Interview an Engineer for your Startup


There has been some concerns about putting our hiring process “out in the open” while we’re still interviewing for our Lead Engineer position, don’t fret — we’ve got plenty more fun case studies like these! If you are looking for your next adventure and want to join an awesome team, shoot me an email: tk@toutapp.com.

Also, there is a great discussion thread going on over at Reddit.


Over the past week, I’ve been interviewing for the Lead Engineer position here at ToutApp. I’ve never been a fan of interviewing. No matter how elaborate the process, we all know it often comes down to that team huddle after the interview and gut-feeling comments like “I really liked her, she had a good vibe.”

I think hiring Engineers is even harder. I personally had to go through the Gnomes problem while at Plaxo and logic problems while interviewing for an internship at Goldman (wth was I thinking), and I hated every minute of it.

So, here at ToutApp, I take a more practical approach. When I interview engineers, I get in front of a white board and ask the person to build me a link-shortening service with an API.

Why a Link Shortener?

It started off as my go-to-example because we built one ourselves here at Tout and got to know the interesting intricacies of it. But the more I thought about it, the more I loved the idea of it because it covers many fundamental concepts behind systems design, service oriented architecture, proper database design and optimizations and even job queues and caching around a very simple and easy to understand use case.

Also, I never thought this would happen (but it did) — if a candidate had trouble grasping the concept around a link shortener, then he or she is definitely the wrong person for the job (instant filter).

Step 1 – Set the Context

The way this works best is to engage the candidate in a way so that he or she starts to draw out the overall architecture, database design, interactions and even certain parts of the code right on the whiteboard.

So, to get things started, I start off the diagram on the whiteboard listing out the key use cases I want my link shortening service to support.

  1. I want a Link Shortener with an API that I can call from my web-app. I want to pass in a URL and I want the Link Shortener to give back a shortened URL that I can present to my user
  2. When the shortened URL is visited by my users, I want the Link Shortener to redirect to the original URL
  3. I want an API where I can query stats around a particular shortened URL giving me details about the users that have visited that URL with as much information as you can give me about each person

I want the candidate to be drawing the design, architecture and code on the whiteboard, so I always start off by drawing out a simple web-app box with two endpoints, one representing the request for a shortened URL and another representing the request for stats around a URL.

Step 2 – Start Drawing

At this point, you want to have the candidate pick up a marker and start talking through how he or she would go about designing the service. The key things you want him to write out are:

  1. The routes your web-application should support
  2. The models that you’d create for the web application and the database tables
  3. The controllers (don’t get into pseudo code just yet)
While he is drawing things out, I like to challenge his/her thinking on a few things:
  1. I like to make sure they get detailed enough in terms of route names, url paths and model names. I’ve found that the way an engineer names objects and the conceptual model they create to organize their code base is a tremendous indicator of how good they are.
  2. The other key thing I like to probe into is how they create their routes. Do they require a POST vs a GET when accepting a short URL request? Challenge their thinking around this, obviously you can go either way but the considerations they took to get to the decision will tell you how much they really know about the HTTP protocol and building a RESTful API.
  3. Look out for the things you think are obvious. I had one candiate talk about how he would store the tracking data in the session and save on a database call (which made no sense). So even on the simple stuff, make sure you get them to explain how they’d handle the key aspects of the use cases.
  4. Lastly, always link back to the three use cases you started by writing out. Those are three very loaded sentences — keep an eye to see if the candidate remembers to track the redirects, if he or she thinks about authenticating the API calls from your web service, and other things that aren’t obviously spelled out but an experienced hacker would know to think about.

Step 3 – Get into some Pseudo-Code

Although we’re primarily a Rails shop, we’ve interviewed people with backgrounds in Java and .NET as well. I figure that if they are truly strong programmers, they’ll be able to pick up Rails and Javascript fairly quickly. So at this point in the interview, I like to dig a little bit deeper into their coding abilities.

I ask them to write the pseudo-code for the controller action that addresses Use Case #2 – the redirect.

If  the candidate has trouble with Step 2, I always make sure I give them the benefit of doubt and help them along with the design to get them to this particular step. So at this point, assuming that he or she is not completely confused, they should be able to churn out some pseudo code like this:

def redirect
link = Links.find_by_short_url(params[:id])
visitor_information_from_header = parse_http_request_for_user_info(request)
LinkVisit.create(visitor_information_from_header)
redirect_to link.original_url
end

By this step, you’ll really be able to figure out a candidate thats pretty much full of shit and a candidate that understands the basics of programming. If you want, you can make the example more concrete and have them write real code (e.g. in Rails) so that you can test their understanding on how a framework like Rails encapsulates query strings, http headers, and how ActiveRecord works.

Step 4 – Test their knowledge on Architecture Patterns

At this point, I like to throw them a curve ball. I go up to the white board, and start to mark up the calls in their pseud-code with how rough estimates on how long each call would take. I try to use a red marker so that it stands out, and it looks something like this:

def redirect
link = Links.find_by_short_url(params[:id])  [60ms]
visitor_information_from_header = parse_http_request_for_user_info(request) [1ms]
LinkVisit.create(visitor_information_from_header) [60ms]
redirect_to link.original_url
end

I then say that user experience is of utmost importance to us, so we want to make sure that these redirect calls are done in under 20ms. I ask what he could do with this design so that we can accomplish that.

Your candidate should be able to talk through each of these core concepts (from easiest to hardest):

  1. Create a database index on the short_url field in the Links table so that lookups are faster. [+10 points]
  2. Make the short_url the clustered index and primary key for the Links table [+20 points]
  3. Store links created in the past 30 days in an in-memory cache for easier lookup [+50 points]
  4. Create a background job worker + queue that will take the vistor header information and will persist it to the database later [+100 points]

As you can see, even in a simple servie like a Link Shortener, there are a number of things you can do to drastically improve performance. Being able to talk through these simple patterns is what truly sets apart the boys from the men.

I also like to see if they suggest alternate ways of storing the data. Do they say we should use a document-based storage system instead of a Database? Do they want to use the shiny new technology just for the sake of using it and because they just learned a buzzword like NoSQL or can they back it up with sound thinking? This exercise is a great platform to have those kind of discussions and get at the root of his/her thinking process.

Step 5 – Test their knowledge on Software Patterns (optional)

Finally, if you’ve gotten this far, I delve a bit deeper into how he or she would organize the software code. What parts of it would be a standalone library and what parts of it would go straight into the controllers.

This is more to ramp down the conversation but the things I look for is whether they care enough to create a standalone library for hashing URLs, or whether they like to encapsulate the tracking code within the model or whether they prefer to do it all in the controller (and why).

Other things I like to talk about (and In Conclusion)

This exercise has proved to be way more effective than a lot of other styles I’ve employed while hiring in my career. It forces you to interact with each other, gives you a glimpse into their problem solving abilities, how he or she deals with pressure, how they deal with situations and concepts that are foreign and really puts the two of you “in the moment” as if you’re already on a team solving a problem.

In addition to this, I also like to talk through and gauge how much user empathy they have. These days, people that can cross between frontend, backend and design are the true rockstars and exactly what we’re looking for, so I like to get a feel for whether they “just like to code requirements” or they can truly get proactive and help us build product.

How do you interview engineers? Hate/Love this approach? Share your comments below or email me at tk@toutapp.com.


Shameless self-plug:

Want to join ToutApp as our first Lead Engineer? Apply here.

 

  • http://excid3.com Chris Oliver

    Wonderful method of interviewing. I built one of these in my free time and can definitely see how the whole solution requires understanding of pretty much most aspects of typical web app development.

    Another quick little benefit of this is you can also see how they would solve the simple algorithmic problem of shortening the url itself. Do they just take the simple approach of incrementing an integer? Or something like using a different base number system?

    • TK

      Yes thats right! The hashing is an interesting part in itself.

  • http://embarke.com/ ALBsharah

    Great concept here, as I’m a big fan of on-the-spot thinking during an interview.  You can only learn so much by asking pointed questions.  Getting them on a mental roll by opening the door to solving a problem lets you really see what they’re made of.  Nice work!

    Cheers,
    AL

    • TK

      Thanks :)

  • http://twitter.com/annmariastat annmariastat

    I really love your approach. When I interview someone, I ask them to tell me about the best thing they’ve worked on. Then I ask them to explain it to me. If they were the project manager & don’t know beans about coding they go very general. If they get excited and tell me, “We had a problem with the data because …. ” and then proceed to explain how they solved it, including drawing out a flow chart or writing snippets of code, then two things have happened. One, the interview process has been PLEASANT for that person, because he or she got to talk about what was fun and interesting. (And who doesn’t want to work with people who make life pleasant for them?) Two, they have been able to demonstrate their strengths.  I wouldn’t ask about a specific problem – we’re a different type of company from yours – but I do want to know specific ways a person goes about solving a problem.

    • TK

      I like to start broad and then dive into specifics. When speaking about past projects, people tend to take credit for things other team members did — it becomes a collective thing.

  • Arpan

    But do you expect the engineer to remember all the specific interfaces/apis for parsing header information from the requests? 

    • TK

      Not really, most of this is pseudo-code.

  • http://www.fastcandidate.com/ Girish Laskhminarayana

    Great technique. 
    Apart from testing technical soundness what other questions do you ask the candidate?You still end up meeting candidates and rejecting most of them so that part of the frustration remains. It will be interesting to know how much of that you think can be pre-screened online. I have developed a tool which helps you do that (I am promoting this using Tout, by the way) http://www.fastcandidate.com but I dont know how it will perform with this level of technicality and interaction, but I think it can definitely crunch time-to-interview using non-technical criteria. 

    • TK

      Awesome that you are using Tout! Let us know if we can help with anything.

      Aside from technical soundness, culture is hugely important. We try to get the whole team to spend time with the candidate and make sure that the vibe is right.

      • Girish Lakshminarayana

        Thanks. Tout works great for me as of now. 

        I agree about culture and spending time with the team. Getting them to be a part of the hiring process creates buy-in from the team as well.  I have only been able to replicate this in principle (as an anonymous team voting feature) in my app. 

        Thanks for the article :-)

  • Anonymous

    The Gnomes problem? What is that?

  • http://www.facebook.com/gregsabia Greg Sabia Tucker

    Some programmers do not do well when being put on the spot (I’m one of them). I think I would rather be told to build a link shortener app from scratch in x amount of time than have to lay it out on paper and be judged for accidentally leaving stuff out. I guess what I’m trying to say is: I’m very good at programming, and not very good at explaining things, or knowing exactly what an app will need just because it’s laid out on paper. Try not to judge programming skills by the way he/she can explain something on paper.

    A simple alternative I use when I hire new engineers is I’ll have them simply create an application from scratch; any application; it could be something simple, or elaborate; a link shortener, a mini-Twitter, anything. The only caveat? They cannot use any third-party libraries or software except for frameworks. Once it’s completed to their liking, I bring them in for an interview. Instead of pummeling them with questions like an interrogation, we just go over the application they created. Why they added this, why they added that, what framework they chose and why, et cetera. This allows them to show off their skills and creativity, while allowing me to see their passion (or lack thereof) for their work, not to mention literally seeing the code they write.

    • http://www.facebook.com/crutcher Crutcher Dunnavant

      “not very good at explaining things, or knowing exactly what an app will need just because it’s laid out on paper”.

      I work in the field. I consider these critical skills.

      • http://www.facebook.com/gregsabia Greg Sabia Tucker

        I agree, sometimes those skills are necessary, especially when you’re working “in the field.” Although, I hardly think you’re developing large-scale applications for months at a time “in the field.”

    • http://www.jimsc.com jim rubenstein

      I’d agree with Crutcher.  While there is absolute validity to not performing perfectly when put on the spot, or explaining things on the spot or when you’re nervous; the whiteboard method passively tests these skills.  Working as part of a team, communicating effectively, not being a total douche bag, working under pressure, and being able to formulate coherent thoughts are all part of working for a company.  If you don’t possess those skills, there’s a very chance I’d not want you on my team no matter how much of a genius you are.

      That said, there’s a certain amount of nervousness that comes with working with someone you don’t know, and knowing that person is actually judging you as you speak.  The interviewer will take this into account, and it’s not something you should concern yourself with.  If the interviewer DOESN’T account for nervousness or expects you to be spot-on perfect during a whiteboard interview, well…you probably don’t want to work there anyway (:

      I wish people would stop crying about having to do whiteboard work in an interview.  There’s more that is being assessed than just your programming skills and knowledge here, as there is much more to working as part of a small (or even a large) team than just being able to write code.

      • http://www.facebook.com/gregsabia Greg Sabia Tucker

        So you would reject a genius programmer if he/she lacks communicative skills?

        That is where you fail.

        • Guest

          Yes. Communication skills are as much or more important than programming skills in a business environment.

          If you need your software to be built and understood by more than one person and maintained by people other than the original coders, you had better believe that communication skills are critical.

        • http://www.jimsc.com jim rubenstein

          Would you not agree that in order to communicate how “genius” one might be, one would need a certain amount of communication skills?
          If you’re hiring someone based completely on what their resume says, or what ever grand promises of intelligence and genius they possess, that is where you fail.

          I don’t have room on my team for someone who thinks they’re so smart that they’re beyond having to work as part of a team, or bounce ideas around with a team.  Company culture is more important than being the smartest guy on the planet.  I’d take 2 “good” developers with lots of real world experience who work well together, over 1 “great” developer who can’t work with anyone else.

          • Guest

            You’ve just said in this thread that you would not only reject a genius because of his communication skills, but also accept two average programmers just because they can communicate.

            Just because a person is introverted does not mean they can’t work well with people.I’ve some news for you: average “coders” are NOT programmers. Managers like you should not be hiring programmers, because ultimately, you’re not looking for programmers, you’re looking for coders.

          • http://www.projectbeyond.org/ Carlos Killpack

            You don’t have to be extroverted to communicate. It might take more work for an introverted person to master communication skills, but they are still important.

  • http://gregsabia.com/ Greg Sabia

    Some programmers do not do well when being put on the spot (I’m one of them). I think I would rather be told to build a link shortener app from scratch in x amount of time than have to lay it out on paper and be judged for accidentally leaving stuff out. I guess what I’m trying to say is: I’m very good at programming, and not very good at explaining things, or knowing exactly what an app will need just because it’s laid out on paper. Try not to judge programming skills by the way he/she can explain something on paper.
    A simple alternative I use when I hire new engineers is I’ll have them simply create an application from scratch; any application; it could be something simple, or elaborate; a link shortener, a mini-Twitter, anything. The only caveat? They cannot use any third-party libraries or software except for frameworks. Once it’s completed to their liking, I bring them in for an interview. Instead of pummeling them with questions like an interrogation, we just go over the application they created. Why they added this, why they added that, what framework they chose and why, et cetera. This allows them to show off their skills and creativity, while allowing me to see their passion (or lack thereof) for their work, not to mention literally seeing the code they write.

    • TK

      I think its all in how you engage the candidate. I do it in a “let’s work on this together” kind of way, where we talk through it slowly together.

      • Dave

        It doesn’t seem like it’s honest to say “let’s work on this together” if you already know all the answers. That’s not collaborative, it seems patronizing and elitist – but I use both those strong words because there’s no less melodramatic equivalent. It seems slightly patronizing and slightly elitist. Snooty? Is snooty a good word here? It seems snooty.

        I think a better approach may be to find a few programming challenges or puzzles where the interviewer is unaware of the solution. Trying to solve it together will show the candidate what type of developers s/he’s going to be working with. It will show the interviewer an honest perspective on the candidate.

        Also, if a candidate can figure out a problem and I can’t: they’re hired!

        Um, thanks? I think I just figured out how I’m going to do interviews from now on!

    • ZJ

      This thinking fraught with the same perils it had the first time you posted it, and were criticized, in this thread.

  • imdidactic

    Have you thought about how a qualified lead engineer candidate will also be evaluating you and your organization?  As a lead engineer, if you don’t consider my extensive portfolio (which I would present with code samples) and a quick technical conversation about the software solution your business requires as proof of my abilities – if I have to answer mid-level programming questions on a whiteboard, I would immediately expect that the C level decision makers at your company are not prepared to offer the proper support structure for a full development staff.

    I’ll expect your company to make my work life a living nightmare of last minute, undocumented changes, hours of marketing execs blabbing incoherently about the merits of agile development, and my phone blowing up 24 hours a day because someone found a text bug. I will immediately put you on my “only in an employment emergency” list.

    And don’t get me started about what I’ll think if the phrase “equity in place of salary” comes into play.

    • http://twitter.com/NSResponder John C. Randolph

      Bingo.  I’ve walked out of interviews upon deciding that the people I was talking to weren’t qualified to employ me.

      • TK

        Thats not always a good tactic, because in a lot of cases the most qualified engineers in a company only get brought into the interview process later on in the process.

        • PaperTowel

          In my limited experience interviewing. If you meet the low level BS early on, it only continues. Your better off finding the job that interviews you for the job you applied for. If I applied for an entry level job then I’ll answer the entry level questions with enthusiasm; if not then I’ll answer them quickly to move on to what I expect to be the real questions about my ability to perform in the role I applied for at the company.

        • http://twitter.com/NSResponder John C. Randolph

          It’s not a tactic.  It’s me exercising  quality control.

    • TK

      I think one of the best parts about this exercise is that it gives you a chance to work together on designing a system. The interactions alone really get to how well the two of you can work together as you consider different ideas.

      I always caveat this exercise as please don’t take any offense, I know a link shortener can be a very simple thing, but it helps us have a conversation.

      • http://www.jimsc.com jim rubenstein

        The only issue there is, when the candidate has trouble – things get REALLY awkward.  You just want to end things as fast as possible (as the interviewer) so you don’t waste any more of your time, and you don’t lead the candidate on.  Pre-screening before you get here is important, but sometimes they slip through the cracks.

    • http://www.facebook.com/uncannykate Kate Kirby

       The problem is, software jobs vary wildly from shop to shop. I’ve seen people recently come into an interview with 10 years as a lead developer on their resume, and they didn’t know what MVC stood for. You just can’t rely on resume experience in this field, not if you are going the small team of experts model. (If you’re going large team of worker bees of varying skill and a few overlords doing architecture, which is also a common model, you can take more for granted.)

      Which isn’t to say that you don’t have to sell the job to the candidate. The trick is to make it a conversation. Don’t be like the person giving the driver’s test at the DMV – frantic scribbling on clipboards with no real feedback. Say positive things when they’re on the right track. Show some personality. Treat this person as much like they’re already a coworker as possible without solving the problem for her.

      And of course set aside some time for selling yourself. It’s not about equity and salary at this point – though that discussion is important too. You want to sell how you’re developing clean, modern code that’s easy to work with, everyone on the team is smart, you have good project managers, or a strong QA staff – of course, if these things are true. Be honest about your cultures flaws as well – if it’s a shop that people work 8am – 7pm most days in, you’d better say that. Some people don’t care, but if they do, it’s a disaster waiting to happen if they find out their first day. And if the story you’d have to tell is not very compelling, you better be working to fix it!

  • Matt

    20 years or hiring and being hired tells me that screening based on domain knowledge – the most natural thing in the world – is over-rated and over-done.

    I end up being MORE successful in jobs that I managed to get when I had no domain knowledge.  Similarly, the seasoned domain experts seem to be exciting as they walk in the door and but often lose their luster soon thereafter.

    Domain experts have their big rusty chest of war stories and techniques, and they will insist on using them.  After all, you hired them based on some of those things, no?  That’s great unless your needs conflict, which they probably will if you’re building something custom.  If they conflict, you are better off with someone who is fresh.

    What I want to know is: will this person do whatever it takes to succeed (are they hungry), and do they have the capability to know what ‘good’ looks like.  I can’t think of a time when we hired someone that failed because they just couldn’t bone up on gaps in their domain knowledge.

    • imdidactic

      This makes sense for a mid-level programmer, but not lead level.  The only person that should be telling a lead engineer how architect something is the CTO and a lead engineer should be the one guy on the team that more than anyone, including the CTO, that needs to have absolutely specific domain knowledge.

      • PaperTowel

        You can’t be serious. In my limited experience I’ve never come across a situation where I didn’t have the ear of a lead or a CTO on architectural decisions. I think it would be foolish to not expect great ideas from any level in the company.

  • Craig

    Along the same lines, when interviewing for Front-End developers (HTML/CSS/Javascript) we ask them to bring up jsfiddle.net and write a tic-tac-toe game. Everyone knows the game, so the use cases are simple to describe: (1) draw the game board, (2) every-other click drops an X or an O in the cell, (3) detect three-in-a row.   A good candidate will complete 1 & 2 and be headed in the right direction on 3 within 30 minutes.  It’s amazing how many candidates can talk a good game during the phone interview, but can’t manage the CSS to make the game board look like a real tic-tac-toe board (with inside borders but no outside borders).

    • imdidactic

      I’d be inclined to say that you’re losing leads on the best programmers in the field by asking them to do something like this.  A top-level developer will, on average, walk away, unless your business is to make consumer level tic-tac-toe games.

      My interview method is this (not saying it’s the best ever, but I think it demonstrates the idea of assessing ability without possibly menial/offensive tasks):

      1. A few generic software architecture or programming methodology questions that don’t necessarily have a right answer.  The goal is to just gauge a candidate’s ability to think a problem through and communicate a potential solution.

      2. Ask if they contribute to or solo wrote any open source projects.  It shows the ability to collaborate with a team and/or a passion for what they do.

      3. Before the interview I request that they bring the source code for a program that they wrote that they’re proud of.  I then have them walk me through a few classes and the methodology behind their architecture.  This is not only to judge their coding ability, but their coding standards and readability.

      4. I expect a senior level engineer to ask me what I want to see from his portfolio when setting up the interview.

    • TK

      I have to admit, I’m pretty sure I’d fail that test!

    • http://www.facebook.com/profile.php?id=1349222175 John Rees

      Within 30 minutes for 1 and 2? This can’t have taken more than 3!

      http://jsfiddle.net/7ErnR/

      If that makes me a ‘good candidate’ then you just made my day :)

  • comSciStudent

    As a student and aspiring engineer, I found this post to be very interesting and useful.  I bet you have a great team.  Thanks for the effort!

  • Someguy

    How is making the short_url the clustered index going to help any?  You’ll be accessing those randomly, not sequentially, right?

    • TK

      This was totally my mistake, it doesn’t help any at all.

      • Some other guy

        Maybe change “clustered” to “covered” index instead (or whatever DB specific syntax that accomplishes the same thing — like INCLUDES).

  • Anonymous

    Seems to me that this has the worst aspect of hiring nailed:  You will never hire anyone that is smarter than the person you have administering this test.   Maybe that’s part of the goal, but you’re only going to nail the good candidates and never the great ones. 

  • http://www.facebook.com/daniel.ford Dan Ford

    I actually ask this exact same question in my interviewing – I hope I still can now that you’ve gone and posted about it :)

    I like it for all the same reasons, but I usually don’t get to cover as much as you seem to. How long does it take a good candidate to get through all the areas you’ve described here? My interviews are generally 45min to an hour and I find it is barely enough time to get a basic design out of someone and then dive into one or two specific areas.

  • Iworkatgoogle

    So you want an engineer with all these skill sets to not be ambitious by doing their own start up, and to instead work for you? 

  • Nelson

    Not enough focus on people skills. One of the most important thing Engineers do is work with other Engineers. Another thing they have to do, at least the good ones, is understand and prioritize what their customer wants in such a way that the customer gets the most value for their buck.

    • http://www.facebook.com/gregsabia Greg Sabia Tucker

      I don’t know about that. There is a difference between “people skills” and “programming skills.” Most of the time, the two do no come hand-in-hand. In my experiences the best programmers are usually introverted; the ones who like to wear headphones all day and write code in their own little world. Programmers should not have to inherently be “people persons” because that type of construct has nothing to do with the actual development and architecture of the many constituents of a good application.

      • Nelson

         My mistake. I thought we were talking about Engineers. If you just need a programmer then what you say is correct.

  • Anonymous

    This is a nice v1 url shortener.  It is very close to what I’d expect to find in a rails shop.  Depending on expected load, I’d rather make this “less railsy”.

    Ex:
    Throw varnish in front of rails, set your TTLs there.  additionally, have your varnish boxes log the request headers and parse the log files in a separate daemon.  While you propose pushing this into an async job (good!) you are still requiring that the async job service be able to accept new jobs in order to redirect and you are still performing the request parsing within the request/response cycle of your controller =/Finally, I would wonder why you wanted to store all of your detailed fact records about each user instead of aggregating facets you already care about and using off-the-shelf (splunk, loggly, etc) log management for detail storage and analysis.

    At the end of the day, this is a good question to ask of a senior/architect level person because it pulls in their knowledge not only of your application framework but also of proven components outside of that stack.

  • Jonathan

    “Create a database index on the short_url field”

    I would be very hesitant to say that because it seems so obvious.

    “Store links created in the past 30 days in an in-memory cache for easier lookup”

    I would also consider activity of links. You may get some links that are hit every second excluded from the cache. A simple LRU cache might give a better result (probably putting new links in the cache as they are created). Or alternatively you can predict the most active links each hour from the visitor stats. Some links are visited only once and should probably not be cached too long (unless we continue to see hits). You would probably want to do some sort of analysis here to find the best approach.

    I can imagine this technique working very well. You need to be sure the potential employee is told the scope of the project. A link shortner used by 1 company is very different to a link shortner used by the masses. I’m sure you covered that in the interview though.

  • nevermore

    You are going to end up getting great on the spot thinkers …

    Unfortunately this says next to nothing about their abilities as developers, unless you’ve done that sort of thing before (and really, what’s the point if they have?) all you get is “how I solved this before”, if they haven’t, according to you, anyone who would need more than half an hour to come up with a solution would get cut. You don’t attach too much value to programming languages but you attach a great deal of value to domain specifics.

    In my opinion and experience, good programmers will kill you with questions every step of the way, they’ll question every decision, every detail (although not voicing every one, some things we can actually assume). An interview is too short and stressful to think things through and you will get good people telling you dumb things because they don’t have the time to think them through.

    I have some experience working alongside good developers and alongside some REALLY bad ones, one of the key differentiators I have noticed (and I wish I could assume it was common knowledge by now) is that good developers will question you as though they were five you olds. “Why”? … but why? … no but why? … they will drive you crazy this way, they will seemingly not get a single thing you ask and they will seem to be struggling with concepts that even non-technical people would get. I say “seem” because these developers rarely deliver anything that is not exceptionally well tailored to the problem in record time with code ‘real’ five year olds can read.

    The rockstars(TM)  always get it right away, they know what you want before you do and they will code it in half the time, it will look flashy and will wow you to no end. And then you try to use it. You’ll find buttons in the wrong positions, the layers are all mashed together, you change one thing and the entire application crashes. You’ll find anything cpu intensive is slow and their proposals to fix it always involve multi-threading (because that obviously fixes everything). These rockstars will always conveniently be somewhere else when the entire thing comes crashing down and if you’re lucky you’ve got a few real devlopers to fix things.

    I’m obviously generalizing on both ends of the spectrum but I think “thinking on your feet” is not a good gauge for a developers ability, if anything I’m weary of the ones that do.

  • Pingback: 算法精进 – 自勉 | A Wing by Wind

  • Pingback: 程序员精进 – 自勉 | A Wing by Wind

  • Anonymous

    Kudos on your interview question, I appreciate programming interviews that test real-world-ish programming ability. Most puzzles and algorithm questions simply test whether or not a candidate has heard that question before. Walking through a realistic problem from start to finish, on the other hand, gives both the candidate and employer a good sense of working together.

    My own thoughts on the matter (much more snarky and less instructive than yours):

      http://joshcarter.com/software/dear_programming_job_applicant

    Also, on your comment, “people that can cross between frontend, backend and design are the true rockstars and exactly what we’re looking for.” Unfortunately, I’ve interviewed with several web companies that say this, but in practice they don’t interview people for across-the-board work.

    Problem is, the job reqs are written for one side or the other, and the interview goes accordingly. In my case, things go south on salary; the company isn’t willing to pay good money for “just a [x] programmer,” where [x] is the small piece they interviewed for.

    Your company may be very different, of course, I’m just speaking from my own experience. Oddly, I’ve found my best work in embedded systems. I’ve had several employers that truly valued the ability to work from device drivers all the way out to front-end UI — and they put up a salary to match.

    Best regards,
    Josh

  • ZJ

    Really, I think it’s better to ask them to describe the best architecture they’ve built or worked on, then have them draw it… and just keep having them filling.  

    If they can’t do that, then consider going to a wrote scenario – but only as a backup.  If they can’t do the former, then they’ve auto-categorized themselves.

  • http://www.markhneedham.com/blog Mark Needham

    Interesting approach – how long does it typically take you to work through the problem with the candidate? 

    We use a much smaller problem that the candidate works on before hand and then we pair on together. I think that gives you a reasonable idea of telling whether someone can code well but I don’t know that it covers the lead bit that you’re trying to hire for. 

    On the other hand it does help you to see whether they can actually code/have been coding frequently. I’ve often interviewed people for those types of positions who could probably nail a whiteboarding problem but struggle when it comes to actually implementing it because they haven’t done it for ages.

    As you point at the top of the article, it does often end up being about a gut feel anyway and judging whether or not you’d want that person on your team!

  • Pingback: A Primer on Interviewing and Hiring Technical People: Further Reading | Morgan Missen

  • Pingback: A Primer on Interviewing and Hiring Technical People - Morgan Missen

  • Pingback: Interviewing and Hiring Technical People

  • Pingback: A Primer on Technical Interviewing and Hiring

  • Pingback: A Primer on Technical Interviewing and Hiring | MainMain

  • Pingback: Interviewing: Additional Resources | Recruiter Academy

  • chris paul

    The real subject of this post should be- “when the CEO is CTO.”

  • Pingback: Entelo Interviewing: Additional Resources [Updated]