Should Developers Be Allowed to Use AI in Technical Interviews? [2024 Research]
Many developers are adopting AI tools to enhance productivity and code quality. This shift brings forth challenges in accurately evaluating technical skills.
As interviewers looking to assess which developers are the best fit for a given position, we have an important choice to make: Should we allow candidates to use AI during their live coding exercises?
We began by discussing the issue internally, and the consensus was that all things being equal, we would prefer to allow the use of AI at some level. The reasons for this are simple:
- We want to see how developers solve realistic problems in a realistic situation
- Developers already use many external tools while coding (such as search engines and code examples from the web)
- How a developer leverages tools such as AI is an important part of their skill set that will ultimately affect their productivity
- The technology landscape will continue to change, and a less rigid, rules-based approach to vetting will lead to more flexibility and adaptation
That said, we recognized the risk that allowing AI use could end up making it more difficult for us to accurately evaluate candidates, leading to a lower placement success rate.
To help us make an informed internal decision on our AI policy, we embarked on a research project. Our exploration centered on the potential benefits and challenges of allowing candidates to leverage AI tools during technical interviews. In this article, we’re sharing some of our core findings and how we’re refining our vetting process to remain robust and relevant.
Table Of Contents
A Note on AI and the Technical Vetting Process
Before moving into our research, we want to acknowledge that a broader debate exists in the AI and technical vetting sphere; namely, relying solely on AI to evaluate developers from a technical standpoint. We want to assert that our viewpoint is distinct: we advocate for human-led vetting, because only humans can supply the nuanced judgment and contextual analysis of a developer’s performance in an interview.
