As Maxwell Wessel outlines in a brilliant post in the Harvard Business Review:
"... it is more effective to evaluate managers by looking at the situations they have been in than their track record of success. By looking at past success without considering the situation in which the manager was successful, we neglect to account for the wisdom accrued over time when a manager faces a specific set of problems."
Not only does this more rigorous interviewing approach look beyond years and instead look at the actual management challenge the executive faced, it also looks beyond success/failure and looks at what was learned.
Success can be the result of timing, luck and structural and environmental factors the candidate had no part in creating. On the other hand, wisdom comes from tackling tough challenges, whether they were ultimately successful or not. Failure can be a great teacher ... or not. It all depends on the student.
If you want to stop wasting your time on the irrelevant, superficial aspects of interviewing, and start understanding the deeper elements that predict the success of your new hires, read our post on How to Conduct a Job Interview so Top Performers Actually Want to Take Your Job. Or, if you prefer your research and information to be more attractively formatted, just download our Employer Guide to Interviewing.
Of course, interviews are only one component of a great hiring process, our Resource Center has additional topics you might find helpful:
How to Write Job Descriptions to Attract Great Candidates
How to Handle Glassdoor Reviews
How to make your Hiring Process More Certain, Predictable and Consistent
How to Evaluate Your Own Hiring Process
How to Replace Underperforming Employees
Disclaimer: This advice is primarily for professional hiring in a large metropolitan area. Our perspective is shaped by our work in a retained executive search firm, conducting searches for CEO and senior staff positions. We've completed over 600 searches for associations and other nonprofits in major metropolitan areas like Washington DC, New York, and Chicago, but not all of our advice will be relevant if you are interviewing for other types of positions in other job markets.