Let’s say Sechaba is a salesman in a car dealership, and that he is incredibly good at what he does. Each month, he outsells all his colleagues: for every car they sell, he sells two. Sechaba is really good at his job not only because he can assess the needs of his clients as soon as they walk through the door, but he also has a Schalk Brits-smile that fosters confidence in him and the cars he sells. Sechaba is the Tony Robins of car salesmen.

And then the position of manager of the car dealership opens up. What to do? Should Sechaba be promoted, given his excellent performance at the lower level? Or should his colleague Whitney be offered the role of manager, because she not only has a financial degree but is a conscientious and hardworking team player. Whitney seemingly has all the attributes that would make her a better manager, yet what will it do to team morale (and the incentives for future salespeople) if Sechaba is not promoted? Would it not seem incredibly unfair?

Usually, the theory suggests, that is why Sechaba will be promoted over Whitney, even though Whitney would have been the better manager. This is called the Peter principle, a hypothesis developed by Laurence Peter which states that an employee is promoted based on their success in their previous job until they reach a level at which they are no longer competent.

The problem with this hypothesis, though, is that it has been incredibly difficult to find real-world evidence that such promotions are actually bad for firms. One argument, of course, could be that such promotions are actually good for firms: it is the incentive of promotion that inspire salespeople to work as hard as possible. Even if there are costs involved in not appointing the best managers, if one would remove promotion incentives, sales will collapse. But the counter argument is that salespeople could be easily incentivised through other channels rather than promotion, like rewarding them for each sale. Which effect dominates has been a much-debated question.

A new paper in the Quarterly Journal of Economics solves this conundrum. Alan Benson, Danielle Li and Kelly Shue trawl through a massive dataset of sales transactions for a panel of 38 843 workers, 1553 of whom were promoted into managerial positions across 131 US firms to search for evidence of the Peter principle. And they find much evidence to support its existence: frequently, those staff members with the best managerial skills are overlooked for promotion; instead, those staff with the best sales records get the better job.

This evidence could, of course, be interpret in two ways. First, firms make mistakes and promote the wrong people. Or, second, it could imply that the incentive benefits of promoting based on sales performance justify the costs of promoting workers with lower managerial potential. And the evidence, according to the authors, seem to suggest that firms are not so foolish as to just be making mistakes: They find that firms carefully consider the costs of the Peter principle. In settings where salespeople are rewarded with strong pay for performance and where managerial roles entail greater responsibility, firms place less emphasis on the sales performance of new promotions. But often firms just appoint the best salespeople, without considering the other qualities that might be more applicable to managerial positions.

This seems somewhat surprising because of the significant costs in appointing the ‘wrong’ people. To assess the magnitude of the costs associated with firms’ existing promotion policies, the three authors compare the managerial performance of promoted workers with the predicted managerial performance of workers who would have been promoted under a counterfactual promotion policy that maximizes expected managerial quality. ‘We find that average managerial quality, measured by value added to subordinate sales, is 30% higher under this counterfactual policy.’

Yet these costs should be weighed against the costs of lower morale and sales performance of promoting poorer performers. Returning to our example, if Whitney is promoted, Sechaba may simply find a different firm. And, indeed, this is exactly what the authors find: ‘We find that workers are much more likely to leave the firm if a teammate with worse sales performance is promoted.’

The evidence therefore suggests that firms are largely aware of the difficult trade-offs in appointments. If this were true, one would expect managers that have important roles to be more carefully selected than managers that are not that important. And this is exactly what the authors find: in one test, they compare firms where managers manage larger teams versus those with small teams. If firms do indeed carefully consider the costs on both sides, then firms with larger teams should value better managers more than firms with smaller teams. And it is so: ‘[Our] findings suggest that when the costs of managerial mismatch are particularly high, firms are more willing to sacrifice the incentive benefits of performance-based promotion tournaments to promote better managers.’

Many factors play a role in promotion decisions. It would seem that the decision to promote managers is a careful trade-off between having the right skills for the job and rewarding good performance at a lower level. Sometimes its best to promote the best person for the job; at other times, though, you’d rather promote Sechaba because failure to do so would come at the even greater cost of low morale and poor incentives.

These lessons apply not only to sales firms, of course. In any hierarchy where skills vary at different levels, the question of whether to promote based on capacity or performance depend on many factors. Yet one thing is true: in the competitive market, a firm that repeatedly miscalculates the trade-offs, will quickly be replaced by those firms that make better decisions. The decision to promote either Sechaba or Whitney to manager is, ultimately, a profit-maximising (or cost minimising) decision.