Back of the Napkin #2: Selection and Treatment Effects

This is part of an ongoing series by Brian Shih and Sebastian Park where we are exhaustively going through this interview with Jeremy Giffon on the Invest Like The Best podcast piece by piece and writing our thoughts along the way.

Fair warning: This piece includes a bit of inside baseball about, well, baseball. See the footnotes for all the gory details.

As a reminder, we’re planning to write one of these a week, and you can follow along here on Substack, or Seb’s LinkedIn. If you have any suggestions, comments, or want to join in as a collaborator on a particular topic, please reach out! You can reach Brian on Twitter and Sebastian on Twitter and TikTok.

Last time we talked about The Perfect Business and the concept of leverage, and left off here:

And for the best ones [the best businesses], there's also this idea where the organizations are either selection effects or treatment effects. And so a selection effect would be Harvard or a modeling agency, where most of the prestige comes from picking the people. Whereas the seals [The Navy SEALs] or something like that, to some extent, will have a treatment effect, you become a different person by going through this program

And I think the best organizations do both. By virtue of being selected by the company, it really helps you for your entire career, and it gives you something. Like I really admire certain organizations where you can just tell someone's been at a consulting shop or something or been at a really good bank in just the way they write emails or the way they think or whatever.

And that's obviously a double-edged sword, but I really like this idea of you can just tell the polish and obviously, same with military people. Maybe there's a path to becoming a principal. I don't actually think you need that necessarily, but you should be certainly clear about it.

There's nothing worse than all these VCs that have never promoted from within to a partner, but then you ask associates and analysts there, and they're like, "Oh, I'm going to be a partner one day," and it's like, "No, you're just totally wasting your time."^[1]

Continuing his commentary on the perfect business, Giffon dives into a tangent on selection and treatment effects. If you’re building a theoretical perfect business, should you focus on picking the best people, or training them?

Why not both? If we’re running this imaginary perfect business, unmoored from any constraints, hiring the best and extensively training them might indeed be the optimal thing to do. But for mere mortals operating in the real world without infinite time and budget, we must choose – it is almost certainly too massive an undertaking for companies to set out to do both.^[2]

So which will it be? Selection or treatment?

Sports are a great laboratory to study these effects because the outcomes are extremely measurable – you’re winning or you’re not – so we can observe how selection vs treatment effects play out in different sports.

In the NBA, general managers are lauded for their ability to select players, both through a draft and through free agency. NBA teams focus on building the best possible roster, and while the players of course undergo training, no amount of training is going to turn your ninth bench player into Steph Curry. NBA teams are selection effect organizations.

By comparison, in baseball, training appears to be very effective. The LA Dodgers, staffing one of the largest analytics staffs in the league, seems to be able to consistently train their pitchers, develop their batters, and turn late round drafted players into consistent major leaguers, delivering value up and down the roster.^[3] Though selection is important (picking up guys like Mookie Betts and Freddie Freeman certainly helps), treatment appears to matter a lot in baseball.

Please direct any ire at this image towards Sebastian.

Basketball and baseball are both professional sports, with millions of dollars pouring into their teams and organizations, so why are they biased differently towards selection vs treatment effects?

There are a couple of interesting things at work here. First, basketball is a strong link sport, whereas baseball is a weak link sport:

I first became familiar with the concept of strong and weak games in The Numbers Game, a book by Chris Anderson and David Sally on soccer analytics. In it, they define a strong link game as one where the team with the best player usually wins. In contrast, in a weak link game, the team without the worst player usually wins.

In other words, superstars matter a lot more in basketball than they do in baseball.

Your superstar starter gets to play 30-40 minutes of every game (assuming no injuries) accounting for 12-15% of total player minutes and often 25-60% of total point share, but your best hitter or pitcher in baseball is only one of nine batters in the lineup (11% of batting) or pitching at most once every four to five days.^[4]

So it’s natural to focus on selection for basketball, but in baseball – where superstars impact the game far less – improving your weakest link matters more, and we see the benefits from treatment effects.

But baseball’s benefit from treatment effects extends beyond the actual game and into the structural incentives of the league itself.

• In the NBA, drafted and undrafted rookies sign contracts lasting only up to four years.

• In baseball an organization can hold a new player for the duration of their minor league play, and then another six years minimum (usually seven!) once they make the majors.

All this guaranteed time with the same organization means they are more likely to see the fruits of their training labor pay off, and are therefore more willing to invest.^{[5] [6]}

Okay enough sports. Back to startups – should you, a founder of a new fund or startup, optimize for selection or treatment effects?

Any new upstart organization with no amount of scale (in tech or otherwise) is playing a strong link game. Every individual accounts for a high percentage of the team’s output, and there is no bench. Viewed through this lens, the answer is clear: early stage startups are strong selection effect businesses.

This makes sense (and partly explains why investors care so much about early teams), but then why do larger or more mature companies tend to tilt towards treatment effects over time? The answer is ultimately cost and efficiency.

Recruiting and employing superstars is expensive and time-consuming. When your team is five people, it’s worth the effort. When your team is a thousand people, training the weakest links is likely to move the needle more than hiring a single star player.

The interplay between selection and treatment isn't just academic — it's the backbone of team building, whether in sports or startups. For fledgling ventures, every hire counts; it's all about selection.

As you scale?

Training starts to matter more.

Giffon’s perfect business may be able to do both, but most of us can’t afford to. For those of us working under constraints, determining whether you’re playing a strong or weak link game, and identifying the incentives that exist in the ecosystem you’re playing in is critical.

In the end, figuring out what game you’re playing is as important as how you play the game.