How valuable is one ultimate frisbee player?

A rough estimation of player impact

Update (2023-12-13): The discussion below about “usage rates” was not as good as it could have been. See More on “usage rates” for more details.

Backstory

Probably 90% of this essay was written over 9 months ago. At the time, it felt like it wasn't quite 'clicking' the way I wanted it to, so I left it in my Drafts folder and worked on other things instead.

I might have never published it, until last week Ian French posted his Europe's Top 25 Players List. The blurb for his #15-ranked men's player caught my eye:

15. Sofiène Bontemps

Two main reasons I have him ranked so high: the pulls are absurdly good and worth a couple of points a game, and he just comes up with clutch plays over and over. He may not be the guy Belgium/Mooncatchers turn to every point to run things through but so many times when they struggle it’s Bontempts pulling out something spectacular to turn the tide. Those types of players are priceless.

I do think pulls are important—I've written about them before. But as I'll explain below, I think it's almost certainly impossible that one player's pulls are worth "a couple of points a game". I don't know how seriously Ian took that comment, so I don't think of this article as a rebuttal to it. Rather, I take that comment as a hint there's room for more discussion about player impact in ultimate frisbee. It's the nudge I needed to get this article out.

---

Introduction

There hasn't been too much discussion of advanced stats in ultimate frisbee, but there has been a little bit, like the Better Box Score Metrics articles on Ultiworld (Side note: Did we ever figure out who Paul Würtztack is? This is one of the most important unsolved frisbee mysteries!! I do have my completely unfounded guesses...).

In this article I'm going to write about advanced stats, without actually doing the hard work of trying to collect the data and calculate them. Rather, I'll explain on a basic level how we determine what impact one player has, and I'll show some estimates for what those numbers might look like.

Figuring out one player's value is a complicated task. We can't just ask "what did they do?". We have to ask "what did they do above and beyond what other players could have done?".

To give a specific example, Allan Laviolette scored 2 goals per game for Raleigh in the 2021 AUDL season, but missed 2022 due to injury. If we were to be overly simplistic, we could say "he scored two goals a game, so he's worth two goals per game to the team" (not to mention his assists).

But if he's not out on the field, someone else is playing in his place. They'll also score some goals and throw assists. To determine Allan's unique value, we'd need some way to calculate the value he provides that couldn't be provided by a player who might play if he wasn't available.

This is, of course, a hypothetical question with no easy answer. But there are ways we can attempt to get a rough answer. There are many stats that try to do something like this, but generally we can think of them as trying to answer the question posed above.

In other words, how does the group of players A-B-C-D-E perform if player E is replaced with player F, G, or H? The difference can tell us how effective player E is. How does the team perform "With or Without You"?

---

Better Box Score Metrics

Better Box Score Metrics has at least one article where they try to calculate individual player impact in terms of goal differential per game: Better Box Score Metrics: The Most Impactful Offseason Moves. In that article, the answer they come up with matches pretty closely the calculations I come up with below: the best players in the league are worth about 1.5-2 points of goal differential per game.

This calculation is for the AUDL, where scoring 20-30 points per game is normal. Here's their list of the most impactful players who changed teams before the 2022 season. Their table doesn't display well on my monitor, so I've copied the data into a simple spreadsheet:

Since impact is relative to each team's baseline performance (as mentioned above), they present two separate lists—the “departing” players whose impact will be the largest loss for their old team, and the “arriving” players whose impact will be the largest gain for their new team. (As a quick example, Antoine Davis moving to NY Empire doesn't have a huge positive impact on that team, because Empire were already very good.)

Of course, this list is only players who changed teams, so the best players may be worth a bit more. However, the list includes top players like Sean McDougall, Jordan Kerr, and Jon Nethercutt. Not many players are going to be more impactful than those three (both Kerr and Nethercutt were 1st Team All-AUDL in the 2022 season).

---

What can basketball tell us?

It's great that Better Box Score Metrics has calculated this data, but what if it's completely wrong? I doubt it is, because it matches closely with estimates we'd expect from a rough comparison between frisbee and basketball.

In his book Thinking Basketball, Ben Taylor analyzes NBA data to determine how much the best players improve their team. The answer he comes up with is that all-time players have been worth an improvement of about 7-10 points in a team's average point differential:

In other words, it's 95 percent likely that [Nate] Thurmond improved [the 1967-1968 Golden State Warriors] by at least 7.1 points. And that's the best observable result in NBA history.

According to Ben Taylor's calculations, one great player can't turn a bad team into a champion—the point differential is too great. A league-worst team like the 2012 Bobcats would still be below average after adding a single player.

Very roughly, teams score about 100 points per game in the NBA. (In 2015, the year before his book came out, the average was exactly 100 points. It's since risen to 114 points per game this season.)

To make things easy, we can use nice round numbers: let's say the best basketball players improve their team's point differential by 10 points per game in a sport where there's 100 points per game.

If we blindly apply this ratio to ultimate frisbee, we can say that the best ultimate players would improve their team's point differential by about 1.5 points per game in a club-division game to 15, or by about 2 points per game in the AUDL.

This is a solid starting point, but I think there's a few factors in frisbee that make individual players less valuable. There's also at least one factor that can make great players more valuable compared to basketball.

---

Usage in frisbee vs. basketball

I expect frisbee players have a lower individual impact than our "NBA ratio" of 10-per-100 or 10%. There are a number of reasons that it's harder for any one player to affect a frisbee game.

First, playing time. I'll use LeBron James as an example, widely regarded as the 2nd best player of all time. Over the course of his career, Lebron has averaged 38 minutes per game, equivalent to 79% of the game spent on the court. In the playoffs, that number is 41.5 minutes/game (86%).

From looking at a few random AUDL games, the top players are usually playing less than that. I clicked through a few random games and found the players with the most play time playing 70%, 55%, and 57% of the game. Last year's championship game gave me the biggest outlier example that I found in my quick search: 83%. The player playing the most points in the AUDL championship game is still playing less than Lebron plays in an average playoff game. "Percentage of points played" (AUDL) is of course not exactly the same as "percentage of a game's minutes played" (NBA), but I think they're similar enough for a valid comparison.

If the best players are spending less time on the field adding value for their team, the total value that they're able to add will be lower.

Another way to look at play time is as a percentage of the total "person-minutes" played by the team. Basketball has 5 players playing a 48-minute game (NBA rules), so a player like Lebron is playing:

(80% of the minutes) * (1 of 5 players on the court) = 16% of the total person-minutes.

An ultimate players is playing:

(70% of the minutes) * (1 of 7 players on the field) = 10% of the team's person-minutes.

In other words, there are more people in frisbee than basketball, so even if they play the same amount, the frisbee player's impact is likely to be slightly less.

---

A second factor is how the rules of the game limit individual effort. On offense in basketball, a star player can move with the ball by dribbling, and create scoring opportunities all by themselves. The NBA uses the stat "usage rate", which roughly means how often a player is directly involved in the play when they are on the court. Some NBA stars have a usage rate of 35-40%.

From here. Usage rates in the NBA have been rising recently.

A "possession" in frisbee doesn't really map cleanly onto the idea of a "possession" in basketball. There are probably players who touch the disc on 90%+ of their team's offensive possessions—but this is clearly a poor analog for usage rate, since a frisbee possession is much longer.

I doubt there are players that are directly involved in anywhere near 35% of their team's throws. This would be like a player getting the disc every third pass, every time they are on the field. (Sanity check: Last year, the LA Aviators had 3,685 total completions, while their star Pawel Janas led the league with 860 completions—23% of the team's total.)

Since you can't move with the disc in frisbee, a great frisbee player either needs someone to throw the disc to them, or someone they can throw the disc to. So while a single, individual talent may be more impactful in basketball, in frisbee there might be more benefit to putting two great players together than you get from adding a second great player in the NBA.

This point—that basketball is more star-focused than frisbee—seems to be commonly accepted. In an emailed response to my Ultimate and basketball article, Travis Norsen told me:

I've always felt that basketball generally tends to have a kind of "1v1", "our best guy against your best guy with the game on the line" mentality. Of course, there have been great *teams* that everyone recognizes were great precisely because they played like teams (e.g., 2014 Spurs, the recent Denver Nuggets) but these are kind of the exceptions that prove the rule. The culture of basketball largely celebrates the hero individual (Jordan, Lebron, ...)

Although one example is not proof, this comment highlights an idea that seems to be generally accepted—basketball is more individualistic, and reliance on stars is greater.

---

On defense, the rules of the game also limit the opportunities for individual greatness in frisbee. The NBA court is small enough, and the difficulty in scoring is high enough, that a player can have a direct impact on many defensive plays — they can guard the area close to the basket, and within a second or two also be effectively guarding the three-point line. In Ultimate and basketball, I used this clip as an example of how an NBA player can guard the entire court.

Ultimate is a much more wide-open game: good throwers can hit the endzone from 90% of the field (a field that is much bigger than a basketball court). So a great defender in ultimate won't be able to guard the deep space while also taking away the 15-yard under from another cutter.

---

Culture and off-field impact

Let's change gears. There are some factors that make a player valuable that aren't accounted for in the advanced stats, which only measure on-field impact. One example is team culture: a player's ability to bring a positive attitude, to inspire other teammates to work hard towards their goal, and to have solid, fulfilling relationships between players isn't captured in advanced stats.

In the NBA, Steph Curry is widely considered to be a player who helps his team's culture. For example, in the 2022 NBA playoffs he came off the bench for a few games when returning from injury, which set a positive example of humility, trust in the team's plans, and belief in the team's goals.

Another factor in a similar vein is that players will play harder when they believe they have a chance to win. In that way, a great player can improve their team even when they're not in the game.

Without a very valuable player, a teammate might feel like "we're going to lose even if I try hard". On a better team that same teammate might believe: "if I try my hardest I can help this team be great". I think it's obvious that the latter outlook will lead to that teammate playing better, so in that way, a valuable player can improve the team even when they're not on the court.

Another factor not captured in the advanced stats is that winning players want to play for winning teams. We see this often in the NBA: good players want to play for good teams, where they have a chance to win the championship. Having LeBron is just beneficial because he's good on the court, but because other players will now want to play for your team (because they have a better chance of winning).

Attracting one good player to your team won't turn a bad team into a champion, because the impact of any one player is limited. But if that one player allows you to get other good players that you wouldn't have been able to attract otherwise, then the one player could be worth a lot more than the advanced stats say.

---

Replacement Players

One final wrinkle in comparing frisbee to the NBA is that value is relative to the other players in that league. You could think: how much better is Player X compared to the average player in the AUDL?

Basketball analysis measures value relative to a replacement player. A replacement player is, roughly, someone of the skill level of the worst players that are playing in that league—players that are just on the border between the big leagues (NBA) and the minor leagues (NBA G-league).

(The reason they compare to the worst player instead of the average player is that an average player will already be playing for another team, but a replacement player will be available to play for you at short notice, because they're not quite good enough to have another team already.)

This is one area where I think the best frisbee players are likely to be more valuable, compared to NBA players. In the NBA, pretty much every player has been playing since they were <10 years old, and are practicing like it's their full-time job (and have been since they started high school).

The same is obviously not the case in ultimate frisbee. Because the player base is smaller, there will be bigger talent gaps at the extremes. This difference does make top ultimate players more valuable relative to basketball, but I doubt it's enough to make up for the other factors discussed above.

The replacement player concept also tells us that great players will be even more valuable when the skill level is lower. The best AUDL players are only worth ~1.5 points per game. But if they play in their local summer league, where the least experienced players might not even be able to throw a frisbee, they may be worth much more—perhaps 4 (net) points per game or more.

Getting back to the question at the beginning, we should think about replacement players when we try to decide how much a player's pulls are worth. The question isn't "how good are their pulls?" but "how much better are their pulls compared to the person who would be pulling if they didn't?". Given that they play for an elite team, their teammates are probably solid pullers as well (though those teammates are probably noticeably worse, given that this player's pulls left such an impression on Ian French!)

---

Final thoughts

The data suggests the best players in frisbee improve a team's point differential by 1-1.5 points (in a club game to 15) or 1.5-2 points (in an AUDL game to 20+). This accounts only for a player's on-field performance—off the field, players can be valuable in ways that won't be captured in these stats.

Given that total on-field impact is capped at about 2 points per game, I think it's nearly impossible that one player's pulls are alone worth that much—pulling is just one part of the game.

This is still a very new field of ultimate frisbee analysis, and I'd love to see more work done to calculate player impact. I'm probably not quite enough of a data scientist to do it myself. Maybe one day we'll have an actual answer to the question of how much a pull is worth.