FAQ

FREQUENTLY ASKED QUESTIONS

Q: What is SPARQ?

Check out the documentation page I’ve put together.

Q: What is simScore?

Bill James, the man generally credited with the start of the sabermetric movement in baseball, is behind the idea of a similarity score. The wiki page does a good job of explaining the basics, and Baseball-Reference talks about the calculation of their similarity score.

I developed a similar idea, but for a SPARQ-based similarity score applied to physical measureables of NFL draft prospects. The idea behind this is that specific athletic profiles have different rates of success. It’s one step further than SPARQ — instead of the only question being “How athletic are you?”, it asks “What kind of athletic are you?” Of course, SPARQ figures into the calculation. The full list of inputs: height, weight, arm length, forty-yard dash, ten-yard split, short shuttle, 3-cone drill, bench press, vertical jump, and broad jump.

Q: Does SPARQ take height into account?

No. The list of inputs for my NFL version of SPARQ, pSPARQ, are as follows: weight, forty-yard dash, ten-yard split, short shuttle, 3-cone drill, bench press, vertical jump, and broad jump.

Q: Well, shouldn’t SPARQ take height into account?

With any metric, you have to establish bounds. It’s difficult to create something that can measure everything, because each added layer of complexity further obscures the result. SPARQ is about a few different concepts — how fast can you run, how high can you jump, and how quickly can you change direction. Weight is a critical factor in all of these — your momentum is different in the 40, power generated is different in the vertical and broad. SPARQ is a weight-adjusted metric because weight directly impacts the test results.

Height does matter, and a taller athlete may be preferable. My personal preference is to do a good job of adjusting for the weight and tests, and then interpret the result in the context of each player’s physical build.

I would guess that the tall, lanky receiver is probably SPARQ’s biggest blind spot — it doesn’t appreciate that tests are harder for Sidney Rice or Josh Gordon to perform than they are for Brandin Cooks or Bruce Ellington. Metrics don’t need to be perfect if we do a good job of understanding what they’re saying and what they miss.

Q: How do you decide which results to use when a player participated at both the combine and his pro day?

Well, there’s no perfect method for dealing with this issue. The answer is that I take the better result and combine the two performances. So, if a player jumps well at the combine and runs well at his pro day, he gets credit for both and the poorer results are washed out.

Now, this does lead to a little bit of inequity. A player’s pro day is a friendlier environment than that which is encountered in Indianapolis. Times are a little faster and jumps are a little higher.

It isn’t a perfect system to use the best-of composite; however, it likely does well enough for our purposes, even if we overrate a player’s speed from time-to-time, as might be the case with Anthony Barr. We’re not building a rocket ship, and a reasonable change in any one category isn’t going to result in a drastic shift in SPARQ.

I’m not in favor of throwing out pro day data for a few reasons. First, many players don’t participate in every event at the Combine. It’s necessary to add in a result here or there just to the data necessary to complete a pSPARQ calculation.

I also don’t understand the concept of throwing out data because it doesn’t occur at the same site. It’s data. We can interpret data, perform studies on data, and adjust data. I’d rather try to establish what pro day data means than throw it out altogether. I’d estimate that 80% of the SPARQ results in my database use pro day data. One of the goals I have with SPARQ is to find the late-round SPARQ values, and those guys aren’t even at the combine.

Q: Stephen Hill, Vernon Gholston, and Greg Little were all great athletes. Why do you promote SPARQ when there are so many obvious misses?

It’s a fair question. There are a vast number of players who have tested out well but failed to make an impact in the NFL. The issue here is hit rate. While Wes Welker had a successful career as a marginal athletic tester, there aren’t very many Wes Welkers around. It’s hard to identify that player, even for the best of scouts. I prefer to take my chances with the better athlete, and the data bears this out: good players also tend to be good athletes.

Q: Aren’t you just identifying the tall and fast players? I don’t need a formula to tell me which players are tall and fast.

To some extent, there’s truth here. Fast players are often good athletes!

Still, it’s not really accurate. SPARQ doesn’t even take height into account, as discussed earlier. What SPARQ does for us is dial back the influence of the 40-yard dash.

I look at combine data constantly, and I’d still stutter a bit if you asked me about a given defensive lineman’s broad jump. Because none of us are really great at understanding the bounds of these values, we fall back on the results we do understand. This means the 40 time is emphasized to great lengths and plays a tremendously large part in shaping the general opinion of a prospect’s athletic potential.

By using a metric like SPARQ and normalizing by position, we can weed out the “40 bias” and have a more holistic understanding of each prospect.

It also isn’t just about tall and fast. By looking at player profiles and success rates, we can determine that some prospects aren’t worth their projected draft position, even considering their plus-athleticism. Similarly, some players who test on the lower side of the scale tend toward an athletic profile that is often successful.

Q: Aren’t the tests imperfect? Times are subjective and depend on the environment/timer.

Yes, they are absolutely imperfect. This is where error dilution helps. As discussed earlier, we can see some differences between the combine time and pro day time for a given player. While some of this may be explained by a friendly stopwatch operator, the reality is that we all have good days and bad days. The Central Limit Theorem tells us that our test results will form a normal distribution. Sometimes we do well, and sometimes we don’t. Most of the time, we’ll fall somewhere near the middle (i.e. our actual ability), but outliers happen.

A composite metric dilutes the importance of any one test. A botched 40 start can’t submarine your SPARQ entirely, and an artificially great one doesn’t make you Calvin Johnson by itself. While each of the test results may be individually flawed, SPARQ helps us to weed the inherent error out.

Q: How do I know when a player’s SPARQ is good? What’s bad?

I believe that we need to relate a player’s athletic ability to their positional average. Vince Wilfork ran a 5.08, but weighed 323 pounds and managed to broad jump 8’5″. He’s a great athlete relative to other nose tackles, but his numbers look appalling next to pretty much any non-Jarvis Landry wide receiver

To this end, I’ve calculated positional pSPARQ statistics which allow us to calculate the player’s standing relative to peers in terms of z-score. A 0 z-score would mean a player is average, while a 2.0 would mean he’s two standard deviations above the peer average. If you’re more comfortable with percentiles, there are online calculators (specify a one-sided distribution) which allow for conversion from z-scores.

As of February 2015, z-scores are calibrated to the league average athlete. This means that a 0.0 z-score for a draft prospect represents the 50th percentile on an NFL roster.

Q: What is UQI?

UQI is a measure of how unique a given profile is. I did a brief write-up with the pertinent details: Similarity Scores and Uniqueness Index.

-ZW

12 thoughts on “FAQ

    1. zachwhitman Post author

      The problem is that the 20-yard is often not available. If it was commonly reported, I would likely add it to the inputs for some positions.

      As 40 and 10 are reported for almost every prospect, it’s easier to stick with only using them.

      Like

      Reply
  1. dankreso

    I have a question about weight and pro day/combine times. Do you pick the best results from either the combine or pro day in each drill to input into the SPARQ calculator? If so, how do you account for a possible change in weight from combine to pro day if using both combine and pro day data? Love reading the blog so far – keep up the good work!

    Like

    Reply
    1. zachwhitman Post author

      Thanks!

      I cover the issue of pro day vs. Combine in the site FAQ. Weight is really the only one I struggle with, and it’s pretty much a case-by-case basis. I may end up using the average of Ty Montgomery’s 2 weights. It’s also unlikely that a player will gain enough weight to dramatically influence their score.

      Like

      Reply
  2. Pingback: NFL Draft 2015: Breshad Perriman is fast - Sports Train

  3. Dan

    For players that don’t participate in all the drills, how do overcome the lack of inputs to calculate a pSPARQ? For example, Amari Cooper didn’t participate in the bench press like many other WRs in 2015 and the past, but he and many others still have a pSPARQ.

    Like

    Reply
  4. Pingback: NFL Draft 2015: A quick note on Davis Tull, the outlier - Sports Fire

  5. Pingback: Byron Jones, Bud Dupree Among Best Athletes In 2015 NFL Draft | NFL Draft | NESN.com

  6. Pingback: NFL Draft 2015: Roster Mirroring on the Offensive Line - Sports Train

  7. Dr-P

    One of my favorite data points, especially for the OL, is date of birth. Younger players who are competitive in college have room to develop in the NFL. There is a noted growth spell after their first off-season. The best example is Tyron Smith who was a teenager when drafted and has already been an all-pro. He has signed his second contract and is still a player 25 and younger to watch.

    The counter example is those college players who do well but are older. I contend that they do well in college BECAUSE of their physical maturity. Yet when drafted, they often do not have much room to develop. The best example was Danny Watkins.

    I know you track age, but how is that used in your data?

    Like

    Reply
    1. zachwhitman Post author

      At skill positions, I read Jon Moore’s work at Rotoviz. At other positions, it goes into my thinking along with production, ability, and SPARQ. I want to tend toward a young class.

      Like

      Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s