FanPost

The Blind Baseball Scout

One of my favorite topics on the McCovey Chronicles is Minorlines and the debate on the potential of Giants prospects, but since I live on the east coast and nowhere near Richmond or Augusta I’m limited in my ability to assess prospects via the old-fashioned eye test. Aside from a few college At-Bats and the scouting videos on the internet, I suspect I’m like most of us, I get my information from the typical sources, Baseball America, John Sickels et. al. I’ve often thought it would be cool to have a sabermetric system of grading prospects, so I finally got off my butt and did what any industrious McCoven would do an made one.

So, here’s The Blind Baseball Scout, which we’ll refer to as P-SABR (if you can think of something better, please let me know). Today we’ll look at some historical comparisons of P-SABR versus Baseball America’s top hitters (there’s no P-SABR for Pitchers yet) by league (AZ, NW, SALLY, CAL and EL) from 2003 to 2008.

I should note that P-SABR is meant to be a quick and dirty analysis tool, and has some obvious drawbacks. It’s compiled using players with the top 100 plate appearances in the league, so some “hot” prospects who are promoted mid-season don’t qualify for P-SABR due to its sample size constraints. An example of this in 2011 would be Bryce Harper, who made two lists for Baseball America, but did not qualify for P-SABR due to too few plate appearances in both the Sally and Eastern League.

Here are some significant players selected by Baseball America between 2003-2008 who did not qualify for P-SABR due to too few plate appearances.

Non-Qualified players found by BBA

Year

League

Asdrubal Cabrera

2005

Cal

Chris Coughlin

2007

Sally

David Wright

2004

EL

Ian Desmond

2005

Sally

Jacoby Ellsbury

2006

EL

Jarrod Saltalamacchia

2004

Sally

Jason Kubel

2004

EL

Justin Upton

2007

Cal

Matt Wieters

2008

EL

Nate Schierholtz

2004

Sally

Nick Markakis

2005

EL

Nick Swisher

2003

Cal

Nolan Reimold

2007

EL

Pablo Sandoval

2008

EL

Ryan Braun

2005

Sally

Ryan Zimmerman

2005

EL

Stephen Drew

2005

Cal

Yunel Escobar

2005

Sally

While this sample size demand appears to be a considerable limitation, P-SABR seems to be adept at finding players that Baseball America overlooks as well. Here are some players that appear on P-SABR’s rankings, but did not make Baseball America’s top lists (by league).

Players found by P-Sabr, Not BBA

Year

League

Carlos Gomez

2005

Sally

Dan Uggla

2003

Cal

David Murphy*

2005

EL

Kevin Kouzmanoff

2004

Sally

Logan Morrison*

2007

Sally

Martin Prado

2004

Sally

Matt Downs

2007

NWL

Matt Kemp

2004

Sally

Melky Cabrera*

2005

EL

Michael Brantley

2006

Sally

Michael Brantley*

2005

AZL

Mike Fontenot

2003

EL

Nate McLouth

2004

EL

Nick Markakis

2004

Sally

Ryan Raburn

2004

EL

There are also players that P-SABR just did not rate high enough to make its list. The first thing you’ll notice about these eleven players is that 5 of them are (or in the case of Pablo, were) catchers, others like Span, Bourne and Bourjos owe a good to significant proportion of their value to their defensive skills, which represent two areas of potential improvement to the P-SABR system, Positional and defensive valuations. As it stands now, P-SABR has no way of accounting for a prospect potential value based on position or defensive skill.

Adam Jones

2003

AZL

Brian McCann

2003

Sally

Chase Headley

2005

NWL

Chris Ianetta

2005

Cal

Denard Span

2005

EL

Michael Bourne

2005

EL

Miguel Montero

2005

Cal

Nick Hundley

2005

NWL

Pablo Sandoval

2004

AZL

Peter Bourjos

2008

Cal

Ryan Howard

2004

EL

Over the next couple of days we’ll look at comparisons of Giants prospects over the last few years, and then take a look at what P-SABR thinks of Giants prospects of 2011. But first here’s the statistical paper that analyzes P-SABR, complete with hypothesis testing and explanations of the grading system – so be warned –Geeks only!

The Blind Baseball Scout

Abstract

The Blind Baseball Scout (P-SABR) is a Sabermetric model that measures minor league production with the goal of predicting a players future value as, and likelihood to become a Major League Ballplayer. The P-SABR assigns points to players across 9 different statistical areas, the sum of which produces a total score for the entire season. The top players represent the players with the highest scores. In order to test this model the results have been compared against the top hitting prospects from Baseball America (BBA), the nations leading scouting publication, using the proportion of players who make the Major Leagues and the population of “top players” mean WAR value as measurements of value from both systems. P-SABR was run from 2003 to 2008 in five select leagues, the Arizona Rookie league, The Northwest Rookie League, The Sally League, The California League and the Eastern League and compared to BBA’s results. The results of both systems were analyzed statistically on the basis of proportion of players who made the major leagues. The populations of both methods were then tested based on their mean “WAR” value (based on WAR {Wins over replacement}, a Sabermetric valuation of individual players). Both comparisons were tested by the hypothesis that P-SABR would be equal to BBA in finding future major leaguers and forecasting the value of those players at a 98% confidence level, with the alternative hypothesis being that they will not be equal.

Introduction

The goal of this project is to determine a viable process in applying the use of sabermetric data in projecting the future performance and value of minor league prospects.

Sabermetrics is generally considered an effective and objective method in the valuation of Major League Baseball player performance, however because of a number of factors it is not nearly as reliable in predicting the future performance of Minor League players. This process will attempt to address the factors that make the application of sabermetrics to minor league performance ineffective by giving specific skills, flaws and factors greater weight in order to bring equilibrium to the players performance.

In order to measure the effectiveness of this system, I will test the hypothesis that this system is equal Baseball America, the nations leading scouting publication, by comparing their top hitting prospects from 2003 to 2008 in five select leagues, the Arizona Rookie league, The Northwest Rookie League, The Sally League, The California League and the Eastern League to this system. I will compare proportions of ranked players who go on to see Major League playing time in both systems, as a test of proportion. I will also compare the value of those players who made it to the major leagues based on their Offensive WAR (Wins Above Replacement) value as a test of means. In this later test, the mean (WAR value) of both systems will be taken for both systems.

The aforementioned variables that make Minor League predictions so difficult are age, where “very young players, as a whole, return 25 percent more value than expected by their draft slots” (Rany Jazayerli), Strike-out rates , where “it appears as though the success rates for prospect development drop sharply when strikeout rates hit about 22%.” (Minorleagueball.com). Then there are the more traditionally acknowledged variables like Walk Rates, Batting Average, Extra-Base Hit percentage, Home Run Percentage, On Base Percentage. There’s also my own metric Stolen Base Efficiency, which attempts to project the value of a hitters speed and running ability. The best performers in each category are assigned a score, which decreases towards zero incrementally as we reach the median, once the median is reached scores progress negatively incrementally towards the worst performers.

The data will be obtained from Baseball-reference minor league data sorted by Plate Appearances. The top 100 plates appearances will be each sample. This will be used to ensure the largest possible sample sizes. This has the advantage of eliminating top performers who do it over relatively short periods of time, the disadvantage is that sometimes really good prospects spend a short time in a given league because of good performance, are promoted to another league and (consequently) will be undefined under this system.

Notes on the data compilation

Page 1 is the source data compiled from Baseball-reference. Com

Page 2 is the age of the players. As mentioned above age is one of the two primary factors in this approach. Previous research from Bill James and Rany Jazayerli are central to the theory, which states, “The younger the player, the greater the slope of the curve—meaning the greater the rate at which he improves” (Javeri, Rani. taken from the web) It is for this reason that age is weighted so heavily in this model.

AGE

Score

17

40

18

35

19

30

20

20

21

-3

22

-15

23

-20

24

-30

25

-35

As you see here players that who are younger than the mean are given points strictly on the basis of the basis of their age. The farther below the mean the greater the positive score, and conversely the greater the age above the mean the greater the negative score.

Players who are at the mean are given a small negative score. This is primarily because they are often (in this league) college players who have experience that is potentially at or above the level of the league.

This weighting system based on age and the weighting system based on K% are the only pieces that will change from league to league. At the lower levels (NW league and AZ league) older players are given a greater disadvantage (negative score), this is based on the theory that their advanced age and experience is an even greater advantage when the competition is so young and inexperienced. Players with higher strike out rates at lower levels are also given a greater disadvantage (negative score) at the lower levels, based on the assumption that the inability to make contact will be a further hindrance at higher levels when the competition is stiffer.

Page 3 is a simple effort to manage sample size. Sample size plays a very important role in this experiment, as it is the basis of the population selected (the top 100 players based on # of Plate Appearances) and it potentially a major weakness in the hypothesis. It is common for organizations to promote their players who are performing well to a higher league, which often mean that those players (good performers) won’t eligible for this method. We will see many players in Baseball America’s (BBA) top rankings who do not qualify for this method.

Mean PA

468.33

Plate App.

Score

x-525

7

524-475

3

474-425

0

424-375

-5

374-x

-10

A small positive is given for players who have the most PA’s and a small deduction is given to those who have the fewest. This theorizes that the more PA’s one has the better one’s performance will represent ones talent. The mean should fall somewhere in the middle and will be given no bonus or deduction.

Pages 4, 5 and 8 are measurements of one very important measure of value, the ability to not make an out. As Bill James said “the difference between winning teams….is the difference between ‘outs’ and ‘runners on base’ “(James, Bill, The New Bill James Historical Baseball Abstract, 2001, The Free Press. Page 642), these three statistics attempt to measure this skill. First is Batting Average (BA).

<!--[if !supportEmptyParas]--> <!--[endif]-->

Mean BA *

=

Score

1.25

0.3297125

17

1.2

0.316524

13

1.175

0.30992975

10

1.15

0.3033355

7

1.1

0.290147

3

1

0.26377

0

0.95

0.2505815

-5

0.9

0.237393

-10

0.85

0.2242045

-15

0.8

0.211016

-20

0.75

0.1978275

-25

Averages that are at the mean are credited with 0 and are progressively (i.e. Mu + 10% =3) awarded point as players are above the mean.

The next measurement is on base percentage (OBP), where we see a similar grading system where the highest points are gained above Mu + 20%.

Mu ( x) + Mu

X

=

Score

0.2

0.3987

15

0.17

0.3887325

13

0.13

0.3754425

10

0.1

0.365475

7

0.07

0.3555075

5

0.03

0.3422175

3

Mu

0.332

0

-0.03

0.3222825

-5

-0.07

0.3089925

-10

-0.1

0.299025

-15

-0.13

0.2890575

-20

-0.17

0.2757675

-25

And lastly, there is BB%, which measures the percentage of walks a player takes per plate appearance. “1” represent Mu * 1.

Mu (x)

=

Score

1.65

0.13645191

10

1.4

0.11577738

7

1.25

0.10337266

5

1.1

0.09096794

3

1

0.08269813

0

0.9

0.07442832

-3

0.75

0.0620236

-5

0.6

0.04961888

-7

0.45

0.03721416

-10

0.25

0.02067453

-15

Next on page 6 and 7 we have two measurements of power hitting ability. Extra-base hit percentage (XBH%) and Home Run percentage (HR%).

XBH%

Mu (x) + Mu

X

=

Score

0.5

0.124798452

15

0.45

0.120638504

13

0.3

0.108158658

10

0.15

0.095678813

7

0.05

0.087358916

3

Mu

0.0832

0

-0.05

0.07903902

-3

-0.15

0.070719123

-7

-0.25

0.062399226

-10

-0.35

0.054079329

-15

-0.45

0.045759432

-20

-0.5

0.041599484

-25

HR%

X

Mu (x) =

Score

3.5

0.06884643

15

2.75

0.05409362

13

2

0.03934082

10

1.5

0.02950561

7

1.25

0.02458801

3

1

0.01967041

0

0.9

0.01770337

-3

0.75

0.01475281

-7

0.6

0.01180224

-10

0.5

0.0098352

-15

0

0

-20

On page nine we have Strike-out percentage (K%). Thanks to the work of RedSoxFaithful we have a sense that as K-rates in prospects increase, their chance for major leagues success goes down. ( Minorleagueball.com ) This is how we’ll weigh Strike Outs in our model in the higher leagues (Sally, Cal & Eastern).

X

Mu (x) =

Score

0.45

0.10074501

15

0.55

0.12313279

13

0.65

0.14552057

10

0.75

0.16790835

5

0.95

0.21268391

0

1

0.2238778

-3

1.1

0.24626558

-10

1.2

0.26865336

-20

1.35

0.30223503

-30

1.5

0.3358167

-35

On Page ten we have an attempt to measure speed as a tool for success. This is not really attempt to measure how well one can steal bases, it works on the theory that a player who has potentially valuable speed will be asked to steal often and will have some measured success (Stolen Bases), so this is not the percentage of successful stolen bases, this stolen bases divided by Plate Appearance. The negative grading is significantly less in this instance because, while speed can be a valuable tool, its absence is not necessarily equally a detriment.

X

Mu (x) =

Score

4

0.11559657

25

3.5

0.101147

20

2.75

0.07947264

10

1.75

0.0505735

7

1.35

0.03901384

3

1

0.02889914

0

0.9

0.02600923

-3

0.75

0.02167436

-5

0.5

0.01444957

-7

0.25

0.00650231

-10

You’ll notice that in virtually all of these points systems the distribution is weighted heavier on the negative side, or one gets penalized more performing below or at the mean than one gets rewarded for performing above the mean. This is meant to help create a negative linear (or logistical?) model which results in markedly fewer players having a positive rating than players with a negative rating, based on the simple fact that more players will fail to make to the next level than players who move on and eventually become major leaguers. This chart is exemplary of this principle.

(For some reason I am unable to insert my graph here- sorry!)

Sample Size

As mentioned before, sample size weighs heavily in this system. Without the benefit eyes to see (a players movement, physicality and swing) and ears to hear (a coaches or scouts observations or fears) P-SABR is at significant disadvantage. Sample size represents a factor that we can attempt to account for. As an example, here are the top hitters from the Eastern League 2006 –

BBA Top Hitters

WAR

P-SABR

WAR

1

Adam Lind

4.7

Carlos Gomez

1.3

2

Jacoby Ellsbury

DNQ

12

Brandon Moss

-1.2

3

Carlos Gomez

1.3

Kevin Kouzmanoff

5.1

4

Trevor Crowe

DNQ

-1

Adam Lind*

4.7

5

Kevin Kouzmanoff

5.1

Gary Burnham*

0

6

Kory Casto

-1

Nate Schierholtz

1.7

7

Alexi Casilla

DNQ

2.6

Jeff Fiorentino

0.5

8

Kory Casto

-1.4

9

Luis Antonio Jimenez*

0

10

Chad Spann

0

In this example, only 7 hitters this year made BBA rankings for the top 20 players of the league and 3 of those players (Elsbury, Crowe and Casila) did not qualify for the P-SABR system due to an insufficient amount of Plate Appearances. There are two ways we can deal with this, one, remove the players who would not qualify for P-SABR and compare only those top players who qualify (4 players), or two, expand the P-SABR sample to account for this sample size factor. The later was chosen because it is potentially a better test of P-SABR’s value, where potentially P-Sabr can find a greater proportion of MLB players (n=10) or lesser proportion, as in this case, where BBA had all (7) of their selections make it to the major leagues and P-SABR had only 70%. It also expands the population in determining mean value (WAR), where in this case BBA selections have a mean value of 3.36 (WAR) and P-SABR a 1.07 (WAR) thus far in their respective careers.

Conclusion

After testing the data it’s evident that P-SABR has some real value in finding and projecting the value of minor league players. Two sample Hypothesis tests for proportion and mean were run on each league at a 95% confidence intervals, then run on all the data combined at a 98% confidence interval, where the final P values were .0004 for proportion (rejected proportion) and .0217 (failed to reject mean), suggesting that while BBA is better at finding players who will become MLB players, P-SABR finds enough players who have value as future MLB players to suggest that the system has potential.

The null hypothesis of proportional equality was rejected in 2 out of 5 leagues, the California League and the Northwest league as well as in the overall conclusion. Clearly, BBA has an advantage in selecting players who are most likely to become Major league ball players, finding 226 future major leaguers out of a sample of 351, while P-SABR found only 213 out of 412.

In the testing of mean, the null hypothesis of equality was rejected once in the Eastern league, but failed to reject overall. This suggests that P-SABR has some value in identifying players who may have been overlooked by BBA and their system. The Mean WAR for all players selected by BBA was 2.75, while it was 1.8 for P-SABR, suggesting that there’s also room for improvement in the system.

It is also clear that there are areas that P-SABR could be improved, most notably a positional ranking, as the players that were most likely overlooked by P-SABR had positional values (i.e. catcher or middle infielder) that should have raised their overall value when compared to their peers. The most illustrative example of this would be Brian McCann (Sally league 2003), who ranked 11th in P-SABR’s system and 8th in BBA’s, but has gone on to be a 21.7 WAR player thus far in his career.

Since P-SABR is limited to analyzing a players potential value as a hitter, the expansion to a more rounded approach is recommendable. One way to attempt to account for a players future defensive value is to include positional rankings, by providing a scale of value for a players defensive position, such as 20 points for a catcher, 10 points for a Shortstop and Center fielder, 5 points for a second baseman, 0 for a Third baseman, -5 Left and Right Fielders and –10 for First baseman.

References

1-James, Bill, The New Bill James Historical Baseball Abstract, 2001, The Free Press. Page 642


2(http://www.minorleagueball.com/2011/4/22/2123847/the-significance-of-minor-league-k-rates. By Redsoxfaithful


3- http://www.baseballprospectus.com/article.php?articleid=15295#commentMessage

by Rany Jazeyleri.


4- Baseball America, The Baseball America Prospect Handbook 2004, 2004, Simon & Schuster


5- Baseball America, The Baseball America Prospect Handbook 2005, 2005, Simon & Schuster


6- Baseball America, The Baseball America Prospect Handbook 2006, 2006, Simon & Schuster


7- Baseball America, The Baseball America Prospect Handbook 2007, 2007, Simon & Schuster


8- Baseball America, The Baseball America Prospect Handbook 2008, 2008, Simon & Schuster


9- Baseball America, The Baseball America Prospect Handbook 2009, 2009, Simon & Schuster

Hypothesis test of 2 means

Mu 1

Mu 2

P-SABR WAR

BBA WAR

Mu

1.79782082

2.75156695

StDev

5.0269922

6.11363269

Claim µ1 = µ2

Test Statistic, t: -2.3287

Critical t: ±2.33186

P-Value: 0.0202

Degrees of freedom: 677.7948

98% Confidence interval:

-1.90931 < µ1-µ2 < 0.0013097

Fail to Reject the Null Hypothesis

<

Sample does not provide enough evidence to reject the claim


This FanPost is reader-generated, and it does not necessarily reflect the views of McCovey Chronicles. If the author uses filler to achieve the minimum word requirement, a moderator may edit the FanPost for his or her own amusement.

X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join McCovey Chronicles

You must be a member of McCovey Chronicles to participate.

We have our own Community Guidelines at McCovey Chronicles. You should read them.

Join McCovey Chronicles

You must be a member of McCovey Chronicles to participate.

We have our own Community Guidelines at McCovey Chronicles. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9351_tracker