More Findings of Stats Analysis using v1.21

Matrix Games Forums

Forums Register Login Photo Gallery Member List Search Calendars FAQ

My Profile Inbox Address Book My Subscription My Forums Log Out

More Findings of Stats Analysis using v1.21

View related threads: (in this forum | in all forums)

Logged in as: Guest

Users viewing this topic: none

Printable Version

All Forums >> [Current Games From Matrix.] >> [Sports] >> PureSim Baseball >> More Findings of Stats Analysis using v1.21

Page: [1]

Message

<< Older Topic Newer Topic >>

More Findings of Stats Analysis using v1.21 - 4/18/2006 2:11:40 AM

BryanK

Posts: 140
Joined: 8/21/2005
Status: offline

I went back and validated the HR and BB findings from yesterday with a second associaiton and dove into the pitcher usage stuff I mentioned yesterday.

Findings:
1. Based on what I saw in a second test association I ran last night, the best fix for the HR and BB level issue might be more involved than just scaling levels in the XML. It appears that for the first few years of an association, regardless of the year it starts begins, Puresim produces historically accurate HR numbers. Then it quickly regresses to an unrealistic level that is dead on with my earlier test. I observed similar results for the BB levels.

2. Pitcher utilization is perhaps the weakest part of Puresim's stat engine.
A: From an IP perspective, there isn't enough distinction between durable starters and average ones
B: All starters log pretty much the same number of starts
C: Relievers who make any appearnaces make too many apperances

The suggestions that come to mind for item 2 are generally related to scaling down pitcher durability or adjusting the distribution thereof. I go into more detail in the slides below. A quick thought on 2-B, though, would be to add a slot for spot starters who could take a pitcher's turn in the rotation if said pitcher is too tired to go. That might be a bit tricky to impliment properly, though.

Granted, the game is using the pitchers it does as often as it does because they're better than the other arms available. If durability gets scaled down, the game will probably compensate by using bad pitchers more often, which could possibly lead to higher offensive stats. So... this might be a bit of a hornets nest. But based on the data, though, I think it's clear that pitcher usage is the area that offers the most room for improvement.

For the new slides click HERE
For the updated summary sheets, click HERE

Also, does anyone know where to buy a second hand license for JMP v5 software... if in fact those licenses are transferable? I only have it on my work computer, and so I'll lose access to it when I leave the company at the end of the month. Without that package, it'll be nigh impossible to break out the distributions that made this analysis so cool, and the sticker price for the program is like $1600.

< Message edited by BryanK -- 4/18/2006 8:37:41 AM >

Post #: 1

RE: More Findings of Stats Analysis using v1.21 - 4/18/2006 2:44:51 AM

Amaroq

Posts: 1100
Joined: 8/3/2005
From: San Diego, California
Status: offline

quote:

C: Relievers make too many apperances

Speaking anecdotally, rather than statistically, I think this amounts to "R1 makes too many appearances". Its not uncommon to use an 11-man staff, 5 pitcher rotation, 5 relievers, and 1 closer. In that instance, I've seen R1 come through with 80+ appearances, and R4 finish the season with 11 innings pitched. This seems similar for the AI, and I've seen AI pitchers make 110 appearances on a 162-game season.

In this instance, I think the modern "R1" is used as a setup man, often pitching the 8th with the closer pitching the 9th, or pitching the 9th in a tie game or when losing by one. Instead, we get R1 used whenever the starter gets knocked out of the game.

I'd like to see things spread between the various relievers.

(in reply to BryanK)

Post #: 2

RE: More Findings of Stats Analysis using v1.21 - 4/18/2006 3:31:11 AM

BryanK

Posts: 140
Joined: 8/21/2005
Status: offline

It means that all relievers who make at least one appearance wind up making too many of them. Since the duration of those appearances is in line with real-life levels, that means that either the starters are logging too few innings thereby creating a greater need for relief or the number of total relief appearances is being divided among too few pitchers. Given that toal IPs are too high at the high end, I'm inclined to believe that it's the latter rather than the former.
To be more explicit, what I think it happening is that the relievers are recovering too quickly, hence teams can effectively get by using only 5 relievers on a regular basis instead of 6. This is generally consistent with your observation that R1 gets too many outings and innings while R4 is never really used.

< Message edited by BryanK -- 4/18/2006 7:10:25 AM >

(in reply to Amaroq)

Post #: 3

RE: More Findings of Stats Analysis using v1.21 - 4/18/2006 9:06:44 PM

BleacherBum

Posts: 56
Joined: 8/21/2005
Status: offline

quote:

ORIGINAL: BryanK
1. Based on what I saw in a second test association I ran last night, the best fix for the HR and BB level issue might be more involved than just scaling levels in the XML. It appears that for the first few years of an association, regardless of the year it starts begins, Puresim produces historically accurate HR numbers. Then it quickly regresses to an unrealistic level that is dead on with my earlier test. I observed similar results for the BB levels.

One factor to consider in your studies is that when you get a few years into an association, the GM AI is going to have an impact on your league. The first year of any fictional assoc you get a nicely balanced set of players based on the player generator. Years later, your league is influenced by player developmental decisions. I recall that the CPU GM's are much worse at developing pitchers than they are hitters. In my career associations, after a few seasons my team's pitching staffs would rise to the top - all 5 starters would rate in the highest percentiles of the league. I investigated, and discovered many instances where the CPU GM's were making poor decisions with pitchers, such as promoting 18 or 19 year olds directly to the big leagues as starters, instead of developing them for a few years in the minors. The net effect was that the overall pitching quality of the league would deterioriate over time, leading to inflated offensive stats, especially HRs and walks. This might explain your observation that HRs and BBs drift upward over years.

(in reply to BryanK)

Post #: 4

RE: More Findings of Stats Analysis using v1.21 - 4/18/2006 10:06:21 PM

Amaroq

Posts: 1100
Joined: 8/3/2005
From: San Diego, California
Status: offline

BleacherBum, have I ever mentioned that you're brilliant!?

I'd noticed that my pitching staff always trended towards being the best in the league, but hadn't realized it was related to the AI's player development routines - I just though I was valuing pitching more than the average AI manager.

(in reply to BleacherBum)

Post #: 5

RE: More Findings of Stats Analysis using v1.21 - 4/18/2006 10:45:15 PM

PadresFan104

Posts: 1223
Joined: 8/8/2005
Status: offline

Amaroq - Maybe you should re-post this in the beta forum for Shaun? I know Shaun is probably reading this thread, but the point about Pitcher development is a great one and it would be nice to have the issue tracked.

(in reply to Amaroq)

Post #: 6

RE: More Findings of Stats Analysis using v1.21 - 4/18/2006 11:06:36 PM

Amaroq

Posts: 1100
Joined: 8/3/2005
From: San Diego, California
Status: offline

Can do.

(in reply to PadresFan104)

Post #: 7

RE: More Findings of Stats Analysis using v1.21 - 4/19/2006 12:11:49 AM

KG Erwin

Posts: 8981
Joined: 7/25/2000
From: Cross Lanes WV USA
Status: offline

quote:

ORIGINAL: BleacherBum

quote:

ORIGINAL: BryanK
1. Based on what I saw in a second test association I ran last night, the best fix for the HR and BB level issue might be more involved than just scaling levels in the XML. It appears that for the first few years of an association, regardless of the year it starts begins, Puresim produces historically accurate HR numbers. Then it quickly regresses to an unrealistic level that is dead on with my earlier test. I observed similar results for the BB levels.

One factor to consider in your studies is that when you get a few years into an association, the GM AI is going to have an impact on your league. The first year of any fictional assoc you get a nicely balanced set of players based on the player generator. Years later, your league is influenced by player developmental decisions. I recall that the CPU GM's are much worse at developing pitchers than they are hitters. In my career associations, after a few seasons my team's pitching staffs would rise to the top - all 5 starters would rate in the highest percentiles of the league. I investigated, and discovered many instances where the CPU GM's were making poor decisions with pitchers, such as promoting 18 or 19 year olds directly to the big leagues as starters, instead of developing them for a few years in the minors. The net effect was that the overall pitching quality of the league would deterioriate over time, leading to inflated offensive stats, especially HRs and walks. This might explain your observation that HRs and BBs drift upward over years.

The above observations do make sense. Tweaking the GM AI to improve pitcher management would add more challenge to those of us who desire to play career leagues. I'm much more interested in improved game-play than ear- or eye-candy, and I'm sure the vast majority of players would agree.

_____________________________

(in reply to BleacherBum)

Post #: 8

RE: More Findings of Stats Analysis using v1.21 - 4/19/2006 12:52:33 AM

BryanK

Posts: 140
Joined: 8/21/2005
Status: offline

Excellent points, BleacherBum. I hadn't thought of that, and it certainly does make sense.

To be clear, though, my analysis found that walk totals actually drift down over time. That is, the pitchers get more control. That said, it's a little counterintuitive since I made the point with a graph showing AB/W, and thier increase over time. All that means is that pitchers face more batters before giving up a walk.

Still, the player development point is an excellent one, so thank you for pointing it out. Do we know if and to what degree the game penalizes player development if it is not handled properly?

(in reply to BryanK)

Post #: 9