ADVERTISEMENT

Podcast on predictive CFB model

P

PSU2UNC

Well-Known Member
Feb 9, 2016
7,250
7,546
1
Not sure if this has been discussed before, but there is a podcast called "SEC Fans" who have a predictive model that they use to analyze not only SEC games but also other big games. They have a pod on the OSU-PSU game that you can find below.

http://secfans.com/

What made me laugh was that when the model predicted PSU would win by multiple scores they went out of their way to find issues with the model input data to create a win for OSU. They did this by saying that you have to throw out the App St game data because App St only played 3 games (one was canceled by hurricane) and one of those was FCS (which they don't count in the model).

While statistically, that may be true, App St is by far the best team PSU has played. So by removing the best team PSU has played (which by the way, statistically makes PSU look worse) they were able to fudge the numbers so that OSU wins a shoot out.

I honestly don't understand anyone who thinks PSU doesn't have a very, very good chance at wining this game.
 
  • Like
Reactions: JJTopp99
We'll find out who wins on Saturday. I certainly hope it's Penn State. Playoff predictions are biased as hell and people will use every argument possible to support who they think should be there. Consider that the vast majority of preseason predictions included Alabama, Clemson and Ohio State. The fourth team varies. They people who made those prediction can't admit they aren't very knowledgeable so they'll go to the mat to defend their choices. One loss? Fluke. Two losses? They still deserve to be the first two loss team in the playoff.

As for the mentioned predictive model, if it picks PSU to win but those people choose to ignore it, they're essentially saying the model is worthless. If it's worthless, why talk about it at all?
 
  • Like
Reactions: Lyons212 and JJTopp99
When their numbers spit out something they didn’t agreed with, they just pulled out information until they got the result they wanted.

And then pretended they are all about the data.
 
Penn State is winning Saturday, but they need to maintain their focus throughout the season. A single slip up and they are probably the fifth team in.

Everybody thank Jimmy Delaney for belittling the value of a Big10 championship.
 
  • Like
Reactions: JJTopp99
PSU2UNC said:
Not sure if this has been discussed before, but there is a podcast called "SEC Fans" who have a predictive model that they use to analyze not only SEC games but also other big games. They have a pod on the OSU-PSU game that you can find below.

http://secfans.com/

What made me laugh was that when the model predicted PSU would win by multiple scores they went out of their way to find issues with the model input data to create a win for OSU. They did this by saying that you have to throw out the App St game data because App St only played 3 games (one was canceled by hurricane) and one of those was FCS (which they don't count in the model).

While statistically, that may be true, App St is by far the best team PSU has played. So by removing the best team PSU has played (which by the way, statistically makes PSU look worse) they were able to fudge the numbers so that OSU wins a shoot out.

I honestly don't understand anyone who thinks PSU doesn't have a very, very good chance at wining this game.
Click to expand...
App St went for 7.5 yd/pass vs PSU which is about average for FBS. They went for 21 yards per pass in their only other data point vs Charlotte - they only threw 14 passes. I know it sounds fishy to remove App, but their game vs Charlotte makes for really strange data. As they said, they like to have 4 data points to run their model and App only has 2.

App St played well, but don't fool yourself into thinking that they are a great team. They are probably somewhere between Louisville and Buffalo.
 
  • Like
Reactions: JJTopp99
Ian said:
How accurate has it been with previous predictions?
Click to expand...

That's really all that matters. What is it's verified prediction power?

If I wanted to produce a computer model that predicted a 43-20 PSU victory on Saturday night --- I can do that. I can produce a computer model that can predict ANY output. Doesn't mean it's a good model, of course.
 
  • Like
Reactions: JJTopp99
GreggK said:
When their numbers spit out something they didn’t agreed with, they just pulled out information until they got the result they wanted.

And then pretended they are all about the data.
Click to expand...

On one hand, in science, you have to control for anomalies or your results will be compromised.
On the other hand, remember this the next time you hear about some new study that seems to be biased in nature, and especially the social sciences where it's nearly impossible to repeat the results. Stats can be made to say anything you want, hence the phrase, "Lies, damn lies, and statistics."
 
  • Like
Reactions: JJTopp99
michnittlion said:
That's really all that matters. What is it's verified prediction power?

If I wanted to produce a computer model that predicted a 43-20 PSU victory on Saturday night --- I can do that. I can produce a computer model that can predict ANY output. Doesn't mean it's a good model, of course.
Click to expand...
The model appear to be a fairly good predictor (although it's not clear to me how frequently they make qualitative adjustments like discussed above).
 
Grant Green said:
App St went for 7.5 yd/pass vs PSU which is about average for FBS. They went for 21 yards per pass in their only other data point vs Charlotte - they only threw 14 passes. I know it sounds fishy to remove App, but their game vs Charlotte makes for really strange data. As they said, they like to have 4 data points to run their model and App only has 2.

App St played well, but don't fool yourself into thinking that they are a great team. They are probably somewhere between Louisville and Buffalo.
Click to expand...
Two things:

1) I work with data every day and I almost never discard outliers. The only outliers that get discarded is when I think the data is inaccurate. If the data is "weird" or "strange" it has to stay in the analysis because that is often where the most interesting stories lie.

2) App St is a top 25 team. They will probably win their division and will almost certainly go to (win?) a bowl game.
 
  • Like
Reactions: JJTopp99
GreggK said:
When their numbers spit out something they didn’t agree with, they just pulled out information until they got the result they wanted.

And then pretended they are all about the data.
Click to expand...


Sort of what they do with climate change models, lol.
 
  • Like
Reactions: heckmans
PSU2UNC said:
Two things:

1) I work with data every day and I almost never discard outliers. The only outliers that get discarded is when I think the data is inaccurate. If the data is "weird" or "strange" it has to stay in the analysis because that is often where the most interesting stories lie.

2) App St is a top 25 team. They will probably win their division and will almost certainly go to (win?) a bowl game.
Click to expand...



I want to see Pitt play APP State in a bowl game this year given that half the Lair Board is of the opinion that APP State is "overrated". "Overrated" is probably the most frequently used descriptive word on the Lair by the way because every team over there is overrated (except Pitt of course).
 
PSU2UNC said:
I honestly don't understand anyone who thinks PSU doesn't have a very, very good chance at wining this game.
Click to expand...

Not just a good chance. Not just a very good chance, but a "very, very good chance." Ah, then you don't understand me. That's OK. I would say we have a chance to win, but I expect OSU to win by 10 or more. I obviously hope I'm wrong, but I just believe they have the better team and the Whiteout will not be enough to overcome that. If the game were played in November, I would give us a slight edge, but we have inexperience on defense that I expect will be our downfall.
 
fairgambit said:
Not just a good chance. Not just a very good chance, but a "very, very good chance." Ah, then you don't understand me. That's OK. I would say we have a chance to win, but I expect OSU to win by 10 or more. I obviously hope I'm wrong, but I just believe they have the better team and the Whiteout will not be enough to overcome that. If the game were played in November, I would give us a slight edge, but we have inexperience on defense that I expect will be our downfall.
Click to expand...
I strenuously object. ;-)
 
  • Like
Reactions: fairgambit
fairgambit said:
Not just a good chance. Not just a very good chance, but a "very, very good chance." Ah, then you don't understand me. That's OK. I would say we have a chance to win, but I expect OSU to win by 10 or more. I obviously hope I'm wrong, but I just believe they have the better team and the Whiteout will not be enough to overcome that. If the game were played in November, I would give us a slight edge, but we have inexperience on defense that I expect will be our downfall.
Click to expand...

They have had a better team than us on paper each of the last three games. And we could have/should have won all three.
 
GreggK said:
They have had a better team than us on paper each of the last three games. And we could have/should have won all three.
Click to expand...
Last year was a 1 point OSU win. Yes, we could have/should have won that one. The previous year was a 3 point Penn State win and they will argue they could have/should have won that one. Three years ago it was a 28 point OSU win. No way we could have/should have won that one.
 
GreggK said:
They have had a better team than us on paper each of the last three games. And we could have/should have won all three.
Click to expand...
I'll give you the benefit of the doubt that you're forgetting how lopsided 2015 was, but if not, you kind of lose all credibility on this topic if you think we should have won that game.
 
JJTopp99 said:
I want to see Pitt play APP State in a bowl game this year given that half the Lair Board is of the opinion that APP State is "overrated". "Overrated" is probably the most frequently used descriptive word on the Lair by the way because every team over there is overrated (except Pitt of course).
Click to expand...

If Pitt were bowl eligible and that is a BIG if.
 
PSU2UNC said:
Two things:

1) I work with data every day and I almost never discard outliers. The only outliers that get discarded is when I think the data is inaccurate. If the data is "weird" or "strange" it has to stay in the analysis because that is often where the most interesting stories lie.

2) App St is a top 25 team. They will probably win their division and will almost certainly go to (win?) a bowl game.
Click to expand...
You probably wouldn't make a professional conclusion based on 2 real data points either. I don't think you can compare early season college football data to whatever data you are crunching. There is a lot of randomness and not a lot of data to iron out those wrinkles. Your data is likely more consistent, reliable, and plentiful.

I think you are seeing App St through the blue and white glasses. I've fallen into that trap before too, where I overrate a team based on their performance vs PSU. App St really only had 2 good quarters against us. For App St to be top 25, they would have to be comparable to a team like Boise St, Missouri, BC (I'm not looking at AP top 25 - that poll sucks). They likely aren't even a true top 50 team.
 
Not necessarily defending their picks (though I do agree with them):

But they pulled App State out because of how their system is setup. They average all the FBS level games a team plays, and then match that against the average % of their usual output a teams defense holds their foes too. For example if App State averages 100 yards passing a game, and Penn State holds their opponents to 90% of their average passing output, then their formula calculates that App State will put up 90 yards against PSU.

Unfortunately App State has only played one FBS level team to this point, Penn State. That means Penn State has held them to their exact averages, and it messes up the whole formula.

Honestly their biggest mistake was trying to use the formula at all in this situation. They even mentioned that both of these teams have been plugging in their back ups early, which skews the results dramatically. But what else can you expect; they're SEC fans.
 
Westy81585 said:
Unfortunately App State has only played one FBS level team to this point, Penn State. That means Penn State has held them to their exact averages, and it messes up the whole formula.

Honestly their biggest mistake was trying to use the formula at all in this situation. They even mentioned that both of these teams have been plugging in their back ups early, which skews the results dramatically. But what else can you expect; they're SEC fans.
Click to expand...
See my post above. App played Charlotte as well and averaged 21 yards per pass, which is crazy. The problem is that PSU actually held App below their average passing because their average is skewed way high. I think they said at one point that if you assumed PSU held App to their average, then the model would have been more reliable.

I think you are right on the 2nd part though. They probably should have just said that the model isn't applicable in this situation because App screws it up. Their non-model handicapping is pretty good.
 
  • Like
Reactions: Westy81585
Grant Green said:
You probably wouldn't make a professional conclusion based on 2 real data points either. I don't think you can compare early season college football data to whatever data you are crunching. There is a lot of randomness and not a lot of data to iron out those wrinkles. Your data is likely more consistent, reliable, and plentiful.

I think you are seeing App St through the blue and white glasses. I've fallen into that trap before too, where I overrate a team based on their performance vs PSU. App St really only had 2 good quarters against us. For App St to be top 25, they would have to be comparable to a team like Boise St, Missouri, BC (I'm not looking at AP top 25 - that poll sucks). They likely aren't even a true top 50 team.
Click to expand...

If I had a model that I believed worked, I wouldn't throw out data points that I "didn't like." Alternatively, the modelers could say "look our model doesn't really work until week 6" (which would be totally acceptable and probably true).

You are entitled to your opinion of App St. In their other two games they have won (against poor competition) by scores of 45-9 and 72-7. I'd compare them favorably with Missouri, BSU and BC.
 
PSU2UNC said:
If I had a model that I believed worked, I wouldn't throw out data points that I "didn't like." Alternatively, the modelers could say "look our model doesn't really work until week 6" (which would be totally acceptable and probably true).

You are entitled to your opinion of App St. In their other two games they have won (against poor competition) by scores of 45-9 and 72-7. I'd compare them favorably with Missouri, BSU and BC.
Click to expand...
They do say in multiple videos that the model isn't effective until at least week 4. In this case, it should probably be more like week 5 or 6 due to Apps lack of quality FBS opponents. I will agree that they should probably have just ditched the model in this case, but I don't think throwing out App st is a terrible alternative. Again, CFB early season data can't be compared to other studies with robust data sets where single outliers aren't going to severely skew results.

It is not just my opinion on App St. I guarantee that App would be a 7 to 14 point underdog to Boise, Missou, and BC on a neutral field. They will probably win their conference and get some AP votes, but that means nothing to me in terms of their actual power ranking.
 
Grant Green said:
They do say in multiple videos that the model isn't effective until at least week 4. In this case, it should probably be more like week 5 or 6 due to Apps lack of quality FBS opponents. I will agree that they should probably have just ditched the model in this case, but I don't think throwing out App st is a terrible alternative. Again, CFB early season data can't be compared to other studies with robust data sets where single outliers aren't going to severely skew results.

It is not just my opinion on App St. I guarantee that App would be a 7 to 14 point underdog to Boise, Missou, and BC on a neutral field. They will probably win their conference and get some AP votes, but that means nothing to me in terms of their actual power ranking.
Click to expand...
You are entitled to your opinion. Circle back with me at the end of the year to see if you have revised your opinion of App St.
 
You must log in or register to reply here.

Similar threads

NewEra 2014

Musings on the CFB playoff

Replies
40
Views
2K
Penn State Football
olelion
olelion
R

ESPN Podcast Preview PSU vs OSU

Replies
7
Views
1K
Penn State Football
NewEra 2014
NewEra 2014
L

The ACC still doesnt get it

Replies
41
Views
2K
Penn State Football
LandoComando
L
T

Thoughts after rewatching Minny

Replies
178
Views
8K
Penn State Football
PSU63-OSU14
P
bdgan

Playoff Picture

Replies
36
Views
2K
Penn State Football
rcy4499
rcy4499
ADVERTISEMENT

Latest posts

ADVERTISEMENT
Top Bottom
Back