For those that use PFF rankings

[jarntt]

I've always said they have no clue who is asked to do what on a particular play and thus can't really judge players accurately, so it is as subjective as anything else. For example they may be blaming a CB who gets beat on a play that wasn't even his responsibility.

Kyle Williams doubts outsiders can know if he’s playing well


Kyle Williams doubts outsiders can know if he’s playing well
Posted by Michael David Smith on November 30, 2016, 9:28 AM EST

Getty Images
Bills defensive tackle Kyle Williams had an interesting response to being told he was ranked as one of the best players in the NFL: He doubted how anyone would know.

Williams was told by Tim Graham of the Buffalo News that in 2010, Pro Football Focusranked him as the third-best player in the NFL. Williams responded that he doesn’t buy into such rankings.

“I don’t even know what that means, to be honest with you. How do they know how to grade me if they don’t even known what I’m supposed to be doing?” Williams asked.

In Williams’ view, even if you watch him on every single play of an entire season, you still don’t know how well he was playing unless you also know what the Bills’ defensive play call was. Williams points out that what constitutes a good play in one defense could be a bad play in another defense based on what the defensive tackle is asked to do, and those outside the Bills facility won’t know what the defensive tackle was asked to do on any given play.

“If you don’t know what my assignment is or what the O-line’s trying to do, I mean . . . Good Lord, especially in this defense,” Williams said. “I don’t know how anybody would know how to begin to grade this defense if they didn’t know exactly what our job is on a certain play. We have some interesting jobs in this defense that I’ve never been asked to do in other ones. So to grade that on some kind of scale? I don’t know how they could do that.”

There’s certainly some value in tape scouting even if you don’t know a player’s assignment. After all, that’s what every NFL team does when scouting players on opposing teams. But Williams raises a fair point about the limitations of watching a team from the outside.

 

[Clayton]

I feel thats pretty obvious but sometimes it is a little too easy to get caught up in advanced metrics. I don't, however, think its just as subjective as everything else. I do think there are some coaches who overvalue and undervalue players which is why not all teams are 8-8 every year.

 

[Fountain City Blues]

Never been a fan of PFF grades, but I do keep tabs on their top grades for each game just as a sounding board of sorts.

 

[Clayton]

Here is an example of a ridiculous QB stat that actually almost works. Its called ALEX on footballoutsiders website.

"ALEX represents Air Less EXpected on third downs, the average difference between the length of the quarterback's throw and the distance needed for a new set of downs. The number listed here only includes third downs and is not adjusted for passes thrown away or batted down"

That has nothing to do with the QB and everything to do with the system, right? Except the guys at the top of the list? Super Bowl QBs. Big Ben. Rodgers. Eli. Cam. Brady. The only anomalies Tyrod Taylor and Josh McCown.

QBs with low ratings this year? Bradford, Kaepernick, Palmer, Ryan, Keenum, Wentz, Alex Smith, Fitzpatrick, Dalton, Stafford, Rivers, Fitzpatrick, Osweiler. AKA the 'I'm not really afraid of these guys in the playoffs crew'


So, yeah. Its just a meaningless stat but its a fun stat.

 

[eaglesnut]

Any piece of monopolized data has a value heading towards zero. It can mislead you just as easily as lead you to an actual representation of what it is evaluating.

 

[ATL96Steeler]

I feel thats pretty obvious but sometimes it is a little too easy to get caught up in advanced metrics. I don't, however, think its just as subjective as everything else. I do think there are some coaches who overvalue and undervalue players which is why not all teams are 8-8 every year.

It has some merit, particularly when you don't see every team play...I agree it's subjective at best, but from the eye test...when I watch a game and see the player grade for that game most of the time I would agree.

If their site has traffic and paying advertisers...more power to them.

 

[eaglesnut]

Here is an example of a ridiculous QB stat that actually almost works. Its called ALEX on footballoutsiders website.

"ALEX represents Air Less EXpected on third downs, the average difference between the length of the quarterback's throw and the distance needed for a new set of downs. The number listed here only includes third downs and is not adjusted for passes thrown away or batted down"

That has nothing to do with the QB and everything to do with the system, right? Except the guys at the top of the list? Super Bowl QBs. Big Ben. Rodgers. Eli. Cam. Brady. The only anomalies Tyrod Taylor and Josh McCown.

QBs with low ratings this year? Bradford, Kaepernick, Palmer, Ryan, Keenum, Wentz, Alex Smith, Fitzpatrick, Dalton, Stafford, Rivers, Fitzpatrick, Osweiler. AKA the 'I'm not really afraid of these guys in the playoffs crew'


So, yeah. Its just a meaningless stat but its a fun stat.

I've always liked to evaluate the worthiness of any particular stat in the way that you have done above, but it inherently leads to confirmation bias.

Trying to remember one stat I found previously that was directly related to punting that just so happened to basically list the teams respective QBs in a list of who was the best towards the top and who was the worst near the bottom. Sometimes things can be accurately related even if we don't have a sophisticated enough view of the causal chain.

 

[cdumler7]

I was a big PFF guy when they first came out. Used it quite often in conversations. Then listened to an OL player that had retired talk about it and call it pretty much a joke. Now I wouldn't go that far as I do think all stats have to be understood they are not the end all be all but he talked about how the people they have hired to watch these games have never actually worked with an NFL playbook or understand how an offense is to be functioning especially the OL. A guy might run right by an OL and be credited with a negative grade but that player wasn't his responsibility and was supposed to be left unblocked. Things like that which is what Kyle Williams is getting at here. Something to look at but yeah they do have some bias towards certain things and some lack of understanding fully of what is going on.

 

[Sharkonabicycle]

Well duh. No 'stat' is gonna be perfect. Least PFF puts a decent amount of analysis and stats into their grading, vs. the pathetic excuse for ESPN and NFL.COMs pathetic articles and top 100 players, etc. in the past.

 

[Fountain City Blues]

Well duh. No 'stat' is gonna be perfect. Least PFF puts a decent amount of analysis and stats into their grading, vs. the pathetic excuse for ESPN and NFL.COMs pathetic articles and top 100 players, etc. in the past.

+5.5 comment

 

[richig07]

I've always said they have no clue who is asked to do what on a particular play and thus can't really judge players accurately, so it is as subjective as anything else. For example they may be blaming a CB who gets beat on a play that wasn't even his responsibility.

Kyle Williams doubts outsiders can know if he’s playing well


Kyle Williams doubts outsiders can know if he’s playing well
Posted by Michael David Smith on November 30, 2016, 9:28 AM EST

Getty Images
Bills defensive tackle Kyle Williams had an interesting response to being told he was ranked as one of the best players in the NFL: He doubted how anyone would know.

Williams was told by Tim Graham of the Buffalo News that in 2010, Pro Football Focusranked him as the third-best player in the NFL. Williams responded that he doesn’t buy into such rankings.

“I don’t even know what that means, to be honest with you. How do they know how to grade me if they don’t even known what I’m supposed to be doing?” Williams asked.

In Williams’ view, even if you watch him on every single play of an entire season, you still don’t know how well he was playing unless you also know what the Bills’ defensive play call was. Williams points out that what constitutes a good play in one defense could be a bad play in another defense based on what the defensive tackle is asked to do, and those outside the Bills facility won’t know what the defensive tackle was asked to do on any given play.

“If you don’t know what my assignment is or what the O-line’s trying to do, I mean . . . Good Lord, especially in this defense,” Williams said. “I don’t know how anybody would know how to begin to grade this defense if they didn’t know exactly what our job is on a certain play. We have some interesting jobs in this defense that I’ve never been asked to do in other ones. So to grade that on some kind of scale? I don’t know how they could do that.”

There’s certainly some value in tape scouting even if you don’t know a player’s assignment. After all, that’s what every NFL team does when scouting players on opposing teams. But Williams raises a fair point about the limitations of watching a team from the outside.

I don't know... It seems a lot of the time, if you have a knowledge of the game. That it would not be overly difficult to figure out a players assignment upon watching a play once or twice.

 

[cdumler7]

I don't know... It seems a lot of the time, if you have a knowledge of the game. That it would not be overly difficult to figure out a players assignment upon watching a play once or twice.

Not completely true. A great example is a DE in the 3-4. Some teams look more for guys with bigger size to eat up blocks to help the Linebackers go and make plays. But they still ask them every once in a while to shoot a gap just to keep the offense off guard of what they are doing. So you could see a DL player actually make a great play in the back field where they look like they split two OL players but actually that could be completely against their assignment on the play and they just got lucky that they were where the ball carrier was coming. So when they go back and watch film the coaches will say "you got lucky that worked out but don't do it again" where PFF looks at that and says "wow what a play!"

 

[flyerhawk]

I have issues with PFF grading system but Williams point is off the mark a bit.

Of course no external group can know what a player was supposed to do on a given play. But if you look at a lot of plays by the same team you can pretty quickly pick up a trend of what a player tends to do. If a DT is playing 1 gap then you would expect that he would be responsible for that gap and for some QB pressures. If a DT is playing 2 gap then you would expect to see fewer QB pressures but better run defense. If you understand the game well you would likely pick up on the tendencies of the DT and what he is playing based on his alignment pre-snap and other indicators.

So when it comes to PFF grades I find that they do a decent job of grading linemen, running backs and linebackers.

Where they start to have problems is the off camera player by the secondary and wide receivers and, oddly enough, the QB. With all 3 of those positions their grading is almost entirely results driven and ignores the plays where nothing really happens. If a cornerback is not targeted all game that results in essentially a 0 grade which makes little to no sense. A wide receiver may be tasked with stretching the field and only thrown to once or twice in the game which helps the offense but doesn't yield much in the stats department.

QB is the most arbitrarily graded position. A QB is graded less positively for throwing to a wide open receiver but it graded negatively for not throwing to that receiver. A QB who forces a throw into tight coverage may get graded higher than a QB who throws to an open receiver for a bigger gain.

On the whole though I find the PFF grades to be useful to a point but, as with all measurements, they should be taken in context and used with other traditional measurements as well as the eye test.

 

[jarntt]

I don't know... It seems a lot of the time, if you have a knowledge of the game. That it would not be overly difficult to figure out a players assignment upon watching a play once or twice.

Often times yes, but not always. There are times a corner is supposed to have deep help, but the safety isn't even in the picture. So you can watch the play 20 times and you'll never see him to know it was the safeties fault. Or a safety could have another responsibility and happen to see a corner in trouble and his area is clear and he runs over and people watching think he just got there too late. Or in Williams case, was his responsibility to just take up two guys and keep them off the LBs? Was it to penetrate? Did he have responsibility for one gap and the play went right by him to the other gap? Did the WR run the wrong route and result in a int?

Plus, how much time do you think they spend on this? They are going to watch every single NFL player 3 times in every play? That's 22 guys on the field at once x 3. And my other issue has always been how does employee #1 grade OL play versus employee #2? There is just so much going on that IMO it's nothing more than an interesting thing to allude to. If one guy is ranked best in the league and the next is last I'm sure it's pretty fair to say player one is playing better. But if one is #15 and the other is #27...I just don't think it is very telling. We are in a world of stats. Many don't even see a game and will argue about it. Using your eyes is the best way to judge things. Too many people form their opinions from websites and the morning paper...

 

[jarntt]

I have issues with PFF grading system but Williams point is off the mark a bit.

Of course no external group can know what a player was supposed to do on a given play. But if you look at a lot of plays by the same team you can pretty quickly pick up a trend of what a player tends to do. If a DT is playing 1 gap then you would expect that he would be responsible for that gap and for some QB pressures. If a DT is playing 2 gap then you would expect to see fewer QB pressures but better run defense.

I think you are missing his point. Let's say a team's DTs play two tech almost exclusively. Then all of a sudden on a particular play they aren't. So now you have assumed that his responsibility for that play was the same and it isn't. And I'm not even talking about pressure. I'm talking more about running plays. So on that play he only has to worry about one particular gap and he shoots through it because a LB is supposed to fill the other, but the LB screws up and doesn't. So know without knowing this the DT would be faulted for just doing his job.

 

[flyerhawk]

I think you are missing his point. Let's say a team's DTs play two tech almost exclusively. Then all of a sudden on a particular play they aren't. So now you have assumed that his responsibility for that play was the same and it isn't. And I'm not even talking about pressure. I'm talking more about running plays. So on that play he only has to worry about one particular gap and he shoots through it because a LB is supposed to fill the other, but the LB screws up and doesn't. So know without knowing this the DT would be faulted for just doing his job.

Sure on that one play the grader may miss that they shifted responsibilities.

But if the grader knows their stuff they should spot the change in assignments. Of course they will miss it sometimes. But that is the exception not the rule.

I remember Colin Kaepernick using this defense a lot when his play started to deteriorate. People would question his ability to read the field and he would say that people didn't know what his reads were so how could they know whether he was doing a good job.

If Williams claim was accurate than no other team would be able to properly gauge the quality of a player when it comes to free agency.

 

[jarntt]

If Williams claim was accurate than no other team would be able to properly gauge the quality of a player when it comes to free agency.

That is a little extreme, because the team would watch a lot more closely than a website that grades every single player on every single play and would their own NFL experts looking at him for how he would fit into their scheme. They look at more than just the outcome of the play. But, to some extent, yes. How many times is a player great for one defense or team and not great in another situation? When NFL scouts look at College players, they look at them for how they would fit into their scheme. They don't just look at it like a fan. If a player is playing on the same Line as JJ Watt or Von Miller he may get less attention and thus may end up with better stats and looking better on a play by play basis, but when a team looks at him as a FA they would take that into account. Also, they aren't using a 5 points grading system. So for PFF a guy that gets a +2 instead of a +1 is barely getting more points. When a scout looks at a guy the one play could be off the charts and taken into account with an infinite rating system.

And missing a play or two is exactly the point. Instead of a -2, maybe a guy should have gotten a +1 or +2. If that happens a few times in a game it is a big deal in a ranking list. Again, if you like using other peoples ratings, go ahead.

 

[flyerhawk]

That is a little extreme, because the team would watch a lot more closely than a website that grades every single player on every single play and would their own NFL experts looking at him for how he would fit into their scheme. They look at more than just the outcome of the play. But, to some extent, yes. How many times is a player great for one defense or team and not great in another situation? When NFL scouts look at College players, they look at them for how they would fit into their scheme. They don't just look at it like a fan. If a player is playing on the same Line as JJ Watt or Von Miller he may get less attention and thus may end up with better stats and looking better on a play by play basis, but when a team looks at him as a FA they would take that into account. Also, they aren't using a 5 points grading system. So for PFF a guy that gets a +2 instead of a +1 is barely getting more points. When a scout looks at a guy the one play could be off the charts and taken into account with an infinite rating system. Again, if you like using other peoples ratings, go ahead.

Whether football teams analyze film more intently than some website isn't really relevant. Williams argument was that no one outside of the Bills organization could properly grade his play.

PFF's grading system may be terrible but that doesn't make Williams point completely accurate.

Yes you can grade a player. It may not be perfect but if you have knowledgeable people grading the players they will, at the very least, be able to give a decent baseline.

 

[cdumler7]

I have issues with PFF grading system but Williams point is off the mark a bit.

Of course no external group can know what a player was supposed to do on a given play. But if you look at a lot of plays by the same team you can pretty quickly pick up a trend of what a player tends to do. If a DT is playing 1 gap then you would expect that he would be responsible for that gap and for some QB pressures. If a DT is playing 2 gap then you would expect to see fewer QB pressures but better run defense. If you understand the game well you would likely pick up on the tendencies of the DT and what he is playing based on his alignment pre-snap and other indicators.

So when it comes to PFF grades I find that they do a decent job of grading linemen, running backs and linebackers.

Where they start to have problems is the off camera player by the secondary and wide receivers and, oddly enough, the QB. With all 3 of those positions their grading is almost entirely results driven and ignores the plays where nothing really happens. If a cornerback is not targeted all game that results in essentially a 0 grade which makes little to no sense. A wide receiver may be tasked with stretching the field and only thrown to once or twice in the game which helps the offense but doesn't yield much in the stats department.

QB is the most arbitrarily graded position. A QB is graded less positively for throwing to a wide open receiver but it graded negatively for not throwing to that receiver. A QB who forces a throw into tight coverage may get graded higher than a QB who throws to an open receiver for a bigger gain.

On the whole though I find the PFF grades to be useful to a point but, as with all measurements, they should be taken in context and used with other traditional measurements as well as the eye test.

I'm going to have to disagree a bit here. They clearly value a 1-gap style DT more so than a 2-gap style DT. It is much harder to really tell on a 2-gap DT how much of a true impact they are having because they register so few stats.

Just looking at their list of mid-season All-Pro players by their count the 4 DT's they have listed are all attacking style type guys. Heck even the NT they included it was more about the solo stops they made instead of how they have helped set up other players to go and make plays. I understand the attacking style guys look like the bigger impact players but just depending on the system I would argue if you use a 2-gap DT right and they are high quality they can be just as impactful. A great example is the 2000 Ravens with Sam Adams and Tony Siragusa. Neither were known as great pressure guys but the work they did in the trenches was phenomenal. If I remember right Ray Lewis did an interview where he said that season he only had one time where an OL player actually made it to him to block because those big guys up front just ate up so much space and made it to where he could flow to the ball unhindered.

 

[Broncos6482]

I don't know... It seems a lot of the time, if you have a knowledge of the game. That it would not be overly difficult to figure out a players assignment upon watching a play once or twice.

You're exactly right. Football isn't rocket science. Are you going to know exactly what every single player is supposed to do, on every single snap? Of course not. Can you watch enough film to get a general idea of what a player is supposed to be doing the majority of the time? Absolutely. If 4 offensive lineman block left, and one blocks right allowing a free rusher, it's a pretty safe bet that the guy blocking right got it wrong. You can generally tell what a player is supposed to do based on what everyone else on the team is doing on a given play.

No, it's not an exact science. Yes, they'll get some rankings wrong. It's not infallible. But overall, I think they do a very good job of giving you a general idea of how good a player is.