Brady hurts his leg and Matt Cassel goes 11-5, He is suspended for 4 games they go 3-1. That loss only came after the had dropped to the third string QB. Patriots win whether or not Brady is playing.
In fact Cassel had the same completion percentage and YPA as Brady's career averages for that season. Nowhere near it after he left the team.