That tends to happen when a Brock Ossweiler led team is your biggest competition
Can't think of too many times that Brock Osweiler and Tom Brady were on the field at the same time.
Having said that, I'm pretty sure that Ben Roethlisberger and Matt Ryan were his two "opponents" in the other two games.