Simmons was literally standing around, got pushed in the back near Dudley and then Dudley pulled him to the ground while Simmons had his arms outstretched...
Simmons immediately gets back up on his feet and was clearly irritated that Dudley pulled him to the ground.
So, in other words, Simmons wasn't innocently standing off to the side staying out of the fray. Simmons was the 2nd Sixer there after Butler.
I don't condone Dudley grabbing his arms, but if Simmons isn't there, it doesn't happen. Also, Simmons, pushed or not, was basically chest to chest with Dudley when Dudley wrapped his arms around him and pulls him down with him.