Republican Alabama Rep. Mike Rogers is reportedly stepping down from the House Steering Committee, days after he charged Republican Florida Rep. Matt Gaetz during a late-night speaker vote.

Rogers, an eleven-term congressman and the newly-elected chair of the House Armed Services Committee, angrily confronted Gaetz between the fourteenth and fifteenth Speaker votes. Gaetz had voted “present” on the fourteenth ballot, forcing the lower chamber into another roll call vote. Between the votes, Rogers charged Gaetz and had to be held back by GOP North Carolina Rep. Richard Hudson.

Now, Rogers is stepping down from the Steering Committee, according to Politico. Members of the committee, which doles out assignments to elected officials, are elected by region or selected by party leadership. Rogers will reportedly give up his seat temporarily to Tennessee Rep. Mark Green after the committee votes on seating for the House Homeland Security Committee.

A close ally of Speaker of the House Kevin McCarthy, Rogers reportedly threatened members who opposed the Californian’s speakership with a loss of committee assignments. The 21 GOP holdouts are not expected to lose committees over their move.