Parties may lead to his downfall, but is this the worst of what Johnson has done?