The ACDP, APC, Cope, DA, EFF, IFP and UDM say she's compromised and biased in handling issues affecting the ANC in Parliament.
The parties met on Thursday to discuss burning political issues.
They are now calling for her deputy, Lechesa Tsenoli, to chair the debate next month.
“The deputy Speaker should preside over the motion of no confidence because the Speaker has expressed her bias," said EFF leader Julius Malema.
The parties are also lashing out at the Public Protector for wanting Parliament to amend the Constitution on the role of the Reserve Bank.
“We call on the advocate Busisiwe Mkhwebane to unequivocally apologise to the nation," said UDM leader Bantu Holomisa.
"She [must] immediately stop turning this office into a political sideshow that serves the interests of a destructive faction within the ruling party.”
The parties are appealing to all South Africans to march to Parliament on 8 August in support of the secret ballot.