Anaerobic bacteria - extra internal filter left off

I have a tank with an UGF and an internal sponge filter with an air-hose
venturi.(for redundancy, the tank is not overstocked)

Sometimes I like to turn off the sponge filter to have quieter and
stiller water during the day.

It occurred to me that if either internal filter is turned off, then
harmful anaerobic bacteria may grow in the stagnant filter media (gravel
or sponge) at the expense of the beneficial nitrogen-cycling bacteria.

Is this correct and do you think that it is ever safe to turn off a
redundant(extra) internal filter?