In addition to other reasons listed in replies, there’s a fair bit of bacteria that excrete antifungal compounds into their environment so that they can colonize it more effectively than the mold.

There are viruses that attack fungi, too.

Any environment that hasn’t been sterilized by some means is a constant microbial warzone between bacteria, fungi, algae, archea, viruses, etc. with the current winners being determined by how favorable the conditions (moisture, pH, toxin concentration, etc.) are to each species.