question about earthworms in pond

Have small pond (300 gal) with six 4 to 5 inch goldfish....today I noticed
the pond has dozens of live earthworms in it. I only fed the fish once or
twice since I uncovered a couple weeks ago (the water temp has been
averaging around 50 degrees) so anyway the fish seem to be eating lots of
them....and every plant container seems to have colony of dozen or so in the
bottom...even finding them in pre filter. I noticed a few every time it
rained last year....but many many more now. All different sizes........Is
this a problem?? if so what to do??