I thought room temp or "colder" water was
their best environment.

According to the top Chinese producer of high quality goldfish you are correct.

Why would they die so fast if the filter
stopped for 24 hours?

That's a good question and most likely was a bacterial revolt due to lack of
circulation. A large holding station studied bio filters and reported in as
little as 15 minutes the bacteria began a die off when circulation stopped.