Set up the 20 gallon - leave some capacity free.

Run the filter on the 20 gallon for a while, let the chlorine get out,
temp stabilize, etc. Take the filter for the 20 gallon, run it on the 3
gallon for a few days. Take the inhabitants of the 3 gallon, some water,
and both filters, put them on the 20 gallon. After a week or two, you
can put the 3 gallon filter back on the 3 gallon tank; or put it away if
you won't be using the 3 gallon tank. If you want to be able to use the
3 gallon tank at a moments notice, just run both filters on the 20, and
you can instantly set up the 3. If you put it away, you'll want a week
or two of running the 3 filter on the 20 before setting up the 3 again.

Depending on media compatibility, you can do this faster if you need to
by simply stuffing media from the 3 gallon filter into the 20 gallon
filter, but if you're not in a hurry...