There is a deposit fee for the transponder. You'd get it back when you return it, but you'd have the expense of mailing it back if you were planning to head back east. Plus you typically have to pre-load it with $40 or $50, which is more than you'd spend on a single trip.
So, you might save a couple bucks, but it would come at the expense of letting EZ-Pass hold on to your money for a while.