I still would not do it, because I'm assuming that you are using the extra carriers because the car will already be fully loaded. It also doesn't take much for lots of light-weight clothes to start having significant weight. There is also the drag as you mentioned, plus it can raise the center of gravity of the car, which makes handling difficult. That's on top of the additional load you'll have in the rear of the car, in a car that's not really built for any kind of towing. A hitch hauler empty is going to weigh about 50 lbs. Using both really starts messing with the engineering and design of the car, and if I was only going to use one, I would go with the hitch haul.
If the clothes really don't weigh much, then it wouldn't cost much to ship them.