If I'm wearing boxer briefs which I prefer, low-cut and tight as hell, my junk will stay up and slightly to the right. If I'm wearing my loose comfy regular boxers definitely swinging down and around. Depends on the weather and my level of activity that day. The more I move around the less likely my dick is to stay up even in boxer briefs, it'll get shifted to the down position, cupped nicely with my balls.
Personally I prefer up, but until I find the perfect pair of undies to maintain that I just roll with the punches.