The second version of the GP2X-firmware does have support for the USB-host-port. Go to settings and then system, enable 'menu extension', turn 'usb host' on and enable the drivers you want. Press B to acknowledge and you should be ready to go.
While the DC-DC-converter inside the GP2X is said to be able to source up to 500mA, I'm not sure if the rest of the hardware allows this. If you don't want to put your GP2X at risk, it's better to only plug in self-powered or low-power-devices into the USB-port and to use a powered USB-hub if you need to use devices that have a high power-usage. (In practice, I think there's room to experiment: I plugged in a few devices that overloaded the GP2X and all it did was reset or hang. That's just my experience and no advice or anything.)
I am not responsible for any bad stuff happening to you or your GP2X if you're going to build this yourself. As far as I know the information in this doc is correct, but I can't guarantee anything. If you blow up or otherwise destroy (part of) your GP2X, I'm not responsible. Neither am I if anything otherwise bad happens because of this doc.
Ow, for the void-your-warranty-crew: The gp2x in the pics is about 30 hours old :)