I'll second Kalel33's statements. I recently was looking at the same two camera's and ultimately chose the 40D over the Rebel. In terms of potential image quality, the Rebel is very capable and gives up little if anything to the 40D. If the two get to the same point in terms of imaging, the difference is in how they arrive there. The 40D is simply in a different league, with much better build quality, handling, and features. You'll spend much less time drilling down through menus to change shooting parameters on the 40D as the most frequently changed functions are accessed directly through buttons on the body. The thumbwheel on the back is very useful for setting exposures (exposure comp. in auto modes and aperture in manual) and allows each button to select two variables as the thumbwheel controls one function after pressing a button and the fingerwheel controls a second. The first time I shot a couple hundred images with the 40D was the first time I didn't find myself saying, "now why did they have to make it so hard to change x." It definitely helped that I had read the manual.
Don't ignore the blazing AF speeds and low light functionality, higher ISO, sync speed, shutter speed, and burst speeds either. You'll appreciate the bigger brighter viewfinder every time you look through it, and while spot metering is not often used, it's invaluable when you do use it. These things can make the difference between getting a shot or not.
The only reasons not to go with the 40D are price, weight, and bulk. The 50D has driven prices down somewhat, so that's less of an issue than it used to be. I got my Canon-refurbished 40D w/ 18-55 lens for $770 from B&H, with a 90-day warranty. As with price, it's up to the you to decide if bulk and weight really are issues. More weight, up to a point, makes a camera less susceptible to hand shake (ever try to look through high-powered pocket binoculars? they're less steady than full-sized versions as there's not enough mass to resist small movements of your hands). Extra weight is usually the direct result of more robust construction, and the larger size makes more room for your hands and the controls (even for my relatively small hands).
No doubt, the Rebel's an excellent camera, but the 40D is a step up in many ways and I don't think anyone who shoots regularly would regret choosing it. I don't!