What is the recommended I-beam size for lifting 5 tons (10,000lbs) having a 40' span?

You need to be more specific:

• How is this beam to be supported - is it cantilevered, or simply supported at both ends, or what?
• Is the 5-ton load a point load (as on a crane) or is it distributed along the length of the beam (as for a floor joist)?
• Where on the beam is the load to be placed?
• I assume you're asking about a steel I-beam, correct?
• Are you inerested only in the beam supporting the load without failure, or do you have a criteria for maximum deflection of the beam (quite often in designing floor beams, for example, the suporting beams are way over-designed from a strength point of view in order to keep deflection to a minimum - office workers don't want to feel like they're walking on a trampoline).
• Any constraints on the profile of the I-beam - do you care whether it's tall and skinny versus short and wide?

it's a bridge crane, supported on both ends. Currently we're using a "S18x54.7" x 40' I-Beam but feel it's not adequate.

First of all let me say that any determination of I-beams for a bridge crane must be carried out by a licensed Professional Engineer. So take what I am about to say with the appropriate grain of salt.

I calculate your beam will experience a max load of 14.9 KSI, and will deflect by about 1.1 inches under load. This assumes the two ends are simple supported, and takes into account the stresses due to both the dead weight of the I-beam plus a point load at mid-span of 10,000 pounds. It does not include any additional loads, for example due to the hoisting apparatus. It also does not take into account any dynamic loads, which come about from the acceleration of the load as you start to lift it.

To put 14.9 KSI into perspective, the yield stress of common steel used in I-beams is typically in the range of 36 - 50 KSI, depending on the alloy used. So at first blush this seems OK. However, I have not considered the possibility of lateral torsional buckling, which may be a factor given a relatively slender beam profile and depending on how the load is carried (load applied at the top of the I-beam may be destabalizing) and not knowing if the I-beam has any shear plates welded into it. So again, you should consult a licensed PE to be sure.