Hi, krisparker!
What you do is apply for grants or loans like anyone else. The possibility of obtaining a grant depends on the type of business it is that you are wanting to start. If it is one that will benefit the community in a great and far-reaching way, like something concerning education, or a business that will help those in need who can't particularly afford services or goods like a lot of people can, would be more likely to receive a grant. Obtaining loans are generally easier to get.
A person who has a conviction on their record; has done their time in whatever way, can apply for grants or loans just like anyone else. I don't know of any special grants just because someone has been convicted of a felony.
There are places in the U.S. however, that have organizations that do help people with criminal records to obtain jobs and maybe advise in terms of starting a business. Around where I live and also in the Chicago area, one such organization would be the Safer Foundation. Link is here:
Home - Safer Foundation Perhaps there is such an organization around where you live?
Two other great places to get advice for starting businesses are
http://www.sba.gov/ and
SCORE | Counselors to America's Small Business | SCORE
What kind of business is it that you are wanting to start, please? Depending on what the business is, it might not cost you as much as you might think in order to start it. I have a number of small, service-oriented businesses that I run. If there's something that I need in order to do whatever business that I do, I can generally borrow what I need from someone that I know until I am able to purchase it myself. Those sort of things that I might need would be certain tools.
Also, you can advertise your business on many, many sites on the Internet for free! You can also have any number of free websites! I'm thinking of getting rid of my ads in the Yellow Pages of the local phone book, because of the cost, and just relying on the business that I get from my ads placed on the Internet. I get much more business from my Internet advertising anyway.
Again, what kind of business is it that you are wanting to start, please?
Thanks!