You need ot make sure you have the best CV possible. Make sure you write a skills based CV not a timed based CV. That way you don't put emphasis on being in prison but on what you can do.
Skills Based CV - Great CVs (UK)
Job seeking is tough at the moment, my boyfriend can't get a job skilled, unskilled or between at the moment and he has a degree and relevant work experience so I hate to think what you are up against.
You need something to put in your CV, did you do any courses in prison? You should try volunteering, it gives you skills, proves you contribute and have a good work ethics, it also allows you to network with potential employers. I volunteered in school and got one job and was offered another even though I was not looking for any jobs at all. It is also an excellent way to make friends
Maybe you would like youth work, they often have programes where they want men who have been to prison to help out young people who are on their way down the same route.
You have to be proactive.
It is easy to think of all the negative stopping you doing something but these will only drag you down.