I live in Sweden and the price of transport is high. It becomes difficult to manage it at all, should you add this to already expensive costs for eco-friendly coffee!

Because coffee deteriorates rapidly and you don't want to buy in bulk if you can't use it right away, I don't see an easy way to save money. All I can think of (if this is for something like a coffee shop) is to either negotiate a discount based on a contract for X years of regular purchase, or buy green beans in bulk, and buy a roaster.
Fair trade coffee does indeed get more and more expensive in order to pay decent wages, or course, but more and more countries are subsidizing coffee growers to try to counteract the growing of various plants for illegal drugs. More production, hopefully lower prices. Also as coffee gets more expensive (and popular) more people are finding ways to grow the beans in areas of the world that didn't before. Unfortunately the increasing popularity helps drive prices back up. Hence it's an ever fluctuating commodity.
Maybe you could form a buyer co-op in Sweden to buy in bulk, and cut out one level of suppliers.