Are you expecting some roundabout slightly techie way to do this? I hate to dissappoint you, but it's actually incredibly easy.
Just go to 'Your Twitter Karma' website at http://dossy.org/twitter/karma/, enter your twitter username and password, and click the huge 'Whack' button.
In a minute or two, you will be able to see everyone in your twitter account, people who follow, people whom you follow, people whom you follow but who don't follow you, people who follow you but whom you don't follow etc.
Just select the people who you want to follow or unfollow and click the right button at the bottom of the page i.e. bulk follow or bulk unfollow etc.
For the historically techie minded, when such sites weren't there, then the way to check who was following you and who wasn't, was like this:
On your Twitter home page, towards the top right you will see three numbers and text below each. Click on the text that says 'following'. See image below:
Now you will go to a page which will show you all the people who you are following. There are two symbols to the right of each name, click on the second one and see if the option 'Send Direct Message' is there or not. If it is there, it means the person is following you, if it isn't, it means that the person isn't following you.
Now you can happily un-follow all those who aren't following you back.
Go ahead and tweet, it is a liberating experience.