Firstly stop using a chamois!!!!
You will be hard pushed to remove swirls by hand!
To remove these which are put in during washing you will need to use a polishing machine.
A friend offers swirl removal for free for Minis if you can get your car to the south coast.
PM me for my email address if you are interested.