I find a pleasant no thank you to the taxi drivers works very well, at least it does for me.
I consider Chattel Shops in Holetown tourist shopping not local shopping, for the local experience I frequent the vendors with trays outside the Chattel Shops.
Since the OP asked for places close to the port I thought of Pelican Village. Of course they can also walk along Broad Street for another shopping experience.