I would doubt it. You won't get off the ship at 4 if you dock at 4. Then it is at least 45 minutes to the garden from the port. The Butchart Gardens is huge and it can easily eat up 2 hours for a "quick" walk through then another 45 minutes back. I would suggest you try Beacon Hill Park, it's free and much closer to the port. I'm sure that an excursion will be offered by your ship to Butchart Gardens and that may be the way to go if you want to get to the Gardens and back to your ship with certainty.