In addition to what @Charles4515 has said, if you're going to go to the caves it's imperative that you reserve a time slot. During the cruise season the caves will have large numbers of visitors arriving via cruise line excursions and there is a limit on how many people can descend into the caves at any one time. You don't want to make a long and expensive trip to the caves only to find that you won't be able to see them.
With the new taxi fares the trip may be just a touch more expensive than Charles has estimated. Per the Bermuda Tourism Authority for a 5 to 7 passenger taxi the new fare is $11.55 for the first mile and $4.25 for each additional mile. The trip from the Dockyard is approximately 20 to 21 miles, which would result in a fare of about $92-$95, but just as in the US a tip for the driver is customary, so figure a total cost each way of about $105 to $110 dollars. If you are making the trip on a Sunday or legal holiday, there will be a 50% (not a typo...50% !!!) fare surcharge for a 5-7 passenger taxi.