We would take it. Live 30 minutes from Baltimore port. Easy, get up, have breakfast, finish packing, easy quick drive to port.
Add 4 hour drive to that and it makes it a LOT different.
And, we are booked on the April 15 TA, so will have to change flights to Norfolk, getting in late in the evening, so another hotel before the 4+ hour drive home