Having sailed out of San Diego many times, I have always found the agents to be very kind and helpful.
I suspect the problem is that so many passengers want to be "first to board" that things can get chaotic. I have stayed the night before right across from the ship and notice that some people start lining up at 9:00AM well before the ship has been emptied of the previous cruise's passengers.
One time we did arrive after 1:30 and we were on the ship in no time. If you suspect you might have an issue, I suggest having a nice lunch at one of the restaurants close by and then start the process.