I would actually start with The Old Man and the Sea. It is short enough to get a feel for his writing style and will let you figure out if you are willing to continue to some of his longer works like For Whom the Bell Tolls and A Farewell to Arms.

As for the short stories, I personally would not have been able to appreciate them without reading his novels first.