A rough diamond is a diamond from the mine that hasn't been cut yet.
From E notes : A diamond in the rough is someone who has not been polished yet, but has potential. The rough implies that the person is rough on the outside, but inside there is more to the person. Usually the person just needs a little training and education in order to really shine.