Around 5" seems to be about average, but it's also about girth. I know many women who would prefer a fat 5" to a skinny 8". I think both length and girth factor into whether we call it big, average, or small.
Now as far as guys under 4" - I've heard stories from some partners, but they appear...