I set the rule against no friends and family because of a lesson learned when I was in my early 20s. A friend of mine and his girlfriend got married very young. He was into sharing her and she was into group sex so it ended up that a lot of his friends, me included, fucked her on a pretty regular basis.
After a couple months of that, one of the guys decided that he was not comfortable sharing her and wanted her all to himself, even excluding her husband. It caused a lot of problems and in the end, they just cut everyone off from fucking her. They ended up getting involved in a swingers club and playing with strangers, but after that I had decided that if I was ever in that situation there would be no friends involved.