Taking into account that the key word here is "imagine" you have an opportunity to enhance your sex life with some fun talk. If hubby wants to imagine you with other men, what is the cost to you other than some hot talk whilst engaging in fun with hubby? If he truly said it to you using the phrase "imagine" then you can only assume he wants to think about you with other men and you participate by describing scenarios of you with other men. Once you open up and do this with him it's a great start to finding out what his fantasy is really all about. Many men will take baby steps with the long game expectation that they will eventually talk you into this.
I started revealing my fetish to my wife by telling her that it was a fantasy and I wasn't sure if I wanted to go through with it, taking our sexy talk to the point where I could tell her that it had become my desire and giving her the green light to explore it in any way that she felt comfortable with. She at first just agreed to things I would say during sex-play and eventually started to tease me with it once she felt comfortable. She's taken it to the point where she dresses a little sexier from time to time and even leaves the house dressed sexily without me, going to places where she will come into contact with strange men and describing those encounters to me when she returns home.
If he said "imagine", let your imagination go wild and see where that takes you. If he was more forthright and said he wants you to do this, do some research on the web and let him know how you feel about it. It was a big deal for him to tell you this. Take it seriously.