I’ve come to the conclusion that you need to actually put something playable in front of a bunch of people who don’t know you, to properly gauge how compelling your game is. Ideally a vertical slice where with the necessary polish to convey the feel of the end product.

Let them play and watch. Listen to their feedback, but don’t put too much weight on it. Players usually can’t account for the limits of a prototype, even if it is a vertical slice. Also they usually can’t quite pinpoint what made them like or dislike something, but they will want to give you feedback. Just note what they say and try to figure out later what the underlying issues might have been. More importantly, you need to watch them play. Ideally you want a setup where there are a bunch of games, like at a gamedev gathering, or at least something else to do, where people can freely decide to play your game or not and don’t feel forced to do so. Seeing how they interact with the game on their own terms, seeing how long they play, whether they get their friends to play it as well, is the true litmus test.

To have a successful indie game, I believe, it needs to capture people on its own, just by it’s presentation and gameplay. And it needs to deliver such a memorable experience that people will recommend it to their friends and talk to each other about it.