It can definitely seem unfair at times. I mentioned in another comment that it feels like the game has a story it wants to tell and it punishes you for straying too far from the plot. Fortunately, it marks your conversation choices, so if you have to redo conversations, you know which path you previously took.