I've tried doing this. There is a minimum length for a message to be evaluated (which you can configure somewhere). I think it only applies to posts and not replies.