I’m afraid you’re mistaken. The word “balloon” in the phrase is not actually a balloon, but a bastardisation of the Dutch “paalloon”. This literally means “pole wages”, and is the money Dutch pole fishermen were paid for their work. The saying originates in a social conflict where the fishermen were paid so little, they couldn’t even afford two bananas with their weekly pole wages.