Master degree in EE here. It will almost certainly be fine to swap those batteries like that, I would be more cautious if it was some cheap Temu device but for a vintage Yamaha you’ll be fine.

One thing I will caution you about though, the synth almost certainly lacks under-voltage protection and could drain your 18650s below their minimum voltage. Make sure your cells have built-in protection or add some of your own.