If you have a good relationship with your brother, there is no problem.

Surely, he will understand, maybe even without directly telling him your motives.

Wishing you all the best and many great memories. Life throws them your way when you let it happen.