You’ll be fine, both in Japan and back home I’ve lent people my card and vice versa, the only time I ever had someone look at the name was buying something from Bic Camera. It was over 5k yen so I wanted to get the tax discount, and they looked at my passport and compared the name on my debit card.