Yeah that’s a great idea! I use that too. An encrypted USB drive makes sure you can trust the data not leaking, and spaced repetition can (should) be practiced to memoize the password. Then you’re good I’d say.