For YouTube content I really like the Tubular app on Android. It has Sponsorblock built-in, which you mentioned, but it also has an option to skip silence.

I find it particularly nice for gaming content, or any point where the YouTuber is reading text aloud.

I must have skipped hundreds of hours of pointless silence by this point