Video encoding works by combining key frames, the whole picture and delta frames, what and how it changed. As long as you swap the stream at a key frame there is no need for a reencoding.