It looks like one of the main problems with fusion are the materials.

Long-term research and design strategies for fusion energy materials

the materials required for commercially viable fusion power plants do not exist today. […] The challenge of fusion materials is compounded by the fact that design targets are not precisely defined and are rapidly evolving and by a lack of representative testing facilities.