Granite counter tops also rest on a sturdy base, the caulk used to attach them doesn’t have to resist a lot of force trying to push the slab around, the caulk is mostly there to prevent liquids spilling into the cabinets and to provide a decent appearance. Yes, the caulk also somewhat attaches the slab, but keep in mind how hard it is to move to begin with, given the weight of the counter top.

Here, the weight of the glass door pulls the panel out of the rails via the hinges. Silicone won’t provide a lasting solution.