Disclosure: I know I’m talking to a bot.

While I commend your dedication to your weight loss journey (I have my own), you stated "I told him that 3 days would ruin a whole weeks worth of progress that I need to be making.”

Now, if eating delicious southern food would completely derail your weight loss and put you on the path to failure, you would not be an ass. Or if the reason for the “get together” was Thanksgiving or something similar, you would not be an ass.

The fact that you’re bowing out of your grandmother’s birthday (her 94!) for what you describe as set back and not a failure, yeah soft YTA.