You have to ignore the obsequious optimism bias LLM’s often have. It all comes down to their training set and if they have seen more than you have.

I don’t generally use them on projects I’m already familiar with unless it’s for fairly boring repetitive work that would be fiddly with search and replace, e.g. refactor the common code out of these functions and refactor.

When working with unfamiliar code they can have an edge so if I needed a simple mobile app I’d probably give the LLM a go and then tidy up the code once it’s working.

At most I’ll give it 2 or 3 attempts to correct the original approach before I walk away and try something else. If it starts making up functions it APIs that don’t exist that is usually a sign out didn’t know so time to cut your losses and move on.

Their real strengths come in when it comes to digesting large amounts of text and sumerising. Great for saving you reading all the documentation on a project just to try a small thing. But if your going to work on the project going forward your going to want to invest that training data yourself.