In the compose file for Audiobookshelf, you need to add the service itself to the network you defined at the bottom. Something like:

services:
  audiobookshelf:
    ...
    networks:
      - default
      - nginx

(The default network is to keep it in the network that is created on docker compose up).