Randomly getting ECH errors on self-hosted services.

Submitted ⁨⁨1⁩ ⁨month⁩ ago⁩ by ⁨Darkassassin07@lemmy.ca⁩ to ⁨selfhosted@lemmy.world⁩

In the last couple of weeks, I’ve started getting this error ~1/5 times when I try to open one of my own locally hosted services.

Image

I’ve never used ECH, and have always explicitly restricted nginx to TLS1.2 which doesn’t support it. Why am I suddenly getting this, why is it randomly erroring, then working just fine again 2min later, and how can I prevent it altogether? Is anyone else experiencing this?

I’m primarily noticing it with Ombi. I’m also mainly using Chrome Android for this. But, checking just now; DuckDuckGo loads the page just fine everytime, and Firefox is flat out refusing to load it at all.

Image Firefox refuses to show the cert it claims is invalid, and ‘accept and continue’ just re-loads this error page. Chrome will show the cert; and it’s the correct, valid cert from LE.

There’s 20+ services going through the same nginx proxy, all using the same wildcard cert and identical ssl configurations; but Ombi is the only one suddenly giving me this issue regularly.

The vast majority of my services are accessed via lan/vpn; I don’t need or want ECH, though I’d like to keep a basic https setup at least.

source

Comments

Sort:hotnew top

bobslaede@feddit.dk ⁨1⁩ ⁨month⁩ ago
Any chance you are both accessing your services locally with a local DNS, and publicly with something like Cloudflare?

source
- Darkassassin07@lemmy.ca ⁨1⁩ ⁨month⁩ ago
  I do have external acces to Ombi via cloudflare; but the device I’m seeing this problem on is permanently connected to a VPN hosted from the same server machine as ombi/nginx with ‘block all connections without VPN’ enabled. And this testing has been done from within the same LAN.
  
  It should never see/reach cloudflare for this service.
  
  source
  - bobslaede@feddit.dk ⁨1⁩ ⁨month⁩ ago
    Try with nslookup and see if you’re resolving the domain to both your local ipv4 address, and the Cloudflare ipv6 at the same time. I am using pihole for my local DNS, and it would give me both my local address, and also the Cloudflare ipv6 address.
    
    source
    -> View More Comments
  - solrize@lemmy.world ⁨1⁩ ⁨month⁩ ago
    Can you verify with wireshark that the traffic is only going through your lan? I’m not hip enough for nginx but I used to have to run apache under gdb all the time to trace random errors from the server. That would be next, if the traffic is really local.
    
    source
    -> View More Comments
Moonrise2473@feddit.it ⁨1⁩ ⁨month⁩ ago
I had a similar problem when using nginx proxy manager. In the end I just gave up and directly used cloudflare tunnels+cloudflare ssl

source
- Darkassassin07@lemmy.ca ⁨1⁩ ⁨month⁩ ago
  I think I’ve found the problem:
  
  It seems my issue is pihole being unable to block/modify dns requests for HTTPS records, which don’t match the LAN IPs pihole handed out in A/AAAA records.
  
  I’ve disabled cloudflare proxying so they don’t have HTTPS records to serve, but I’ll have to replace pihole with a better lan DNS solution if I want to turn that back on.
  
  source
sk@hub.utsukta.org ⁨1⁩ ⁨month⁩ ago
this issue is an ongoing discussin over at NPM too, very mysterious

https://github.com/NginxProxyManager/nginx-proxy-manager/issues/3982
source
- Darkassassin07@lemmy.ca ⁨1⁩ ⁨month⁩ ago
  Thanks. That seems to be a similar, but slightly different error. I think the below may apply though.
  
  I believe I’ve tracked down more of my issue, but fixing it is going to be a hassle:
  
  When cloudflare proxying is enabled, there are 3 DNS records involved; A record with cloudflares ipv4, AAAA record with cloudflares IPV6, and the key to this puzzle: an HTTPS record with cloudflares ech/https config.
  
  With pihole I can set DNS records for A/AAAA, but I have no way of blocking/setting the HTTPS record so it gets through from cloudflare.
  
  The A/AAAA records don’t match the HTTPS record, so browsers freak out.
  
  Once I disabled cloudflares proxying, I no longer get HTTPS records returned and all works as intended.
  
  I’ll either have to keep cloudflare proxying disabled, or switch pihole out for a more comprehensive DNS solution so I can set/block HTTPS records :(
  
  source
  - bobslaede@feddit.dk ⁨1⁩ ⁨month⁩ ago
    I’ve fixed the same issue for me.
    Originally I had this in my Local DNS settings in my Pi-Hole:
    
    - service1.domain 10.0.0.4 - service2.domain 10.0.0.4 - service3.domain 10.0.0.5
    
    I changed that to this:
    
    - host1.domain 10.0.0.4 - host2.domain 10.0.0.4
    
    And then I added CNAME Records to the services like this:
    
    - service1.domain host1.domain - service2.domain host1.domain - service3.domain host2.domain
    
    This fixed the whole thing for me :)
    
    source
    -> View More Comments