MuttMutt

First, you need to verify whether you actually have a public IP or if your ISP has you stuck behind CG-NAT, because that dictates your options.

​If you’re behind CG-NAT, ​Cloudflare Tunnel (cloudflared): This is usually the easiest path if you are mostly trying to access web-based services (HTTP/HTTPS) on your server. Your home server initiates the outbound connection to Cloudflare, so CG-NAT doesn’t matter. You just set up a domain (or subdomains) for each service you want to reach. If you need full network-level or SSH access rather than just web apps, check out ZeroTier.

​If you have a direct public IP (even a dynamic one), you can run a reverse proxy like Caddy paired with a free DDNS provider like DuckDNS or FreeDNS. One nice thing about Caddy is that it handles getting and renewing real, valid Let’s Encrypt SSL certificates automatically, so you don’t have to deal with manual or self-signed certs at all.

​Dealing with the WireGuard block, if your country’s ISP is using Deep Packet Inspection (DPI) to identify and drop WireGuard traffic, traditional VPNs like OpenVPN might get blocked pretty quickly too. If you still want a true VPN setup, look into AmneziaWG (it’s a fork of WireGuard specifically modified to scramble packet signatures and bypass DPI) or obfuscated proxy protocols like V2Ray / Xray or Shadowsocks.

dima.pm/integrating-tapo-cameras-with-frigate-inc…

Is the only thing I’m finding with audio output.

No major formal education. Did a DOS class at a local Community College back in the day, only did it for the certificate and made my high school pay for it. I already knew how to do everything they were walking me through anyway, I only showed up a couple days per week to drop stuff off.

The instructor made a big deal one of the last days. He stood near this whiteboard and said, “I’m ashamed, I’m appalled, out of everyone in this class, a high school student has the highest grade.”

I didn’t have the heart to tell him I learned DOS from the DOS Manual when I was 15. I learned how to do some very basic graphic programming and such when I was in elementary school on an Apple II.

I have worked as a network tech and pc repair tech for a WISP (Wireless Internet Service Provider) helped shoot 2.4Ghz WiFi connections up to 8.5 miles and shot a connection 7.6 miles myself. Most of the network troubleshooting was running multiple ping’s to different equipment to watch for issues. Learned on the job how to weatherproof connections using electrical tape and butyl rubber tape. It’s also where I learned how to crimp cables from the installer we had.

Most of the job was cleaning up and checking CPE’s that came in from previous installs, configuring the CPE’s for upcoming installations, and dealing with the phones. The owner was working for another company in IT about 3 hours away. I eventually quit when I was stuck trying to train a new installer and needed help with an issue, the owner quit answering the phone.

I later managed an in house ISP for an apartment building that was converted from a hotel built in the 50’s. I brought in 4 different connections, one from a small local WISP using the building as a tower 6Mbps, two AT&T 3Mbps DSL lines, and a cable modem 6Mbps. And provided internet for about 30 apartments using HPNA. Tried getting the owner to change some things but he was off the thought that if it works don’t fix it. Most people only got between 750Kbps to 1Mbps due to line quality and the technology.

No problem.

I try to explain things as best as I can so everyone can learn. I’ve been using TrueNAS since the FreeNAS 9.2 era so about 13 years now. When I started I had already dealt with a ton of issues and was searching for better than multiple single drives. My first real build was a Dual Xeon X5690 setup in an X8DT6-F so learning this about SAS was really important.

Sadly there are so many acronyms out there (and growing all the time) that is getting hard to just read the news let alone anything else without a little help.

Then it’s probably time to switch boards. Anything that will expand drive connections will need a PCIe slot. Using an adapter that goes into an m.2 is probably going to have issues. Using USB based devices is asking for trouble.

Yep. I’m using one myself.

If you are intending to run software raid then you want something that can be flashed to Initiator Target or IT mode. There is a JBOD function for some RAID cards and you DO NOT want that because it does not send all the same data that software raid wants from the dive. You also do not want to waste money on a card with RAM or Battery Backups since they are really designed for hardware RAID.

A SAS3008 card will really be what you are looking for.

www.ebay.com/shop/lsi-sas-3008?_nkw=lsi+sas+3008

With a SAS backplane or expanders you can connect 256 drives.

I have so many irons in the fire I can never quit…

Right now I am tinkering with a local only HomeAssistant AI voice assistant. I have an Ubuntu VM running docker on my TrueNAS which has an ARC A310. The reason I’m running a VM is because I’m using A Dual Edge TPU module for FrigateNVR while using the ARC A310 for video decoding. Ollama is running Qwen2.5 3B. I’m also running Wyoming Whisper and open-webui for the voice AI. I’m using a ESP32-S3 AI Smart Speaker Dev Board for voice control.

I found a project on GitHub that looked really good but was designed to always pull files and so I forked that to create a local only version and setup a small docker container for the audio files. Still working on modifications of the repository but the setup is working OK but I’m trying to work out the bugs.

I’m also working on a Halloween project. I have a plastic pumpkin with a big red arcade button for a nose, LED’s for eyes, a speaker, some car horns, proximity sensors, and a temper… I call it my Doom Pumpkin. It will go outside a week before Halloween and complain about people touching it’s buttons or lack there of during the day and stay quiet at night. On Halloween it will keep going till later and when someone presses the button it will randomly honk horns in different ways. When someone steps in front it will say funny things to get them to press the button. If they walk behind it will warn them something bad might happen (like saying it farts fireballs.) It’s all setup in ESPHome and will be controlled by HomeAssistant and NodeRed. There is also a theft prevention mode built in that will cause it to just spazz out and blow the horns.

I’ve still gotta pull another 500 feet or so of ethernet then get those and some others dropped into the server closet and shift some stuff around so it feeds into the rack. I’ve also got to pull my current camera lines from my POE injector and run them into my POE switch.

This is all on top of building and installing racking for solar, getting the solar installed, pulling a wall in a couple inches at the top where it pulled away during an earthquake before we bought the place, and fixing a bunch of other stuff.

Supermicro boards go great in Supermicro 4U cases. I’m using an X10DRH-C in a case that holds 36 X 3.5" drives with a SAS backplane. You can grab a CSE-847 for 350 plus tax and shipping on ebay.

A set of cheap rack rails attached to some 2x4’s with a couple casters on the bottom and you have a rack you can stick in a closet and pull our when you need it.

The cheap rosewill cases are ok. But it you need to swap a drive (I have the 15 bay version that’s about 100 bucks I used for my server to begin with and now use it for a router with only two drives) it’s a pain to deal with. You have to pull the top, then remove screws to pull the cage for 5 drives, then pull the drive that you need to swap. It doesn’t sound like a big deal til a drive goes bad, or one of the 120mm fans that cool the drives dies.

Image

The Supermicro is the bottom one, the Rosewill is above it.

The one built in to Caddy for anything that doesn’t really have authentication. Everything else uses its own and i only have a couple services accessible outside the home.

Most everything is only accessible within my home and my guest wifi is on its own vLAN. Even then I’m the only person who actually uses most things that I setup.

It’s one of the reasons why getting a SAS controller is really nice, backplanes make it even better.

I was preforming a burn with 10 X 8TB drives on a controller that already had a couple operating ZFS pools totaling 18 X 4TB drives and a couple SSD’s and there was no slow downs. A SAS3008 controller can support over 6000 MB/s and has a PCI Express X8 3.0 bus.

I’ve been building my servers and network out of old enterprise gear for a while now. It uses more power but the things you can do are truly amazing. Sadly the ai boom drove prices way up compared to a couple years ago, what I bought for 900 in 2024 is running around 1500 today.

I understand. If you buy a used server then add some drives later you can have a great NAS IMHO. I upgraded from an X8DT6-F with 384GB of RAM and a pair of Xeon X5690’s right before things went sideways. The MoBo has the SAS controller already flashed to IT mode so it’s ready for ZFS. But it’s not exactly light on power and with a 2U chassis and a handful of used 8TB SAS drives you are looking at around 1200.

My current server is a X10DRH-C with dual Xeon E5-2683 v3’s with 128GB of RAM in a 4U chassis with 11 X 8TB SAS drives in a RaidZ3 configuration. Just the MoBo, chassis, cpu’s and 64GB of ram is running about 1150. The drives used are 110 each and before you think you should just get those, SAS drives require a SAS controller and you only get those in enterprise equipment.

But if you can pick up a little here and a little there you can have a nice system. But right now isn’t a great time to get in the game.

I’m not saying that.

What you need to do is decide now if the drive you will buy will be used for a RAID array. If it is a desktop drive won’t be in a RAID array on a NAS system. Many NAS’ will have random writes to the pool. Desktop drives aggressively park the heads, the load and unload of the heads wears them. In a NAS system they can actually wear out.

Over the last 15 years drives have become a bit more specialized. You already found out about surveillance drives not being a good fit for much other than surveillance/DVR. Desktop drives are fine for desktop loads and usage but outside of that or single drive usage they are not useful. NAS drives are meant for NAS usage in RAID arrays. Back when the WD green drives were available years ago you could convert them from a desktop drive to a NAS drive using a tool called wdidle (WD Idle) but that isn’t the case any more.

Using a NAS drive on its own will work in a pinch but if it has an error it won’t try to recover it like a Desktop drive would because it’s made with the idea that it will be in an array that will deal with the issue. Plus once you start loading it up you will have to wipe it to put it into an array unless you go for ZFS mirrors or RAID 1. If the NAS Appliances have some sort of special trickery that allows you to expand one disk at a time and add redundancy I’m completely unaware as I’ve never put much stock in them. I’ve been running FreeNAS/TrueNAS for over 10 years.

en.wikipedia.org/wiki/Standard_RAID_levels

If a NAS or Enterprise drive has an error it sends the information to the host to be logged so that the end user can have the information available.

So like an Unrecoverable Read Error (URE) pops up on a sector. A drive that is built for RAID use will just say, “Couldn’t read it” and moves on. A Consumer drive meant for a desktop will try and try and try and try to read that bad sector. In a NAS situation where another drive will be able to fill in the data the controller (hardware or software) will just deal with it by pulling the data from another drive and keep moving.

The drive may not be bad as a whole but it does mean that over time it is more likely that drive will have more errors.

NAS drives are not inherently more reliable, yes they can deal with a bit more vibration and such but it’s the firmware inside that is different. Enterprise drives are another step up again from NAS drives.

What are you using as a router? How are you you trying to setup a reverse proxy?

I will be honest, I would do it for free. Money is nice but helping people is more important to me than that. However my understanding is with OpnSense and using Caddy for reverse proxy with the ACME client for Let’s Encrypt certificates. Beyond that I am outside of my element.

It’s not just about the advanced functions. Many of the older more basic tools are single threaded which will potentially limit the performance. As you figure things out you will want to do more and you may find the current tools more of a limiting factor. But the choice is yours, I have 2 48port gigabit switches and WiFi SSID’s that connect to specific vLAN’s through tagging. I started with some dumb switches and added my 10Gbps backbone switch which I used as a dumb switch for years so I could connect my desktop and server over a faster connection.

In my equipment an untagged port is what a port is where the vLAN is stripped away. A tagged port has the vLAN tag passed to the device. If you can set multiple vlan tags on the same port that port becomes a trunked port. You may also be able to set a vlan as untagged on that port, if a device is plugged into that port it will by default be on the untagged vLAN. If the device is able to handle vLAN tagging it can live on the vLAN’s you set up as well.

It took me a bit to figure it all out and get it working. I spent about 20 hours configuring things before I started making the switch from a single net to multiple vLAN’s. I spent another 8 hours making the change and 5 or 6 more tweaking things.

No. I didn’t use any video’s to set mine up and wouldn’t even know where to find one that is up to date.

Dnsmasq isn’t being deprecated that I know of but when you begin doing more advanced functions some tools work better than others. The “New” rules are fully functional and I suspect the old ones will slowly be removed in later releases. They are revamping some stuff, ISC DHCP used to be the go to but that is being passed out by the creators. www.isc.org/kea/ and OPNsense is cleaning up things so it all works with the rest api and is higher performance.

Trunking is when a port can deal with all setup vLAN’s not sure about your switches as they are pretty basic looking. I’m using a pair of Dell PowerConnect 5548’s and a Quanta LB6M which are much more complex. But basically think of a vLAN as a branch of a tree and the Trunk is the base that connects to everything.

Ok, so first of all a TON of things have changed in OPNsense in the last couple updates so you may just want to pull everything you did. It looks like you are using the old firewall rules which while they are going to stick around they are trying to migrate people away from them.

You should be using KEA DHCP as that is the modern and latest and greatest DHCP server that is vLAN aware. Also UnboundDNS is a recursive DNS server and designed for modern networks, with the advantage of being able to use DNS Blocklists to block ads and other “junk.” Just remember that the blocklists live in RAM so if you don’t have much RAM available I wouldn’t recommend using them.

NOTE: If you are using Dnsmasq DNS&DHCP you will have to turn it off before you can enable KEA DHCP and UnboundDNS as it will tie up the ports needed and they will fail to start. You can copy everything over before making the change and if there is an issue you can switch back just by stopping the new ones and enabling the old one.

With that said one of the biggest things is that with a managed switch you will need to trunk the port that OPNsense is plugged in to for the LAN unless you are using multiple ports on your OpnSense install and then those ports will need to be properly tagged and you will need to trunk any ports that are linking switches together you will have to figure that out on your own but I suggest grabbing a copy of your switch’s manual (and if you use a chatbot upload that file to it for help.) Then you can use vLAN tagging for each port that you want to receive an IP address from a particular pool automatically. You can also trunk ports that you want to use for management but set the default the port will use for access so the device can get an ip via DHCP, this really only works well with Linux. If you are using windows you will need to just create firewall rules that allow your device to talk to the other vLAN’s instead.

When you create your firewall rules you have to understand that you can only preform one action per rule. If you want to allow your vLAN 1 network to talk to vLAN 10 that is one rule. To allow vLAN 10 to talk to the internet that is another rule. You can use floating rules to do the work on multiple vLAN’s but that should be limited. If you select more than one network interface a rule will becoming a floating rule and will process before other rules so if you create a rule to block something later on but have the same interface set on a floating rule the block will not work, it’s better to enable piece by piece than to blanket enable and then try to block. With 5 vLAN’s I have 5 floating rules and 34 regular rules plus 40 automatically generated rules (which handle things like allowing DHCP access and basic protections.)

Here is a firewall rule that allows my “Trusted” vLAN to access my “Camera” vLAN as an example. The Categories are not important but make finding what a rule deals with later on a lot easier, they are set under the firewalls - categories. You should also use good descriptions for this reason.

Image

You will also have to explicitly allow access to services like DNS. This is how I am allowing my “Trusted” vLAN to access DNS services on my OPNsense.

Image

This is how I allow my trusted network to access the internet. If you have multiple WAN’s you can choose a specific one or if you have failover configured you would likely select the “group” you created when you setup the failover.

Image

If you need more help let me know. If you have been tinkering with a bunch of stuff you may want to start over, just backup your current configuration and reset everything to defaults. If you can’t figure it out you can reset to defaults and restore your configuration.

Give me a bit and I will be able to help you. I have a multi vlan setup with vLAN tagged switches and wifi ap’s.

Both OpnSense and PfSense have FreeBSD as a base. That means you really want to avoid realtek NIC’s.

I’m running OpnSense on a Supermicro X10SLL-F, with a Xeon E3-1226 v3, 16GB of RAM, and an Intel X520 10G NIC with the OS installed on a mirrored pair of 240GB Intel D3-S4510’s. I underprovioned the drives by about 50GB to lower write amplification and allow the drive controller a scratch pad space for garbage collection even if the drive fills up with logs and snapshots. This is simply done with:

zfs set quota=190G zroot

I have the following services (beyond the routing and firewall) running:

ACME Client - let’s encrypt certificates

Caddy - to expose my home services for access via reverse proxy

Chrony - network time server

CrowdSec - running all free rules

Intrusion Detection (also known as Suricata) - just under 193K rules

Kea DHCP for IPv4 providing IP addresses for 5 vLan’s

Monit

Ntopng with Redis as the database

NUT

Router Advertisements for IPv6 for 3 vLAN’s

SMART - for dive testing but it basically consumes no resources

UnboundDNS with abuse.ch, Steven Blacklist, and Hagezi multi Pro++ Blocklists.

Firewall has 79 rules, NAT has 8 Destination entries, IPv6 is provided by Route64 over Wireguard, remote management access is done via Wireguard tunnel. vLAN’s are Management, Trusted, Smarthome, Cameras, and a Guest network which is WiFi only. Smarthome and Cameras vLAN’s do not have internet access except for my HomeAssistant server.

I’m using a Quanta LB6M for my fiber backbone which is 10G with a LAG connection to a Dell PowerConnect 5548 and 5548P gigabit network stack.

I’m utilizing approximately 62% of RAM (which begins to have issues once you get over 80% used) and a load average that generally sits at .30 unless the GUI is open. Swap is 0

I have a 1Gbps fiber internet connection through a Callix 803g and a cellular backup via a Netgear LM1200 that is controlled by OpnSense when a 20% or greater packetloss is detected.

If you have a fiber connection or notice packetloss/slowdowns after 9 minutes you may need to set:

net.link.ether.inet.max_age = 540

This is because by default FreeBSD sets a 20 minute ARP maximum age while many ONT’s set a 9 minute maximum age.

If you plan to use Intrusion Detection the required RAM will balloon quickly and the UnboundDNS Blocklists live in RAM as well. You can easily get a similar setup to mine on Ebay for 200 or less, old server boards or workstation boards with IPMI are great to have if something acts up and the router is living in a closet without a monitor. The old server boards IMHO are also super stable and they are designed to run for years without being shut down.

I picked up enough used SAS drives to build an 11 drive RaidZ3 pool. 100 USD each shipped. DIF formatted so had to low level format to native 4k sectors, then run a full run of badblocks, and finally a long smart test to verify no errors. Had a couple bad drives that the seller replaced no questions asked when I provided the smart logs.

SAS controller and backplane opens up a lot of drives that SATA controllers can’t touch.

My old pool drives will be repurposed for a FrigateNVR storage point and a storage point for some other stuff as well as spares for other pools.

I learned B I 2002. I can make either but have to really think about A while I’m making it.

Over the distance I’m running cables (about 60 feet total for the longest runs) it really doesn’t matter. I’m running fiber for anything beyond gigabit anyway so I say aquamarine, violet, and lime green for lyfe.

There can be multiple things at play but one thought it’s you may be hardware constrained. This is not limited to just your server but also how you are connecting. If you are using a budget cell phone with a bunch of things installed and running it will get slower. If you use a bunch of social media apps disable them before starting a test so they are not running in the background or eating into your connection.

Another is it sounds like you are using your Tailscale connection to then reach out through your regular connection to a speedtest. So you’re using the same connection with multiple data streams at the same time. You will be limited to your slowest speed when sending data to or from a server through you tailscale connection to the internet so give up on that 109Mb/s. You also have to take into account what your connecting devices speed is, if you are on cellular with an 18Mb/s connection that is your top speed.

Finally instead of running a speed test to the internet, spin up an instance of openspeedtest on a computer that is not hosting your tailscale connection and test to there. When you are starting to setup a homelab it can be useful to have an in house speedtest anyway.

I have the Coral dual tpu module and have an adapter coming that will allow me to use two of them for a total of 4 modules, paired with an ARC A310 for video decoding.

The only bad thing about the Google Coral is that Google has stopped development and archived the driver so one you get to a certain linux kernel version you will have issues unless you switch from the official version. lemmy.world/post/46390410

If I didn’t already have the Coral I would probably go with the Hailo 8. I’m going to use the ARC A310 for detection on a couple streams and the Coral for the rest since the detection is more basic.

FrigateNVR is a bit of a pain but has worked great with all my Reolink camera’s. I also link them into HomeAssistant. I have one 4K PTZ camera, four 1080p fixed cameras, and one dual 4K lens 180 degree view camera all POE powered and hooked into both FrigateNVR and HomeAssistant.

I use a Quanta LB6M for 10G backbone and a Dell PowerConnect 5548 for Ethernet with a pair of DAC cables linking the two with my router and server connected via OM3 to the LB6M.

Same…

Image

Hardware routers can be killed for many reasons. Age and an inability to upgrade them to suit current needs is the biggest one. OEM’s do the fairly quickly as they build devices auth the resources to work for today and maybe tomorrow. After their product is no longer made they want it to be obsolete not long after so they can claim they can support is no longer viable.

OpenWRT tries to augment and support devices for as long as possible but at the same time the hardware limitations still exist and often the storage will no longer support the latest version due to the size of the new version.

As far as when that will happen depends on multiple factors that have the ability to change at any time. There viable be a massive flaw in a portion of the software that requires a huge rewrite and the addition of code that will make the software too large to fit.

Sometimes someone will create a version with less used features available so that it can still be used but it’s a losing battle

If you want true longevity repurpose an old computer into a router. OpnSense is what I use and recommend. Add in something like a TP-Link EAP650 or two and you have a rock solid platform that can handle a lot more and last much longer. Add Power Over Ethernet (POE) along with an Omada controller and you can position the AP in a place where it will work the best for your use. You can have enough access points to have a full signal anywhere even if your place is the size of the Louvre. When new technology comes out you can upgrade the AP’s as needed when needed and upgrade the router as well in a similar fashion.