I ended up just building a box after looking for the perfect NAS and finding it didn’t exist. The software is usually just crap or the hardware is underwhelming. Got a Node 804 case to slap in plenty of HDD space. Running NixOS so I’m in control of the software. In retrospect I wish I had gotten a rackmount type case. Tossed in an Arc GPU for better transcoding shortly after the initial setup.
For the networking I found some repos with Nix and Gluetun (OCI containers). I don’t see them in my bookmarks, so it was probably a day project when I set up and didn’t keep the references.
That part is still in docker / podman. So any docker network guide just needs to be translated to nix.
Best resource I’ve found is searching GitHub.
My setup closely follows https://github.com/Misterio77/nix-config.
For servarr I just translated someone else’s docker compose setup to nix. There are some ready made nix ones you can look at like https://github.com/rasmus-kirk/nixarr/tree/main/nixarr.
The complex networking I just picked up over time once I knew my way around a little bit.
GitHub is your best resource. lang:nix search terms.
I wouldn’t run NixOS in a container. With native nix containers I’m pretty sure they share the store. For docker I’d use images built with nix (doesn’t run nix itself) or pull from docker hub.
OS: NixOS (high learning curve but its been worth it). Nix (the config language) is a functional programming language, so it can be difficult to grok. Documentation is shit as its evolved while maintaining backwards compatibility. If you use the new stuff (Nix Flakes) you have to figure what’s old and likely not applicable (channels or w/e).
BYOD: Just using LVM. All volumes are mirrored across several drives of different sizes. Some HDD volumes have an SSD cache layer on top (e.g., monero node). Some are just on an SSD (e.g., main system). No drive failures yet so can’t speak to how complex restoring is. All managed through NixOS with https://github.com/nix-community/disko.
I run stuff on a mix of OCI containers (podman or docker, default is podman which is what I use) and native NixOS containers which use systemd-nspawn.
The OS itself I don’t back up outside of mirroring. I run an immutable OS (every reboot is like a fresh install). I can redeploy from git so no need to backup. I have some persistent BTRFS volumes mounted where logs, caches, and state go. Don’t backup, but I swap the volume every boot and keep the last 30 days of volumes or a min of at least 10 for debugging.
I just use rclone for backups with some bash scripts. Devices back up to home lab which backs up to cloud (encrypted with my keys) all using rclone (RoundSync for phone).
Runs Arrs, Jellyfin, Monero node, Tor entry node, wireguard VPN (to get into network from remote), I2C, Mullvad VPN (default), Proton VPN (torrents with port forwarding use this), DNS (forced over VPN using DoT), PiHole in front of that, three of my WiFi vlans route through either Mulvad, I2C, or Tor. I’ll use TailsOS for anything sensitive. WiFi is just to get to I2C or Onion sites where I’m not worried about my device possibly leaking identity.
Its pretty low level. Everything is configured in NixOS. No GUIs. If its not configured in nix its wiped next reboot since the OS is immutable. All tracked in git including secrets using SOPS. Every device has its own master key setup on first install. I have a personal master key should I need to reinstall which is tracked outside of git in a password manager.
Took a solid month to get the initial setup done while learning NixOS. I had a very specific setup of LVM > LUKS encryption /w Secure Boot and Hardware Key > BTRFS. Overkill on security but I geek out on that stuff. Been stable but still tinkering with it a year later.
Download 5 seasons of some show from multiple sources or some artist’s entire discography, and want to normalize all the file names? It is way easier in the terminal.
I’ll check this out, but I use https://github.com/stevearc/oil.nvim for such tasks as I have nvim’s full suite of editor commands to rename all the files way faster than I could in a GUI. I’m sure there are GUI apps to perform a similar task, but I already know how to use nvim.
Yeah I don’t want locally deleted media (to free up space) to sync those deletions to my remote.
My crypted remotes wrap a B2 Backblaze one which doesn’t delete, just hides. Periodically I go clean it up.
You are correct, fixed!
https://github.com/newhinton/Round-Sync. Not in any app store and have to download and install from GitHub.
It is an Android wrapper around rsync rclone.
Setup a remote, setup tasks, and setup triggers. Mine syncs every night. It supports encrypting with your own keys. Large number of remotes supported from self-hosted to cloud.
Looks good to me. Interface to Dest Ports are your match conditions. NAT IP/Port are the translations performed on each packet matched inbound and the Dest.
Traffic going the other way reverses this operation on the Src instead of destination.
That’s an over simplification of NAT, but for basic port forwarding the general principal holds.
Immutable Nixos. My entire server deployment from partitioning to config is stored in git on all my machines.
Every time I boot all runtime changes are “wiped”, which is really just BTRFS subvolume swapping.
Persistence is possible, but I’m forced to deal with it otherwise it will get wiped on boot.
I use LVM for mirrored volumes for local redundancy.
My persisted volumes are backed up automatically to B2 Backblaze using rclone. I don’t backup everything. Stuff I can download again are skipped for example. I don’t have anything currently that requires putting a process in “maint mode” like a database getting corrupt if I backup while its being written to. When I did, I’d either script gracefully shutting down the process or use any export functionality if the process supported it.
Oh nice! I’ll have to dig into that. Wonder if its an implementation issue across vendors. I was always under the impression that DHCPv6 was the common convention if not static.
Ok. So a device didn’t get a dhcp address? No problem… It creates it’s open IP address and starts talking and try to get out on internet on its own…
Its not that different from a conceptual point of view. Your router is still the gate keeper.
Home router to ISP will usually use DHCPv6 to get a prefix. Sizes vary by ISP but its usually like a /64. This is done with Prefix Delegation.
Client to Home Router will use either SLACC, DHCPv6, or both.
SLACC uses ICMPv6 where the client asks for the prefix (Router Solicitation) and the router advertises the prefix (Router Advertisement) and the client picks an address in it. There is some duplication protection for clients picking the same IP, but its nothing you have to configure. Conceptually its not that different from DHCP Request/Offer. The clients cannot just get to the internet on their own.
SLACC doesn’t support sending stuff like DNS servers. So DHCPv6 may still be used to get that information, but not an assigned IP.
Just DHCPv6 can also be used, but SLACC has the feature of being stateless. No leases or anything.
The only other nuance worth calling out is interfaces will pick a link local address so it can talk to the devices its directly connected to over layer 3 instead of just layer 2. This is no different than configuring 169.254.1.10/31 on one side and 169.254.1.11/31 on the other. These are not routed, its just for two connected devices to send packets to each other. This with Neighbor Discovery fills the role of ARP.
There is a whole bunch more to IPv6, but for a typical home network these analogies pretty much cover what you’d use.
Very happy. My two daily drivers (Desktop and Laptop) are on Ubuntu but user space is managed with Nix.
All other machines are Nixos proper. Only thing keeping me back from moving to Nixos fully is I decided to piecemeal my own DE and I’ve just lacked the time to debug some issues related to gnome-keyring, computer locking, and coding up some system setting widgets.
While I have nothing against an arch recommendation, I wouldn’t rely on my distro when there is tooling to get any version of a library directly from pypi.
Having done some MQTT IoT work in the past, dropping a couple resources:
https://github.com/eclipse/mosquitto - CLI interface and C bindings. I’ve only used the CLI interface. Its a nice way to test communication with a remote broker.
RabbitMQ /w MQTT plugin - Message queue based on AMQP. We ran this as the server in a container. There maybe others options like 0MQ (been a few years so don’t know every option). I had my clients post to a wild card topic so that I could consume a single device or all devices. Consumption is not with MQTT though. We’d use AMQP on that side.
I recommended checking out package managers that will simplify using whatever version of a library you want with project level virtual environments.
I haven’t done heavy python dev since 3.7, so I don’t know the full landscape of options these days, but here are some references to dig into.
https://python-poetry.org/ is the one I started using as the first step up from pip.
It looks like there are some new contenders like hatch, rye, and pdm: https://dev.to/adamghill/python-package-manager-comparison-1g98.
There is also pixi referenced from the comments in that article: https://github.com/prefix-dev/pixi
I haven’t tested in Windows, but this is my setup Linux to Linux using rclone which the docs say works with Windows.
Server
Client
I use this setup for my local files and a similar setup to my Backblaze B2 off site backups.
The VFS implementation has been pretty good. You can also manually sync. Their bisync I don’t fully trust though.
I can access everything through android using https://github.com/newhinton/Round-Sync. Not great for photos though as thumbnails weren’t loading without pulling the whole file last I tested a year ago.
One method depends on your storage provider. Rsync may have incremental snapshots, but I haven’t looked because my storage provider has it.
Sometimes a separate tool like rsnapshot (but probably not rsnapshot itself as I dont think its hard links interact well with rsync) might be used to manage snapshots locally that are then rsynced.
On to storage providers or back ends. I use B2 Backblaze configured to never delete. When a file changes it uploads the new version and renames the old version with a timestamp and hides it. Rsync has tools to recover the old file versions or delete any history. Again, it only uploads the changed files so its not full snapshots.
Toooo real. Its like companies have taken the worst of everything and just call it agile. List out every task and estimate them so we have timelines, but don’t actually architect anything as that’s waterfall. Fake waterfall, with fake dates, but fingers will be pointed like they were real commitments, and spend a month doing it for this executive power point instead of fucking off so devs can build the damn thing.