The B650M is a smaller size and therefore has less features, but overall they seem similar. Biggest difference would be the integrated SSD heatsink and better VRM design + cooling of the non-M version. Also the second SSD slot being connected to the CPU instead of the chipset, if you ever want to put in two M.2 drives. One thing to watch out for is that both of these boards use a Realtek LAN chip which sometimes can be problematic with Linux.
I’ll echo the rest of the comments and say Debian with docker. I use podman myself but the setup should be similar enough. I used the official Jellyfin guide from their wiki and got everything working in a day, including transcoding.