If i can figure it out I’ll be sure to post something lol.
So far, i found a python project that is supposed to enable RAG but i have yet to try it and after a reinstall of my linux pc to Popos, im having less than success getting ollama to run.
Me personally, i use my AMD 7700XT to run ollama on my main pc. Can be helpful for troubleshooting as the internet gets worse amd worse to search especially if i dont know what the issue is. Thats my main use case for it but id like to set up something with RAG and use it to help me with documentation if i have questions.
I don’t think its super worth it to use a VPS for an LLM if you already have a decrnt gpu that you can run it on. If yoy dont already have the hardware, plenty of older gpus can run the models pretty well. My 1070ti still kicks ass all these years later and you can find them for $100 bucks or less on ebay used. Ive used it for ollama as well and it does just fine.
Will it be super fast or a really big model? No, but if its for personal use, i dont see any benefit to paying a monthly subscription for it and like i said, works well enough for me.
Its also more secure and private to host it yourself if thats worth anything to you. Thats one of the biggest reasons i self host.
I was recently looking for something like this and couldn’t find what i was looking for, so thanks!
I tried umbrel and wasnt super impressed. It seemed like it would function alright for the small amount of things i was trying to do but broke shortly after installing. I set up pihole and i think jellyfin since i was just testing it out on an older laptop.
I’ve heard decent things about CasaOS but have not used it myself.
What are you doing step-LAN?
Jellyfin was my go to as well. It was really nice to play with and finamp was as well. Once setup it was just as easy and nice to use as Spotify without all the ads and Spotify randomly playing songs after i finish a podcast episode.
If you dont need the drive to be fast and want best bang for your buck storage capacity, HDD is the way to go.
If you want speed and are ok with a higher price, SSD is best. Used enterprise HDD are usually a good value. Im in the US so not sure about where to purchase exactly.
Something to consider is if the drive you are storing the os and files on are the same, using am HDD will make it slower on start up.
Ah ok, thats pretty cool. I dont do much torrenting anymore cause my internet is not only limited but is kinda slow and unsteady.
Might check that out once im back in civilization with a decent connection. I miss gigabit speeds man.
Yea the real fun is always the set up when selfhosting. Then you look for the next project lol.
Im not familiar with gluetun, is that what you’re using to download tracks or something?
I personally really liked using jellyfin and finamp. Both were pretty great once i figured out how to set it up lol.
The issues i had were due to my lack of knowledge and trying to set things up to use it outside of my home network. Jellyfin itself was great as was finamp when i used them.
Now thats something I’d slap on my resume. Hope you get a bonus or something for saving your company money cause it’d be well deserved.
Well now im very jealous lol. Good for you man, thats a banger deal.
88 x 8TB drives? 704 TB of storage?
You got a whole damn data center lol. Good stuff dude. Was this off of ebay or direct from a company? I’m curious to know the total cost.
Fair point. My company keeps drives that get taken out of servers for office use. After that theyre recycled.
Damn thats cool! My work doesnt let me take drives old drives. Though we dont use sas drives so that might be why.
I certainly dont mind used drives for mass storage. I was planning to do some kinda raid anyways.
I haven’t gotten any kind of nas setup yet. Trying to avoid buying too much lol but good to know other folks are using sas drives too.
No use for them you say 🤔
I was actually just looking at buying some sas spinning drives for a nas. Found a seller on Amazon thats unfortunately gone before i could buy em but it was a pretty good deal. Where do you buy yours from?
ATnT should be able to tell you for sure. I remember reading about another person facing a cgnat using ATnT on reddit while i still went there so it very well could be.
And fairly easy setup yea. I did mine using a windows pc for testing as i was kinda in between places at the time and thats what i ended up using for jellyfin as well. Just lives on my media pc at the moment. The docs are pretty straight forward.
I recommend that or zero tier which is even more dead simple. Both are good but cloudflare does care about how much bandwidth you’re using so just bear that in mind if you think you’ll use the server for anything else.
Both are vpn tunnels so either should work just fine.
https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/
https://www.zerotier.com/pricing/
Self hosting can get pretty overwhelming but i find that using docs in addition to youtube videos helps a lot. I also recommend giving Linux a go when you feel up to it. It can be a very nice option if you’re working with older hardware.
I’m not super familiar with docker so im sorry im not much help there but i noticed that you mentioned a ATnT router. Are you using them as an Internet provider?
If so, you might have a carrier grade nat which makes reverse proxy like this not possible even if you do get caddy server working. I had a similar situation with my jellyfin server.
I had caddy server working but when i moved and started using a mobile internet provider, i had to use a vpn tunnel like cloudflare or zerotier to get around it.
All this to say, id recommend finding that out so if that is the case you dont spend anymore time on caddy.
Good luck either way.
The more you know! Do you know any guides that go over this? I’d be interested in setting this up.
Edit:
Its apart of thier docs. https://tailscale.com/kb/1019/subnets?q=subnet
I love this idea, because I often have questions like this as I transition away from proprietary software to open source alternatives
I think listing what software equivalents it might replace listed in the card.
I do see its listed when you click on the info card in the main list, but having it be visible before that might be helpful.
I know a lot of people won’t like the little ad cards when you do click on a software to review its page, but I think the way you implementted it is good. Its not intrusive and clearly states its an ad.
Edit: removed bit about categories.