I also recently got into selfhosting LLM. Having an AMD card meant I had to scourge for solutions since everything expects to have CUDA suppport which means having Nvidia cards. Koboldcpp has a fork with ROCM support which works on my machine, so I’m content with that for now.

I shudder imagining that a stupid company could make .cum domains actually happen. Looking at you Google and your .zip trolling.

There has to be a cultural shift as well. It’s not the early 2000s anymore where a substantial portion of internet users could tinker around their desktop computers. I recently got fiber at home and we’re locked behind CGNAT. I could look for a solution for myself since I grew up opening ports on my router, but imagine someone who grew up with bubble-wrapped smartphones trying to navigate their way through that bs.

So that explains why it’s so cheap.

Porkbun has been fair to me. Recommended.

Imagine yourself running a ChatGPT equivalent. Wouldn’t you be tempted to record all the weird and dangerous stuff people ask just in case law enforcement pays a visit? A person can be a privacy advocate and fight for the cause, but a company has bills to pay and investors to please firsthand.

I’m can’t believe I’m considering purchasing another GPU just so I don’t have to depend on OpenAI or anyone toying around with the models.