Koboldcpp should allow you to run much larger models with a little bit of ram offloading. There’s a fork that supports rocm for AMD cards: https://github.com/YellowRoseCx/koboldcpp-rocm

Make sure to use quantized models for the best performace, q4k_M being the standard.

It doesn’t matter if actual copies of the original images exist, reproducing copyrighted concepts like characters is still infringement. It’s the same whether it’s done by a human or a machine. The real question is who is liable. Obviously whoever distributes the images is liable (except for those exempted by section 230) but there’s no president for the trainers.

I think there are arguments to be made for fair use with open models run by end users where there is no profit motive, but for closed models where use is paid the trainers are potentially profiting directly from the creation of infringing work.

In the case of Machine learning the term has sort of been morphed to mean “open weights” as well as open inference/training code. To me the OSI is just being elitist and gate keeping the term.

I’ve used the tplink ones that they’re using and they’ve been pretty solid. I can’t say how they’d fare in a 24/7 setup though since they’re not really intended for that.

That’s basically only OpenAI, maybe some obscure startups as well. Mozzila is far too old and niche to get away with that anyway.