Zos_Kia

deleted by creator

They have no ability to actually reason

I’m curious about this kind of statement. “Reasoning” is not a clearly defined scientific term, in that it has a myriad different meanings depending on context.

For example, there has been science showing that LLMs cannot use “formal reasoning”, which is a branch of mathematics dedicated to proving theorems. However, the majority of humans can’t use formal reasoning. This would make humans “unable to actually reason” and therefore not Generally Intelligent.

At the other end of the spectrum, if you take a more casual definition of reasoning, for example Aristotle’s discursive reasoning, then that’s an ability LLMs definitely have. They can produce sequential movements of thought, where one proposition leads logically to another, such as answering the classic : “if humans are mortal, and Socrates is a human, is Socrates mortal ?”. They demonstrate the ability to do it beyond their training data, meaning they do encode in their weights a “world model” which they use to solve new problems absent from their training data.

Whether or not this is categorically the same as human reasoning is immaterial in this discussion. The distinct quality of human thought is a metaphysical concept which cannot be proved or disproved using the scientific method.

If I understand these things correctly, the context window only affects how much text the model can “keep in mind” at any one time. It should not affect task performance outside of this factor.

Yeh, i did some looking up in the meantime and indeed you’re gonna have a context size issue. That’s why it’s only summarizing the last few thousand characters of the text, that’s the size of its attention.

There are some models fine-tuned to 8K tokens context window, some even to 16K like this Mistral brew. If you have a GPU with 8G of VRAM you should be able to run it, using one of the quantized versions (Q4 or Q5 should be fine). Summarizing should still be reasonably good.

If 16k isn’t enough for you then that’s probably not something you can perform locally. However you can still run a larger model privately in the cloud. Hugging face for example allows you to rent GPUs by the minute and run inference on them, it should just net you a few dollars. As far as i know this approach should still be compatible with Open WebUI.

There are not that many use cases where fine tuning a local model will yield significantly better task performance.

My advice would be to choose a model with a large context window and just throw in the prompt the whole text you want summarized (which is basically what a rag would do anyway).

If you like to write, I find that story boarding with stable diffusion is definitely an improvement. The quality of the images is what it is, but they can help you map out scenes and locations, and spot visual details and cues to include in your writing.

But we are talking about freelancers, not about SEO or content marketing, more like content filling

Most SEO is done by freelancers (at least in my industry). When i talk about content marketing i mean anybody who writes blog posts and LinkedIn posts for companies. It was already shit long before AI arrived.

Yeah I’m not bashing anybody, my wife did that for a couple years I know how it is. There was a kind of golden period where it would even pay enough to let you do some quality stuff but when VC money stopped raining the market slumped almost immediately.

I think the bitter lesson here is that there’s a bunch of jobs where quality has zero importance.

If you take for example, content marketing, SEO, and ad copy writing… It’s a lot of bullshit, and it’s been filling the web with gpt-grade slop for 20 years now. If you can do the same for cheap I don’t see a reason not to.

That’s even richer. So the term AI should be reserved for the future tech that may or may not come to exist, even though that mythical technology already has a perfectly suitable name (AGI) ? That sounds… useful ! But also very interesting, and intellectually stimulating ! After all, who doesn’t love those little semantics games ?

AI is a technical term that has been used by researchers and product developers for 50 years, with a fairly consistent definition. I know it hurts because it contradicts your pedestrian opinion on how Big Words should be used, but that’s just the way it is. We’re not at a point yet where humanity recognizes your legitimacy to decide how words are used.

Do you have information that any AI company is currently money ? AFAIK all foundational models are still bleeding money and are subsidized by VC money. There is even the distinct possibility that these companies may never be profitable at the current pricing.

I don’t mean to throw shade but that explanation makes me understand even less. Yes, it is a generic term used to describe a whole array of technologies - is that a bad thing now ? I understand that some people might misunderstand if they don’t know much about the subject but isn’t that true of all technical terms ?

I don’t understand that argument. We invented a term to describe a certain technology. But you’re arguing that this term should not be used to describe such technology, as it should be reserved for another mythical tech that may or may not exist some time in the future. What exactly is your point here?

Them dusters always complaining about something smh