All the Inference

It is a stupid word really, "Inference" , it the word used to mean using an AI in production. Why they don't just say that I don't know.

Anyway to actually run many of the models available will almost always cost you something, be it time or money, and with both there is free pain.

Searching a bit I really could not find any free API services for these models, and I guess it is fair enough since they are not exactly light on hardware.

My main query was basically how to approach automation using AI, because then I can spam email people for money or to put me in their inheritance. This does not seem as simple as signing up and using any one thing though.

Well, not so simple if you are a cheapskate, hence the emails.

It seems what all the AI sites tend to not mention in plain language is that they are just "Managers" , all of them just orchestrate responses from ChatGPT for the most part then pipe that to this or that application.

Sure that is nice so there are many interfaces to manipulate and use responses from an AI but it still does not give you access to an AI that won't cost you a dollar every time you want to know the weather.

In that vein there are far more Agent style interfaces and projects than there are actual ones where you can just run an AI model yourself to play with.

The only two I found that are pretty the standard is LM Studio and h2O Studio

Where LM Studio would be the defacto purely based on resource usage.

I fortunately do have a solid laptop so in the next week I will play a bit with some local LLM models and get a feel for the agent orchestration possibly choosing between n8n, autogen, superagi or crew.ai ... It seems to me they do pretty much the same thing but n8n is a nice drag and drop solution like "if this then that".

As much as it would be nice to make a whole long simple to digest list of actual useful tools the ones with true flexibility need a bit of elbow grease. Which is why a local version or reliable free api to run the tools on becomes more important.

I guess for the most part hounding Google Collab for free resources is also a option albeit a hit and miss.