What tools and projects do you use for your local LLM chats and agents?
I’m trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.
What tools and projects do you use for your local LLM chats and agents?
I’m trying to sift through the mess of LLM harness and tools to find how to really get the most out of local LLMs.
The anti-AI sentiment here is ridiculous. They were asking a legitimate question!
I just use ollama locally but I keep meaning to setup OpenRouter because I like to play with larger models that my hardware can’t handle and I’d like to be able to pick the best deals 🤷♀️
AI effectively raped and pillaged all human information for the benefit of a handful of corporations. It’s use has been shown to reduce the skills of school children and even deskill those with established knowledge. Then there’s the environmental considerations in a time where the world is decades behind appropriate climate change management… Id say it’s pretty understandable that it evokes passionate responses.
I’m sure OPs RTX 3060 will have a tolerable effect on the school children and the rainforest.
@FauxLiving @YeahToast “I’m sure my driving to work will doom the world” same mentality.
Also, it’s not so much use of a model as the creation of the model. Training is expensive in terms of resources and energy. Using the model and normalizing AI usage leads to more AI usage and more AI normalization, which leads to new models being trained…
Put it into perspective:
People driving cars pollutes more as does global shipping, regular old consumerism, etc. The environmental impact of data centers is negligible in the grand scheme of things.
Now that AI is making open source software the most secure software ever written (the hard way, hehe), not using it to scan your stuff is now considered worst practices and basically idiotic negligence. No company that uses software or anything with firmware can afford to ignore it anymore.
There’s no way out of it. Accept the new AI reality and push for good regulations. Example: If we made it so that all AI data centers had to be powered via renewable energy on site that would allay most of the energy use/environmental concerns.
Isn’t ollama made by Meta?
I think you’re thinking of the Llama family of llm models which are owned/made by Meta.
Ollama is an open source server software and model repository for llm models. I won’t say Meta hasn’t contributed to the project, but as far as I know they don’t own it and didn’t start it.
Oh, it could be. I don’t use any online or offline AI tools. I’ve played around with Lumo (from Proton) when it first released bc its/was Mistral for privacy, and work tries to get people on board with AI usage (Copilot).