After looking at the overall flow of installing, building, and deploying a model with a code-based local LLM, let's now explore the wider workflow for LLM development and operations by using a different tool that lets you do this no-code as well.
LMStudio introduction
Like ollama, it is a tool that helps you install various external open-source LLMs locally and run them as a local server. The difference from ollama is that its default interface is a GUI, not a CLI.
AnythingLLM introduction
Similar services include streamlit and chainlit, but the difference is that this tool helps you build a client interface for not just LMStudio and ollama but also various individual models — without writing any code. *It runs locally; for external URL deployment a paid membership is required.
Rough order of usage
1) Install LMStudio (or ollama), download the model you want, and run a local server.
2) Install AnythingLLM, register the running local server's info, create a conversation, add context documents, and use it.
1. LMStudio
1) Installing the model. For Mistral, a Q value of 5 or higher is recommended.
2) It provides a GUI where you can test the model and check the system resource usage.
3) It also offers a feature and GUI for running a local server.
2. AnythingLLM
1) After selecting the model (LMStudio), enter the server info* — *server info: obtained by running LMStudio's local server.
2) Also pick the embedding engine and vector DB.
3) Run as a local server. On the first connection, a basic usage guide is provided.
4) You can pick a workspace from the left to start chatting. Looking at the example, the answers are not great. This is not simply a performance issue but rather because there is no context.
5) To anchor the conversation context, first upload a relevant document into the chat window.
6) To add a document mid-conversation, click the button on the left.
7) (1) When you upload a URL or file, (2) it gets added to the list above. Selecting some items (3) moves them to the right area (the chat workspace name), and (4) pressing the embedding button applies that data to the workspace.
8) Unlike before the context was set, it now gives proper answers.
9) When the answer references a URL or document, the reference path is also provided.
Before using them, just based on a quick search, I assumed LMStudio and AnythingLLM were each in competing positions. But once I actually used them, it feels like they create real synergy when paired as a set. As a no-code way to 'just try out an LLM,' I'd say this is probably the best tool around right now.
...
(From here on, TMI warning zone.)
Hmm... but actually...
This market — or scene — moves so fast that even though LMStudio and AnythingLLM are plenty good, offer rich features in detail, and have only been out a short while... personally, what feels important right now is that how recently a tool launched or got updated, and how convenient its interface is, may actually matter less than you'd think.
In my purely subjective view, LLMs seem to be crossing the uncanny valley (image source, Mori (1970)). From that angle, the part to focus on now is that LLM rankings or individual model performance have already leveled out near the top — or at least crossed the necessary threshold.
Of course, things will keep evolving and I hope they do, but now that the basic threshold has been crossed, what really matters is how well you build agentic — basically personalization — capabilities. From this point on, I'd argue, we're really in Mash Up territory. In that sense, what catches my eye is no-code tooling that lets you set up not just the basic LLM but individual agents on top — Microsoft's Autogen Studio, autogen studio .
AutoGen Studio 2.0: Revolutionizing AI Agents
Unveiling AutoGen Studio 2.0 Microsoft's AutoGen Studio 2.0, a significant advancement in AI tools, equips users to design and control AI agents and workflows. The platform's intuitive interface and extensive toolset make it a leading choice in AI developm
autogen-studio.com
Lastly, here's a link to one of those classic TED talks I personally find striking.
https://www.ted.com/talks/lisa_gansky_the_future_of_business_is_the_mesh?language=ko
Lisa Gansky: The future of business is the "Mesh."
At TED@MotorCity, "The Mesh" author Lisa Gansky talks about the future of business: it's about smart or technology-enabled rentals, or — more boldly — P2P, that is, person-to-person connections via the internet.
www.ted.com
