I recently presented at the Nashua Cloud .NET User Group on a topic that is becoming more important for developers and architects: running AI locally.
The session, “Run AI Locally: Getting Started with Azure Foundry Local,” focused on bringing AI models closer to the user’s machine instead of relying only on cloud-hosted inference. Cloud AI is still powerful, but privacy, latency, cost, and offline access can be just as important as model capability.
That is where Foundry Local becomes interesting.
Foundry Local allows developers to run optimized AI models directly on a local device. For anyone familiar with tools like Ollama, the idea feels familiar: download a model, run it locally, and call it from an application. The difference is that Foundry Local is designed with production application scenarios in mind, including hardware-optimized ONNX models, local caching, SDK support, and integration with the Microsoft AI ecosystem.
During the session, I covered why local inference matters. Sensitive data may need to stay on the device. Some applications need faster responses without a network round trip. Other scenarios involve field work, edge devices, limited connectivity, or environments where cloud access is restricted. Local AI does not replace cloud AI, but it gives us another architecture option.
We also looked at how Foundry Local connects with Microsoft Agent Framework. Agent Framework helps developers build agents, connect tools, use structured patterns, and move from simple chat experiences to more useful AI-powered workflows.
One important point from the session was that the same agent concepts can work against a local model and later move to a cloud-hosted model when needed. Developers can prototype locally, reduce cost during development, keep certain workloads private, and still have a path to production scale.
We also covered tool calling on a local model. Real applications are not only about generating text. They need to call functions, use business logic, access data, and interact with systems. Combining local models with tools creates a more practical development pattern.
The key takeaway is this: local AI is not just a developer toy anymore. It is becoming a serious option for privacy-focused, offline-capable, cost-conscious, and edge-friendly applications. At the same time, cloud AI remains essential for scale, larger models, and enterprise deployment. The future is not only local or only cloud. It is a thoughtful combination of both.
Thank you to everyone who joined the session and continues to support the Nashua Cloud .NET User Group.
Watch the session: https://youtu.be/bUG7pR6DX4w
Presentation: https://www.slideshare.net/slideshow/run-ai-locally-foundry-local-microsoft-agent-framework-udaiappa-ramachandran/287946483
If you missed the session, join us at the next Nashua Cloud .NET User Group (NashuaUG) meetup to continue exploring practical, real-world AI engineering.