There's a fundamental contention with current closed AI systems. They work best with deep context, but users are rightfully hesitant to share deeply personal data with them. Few people trust a closed AI startup with their most intimate private data, but that's how these models perform the best - by knowing you well. Additionally, giving these black-box systems control over real-world actions (like home automation or email) raises trust concerns. You better be sure the AI is aligned with you if you're using it to perform actions in the real world (e.g. a humanoid robot), not aligned with a for-profit company.
Running an EXO-style open-source agent that runs entirely on your own devices would've been impossible just a few years ago.
Today, we're excited to announce that we've implemented an open-source agent that can perform actions for you, just like Alexa:
The agent is in early alpha testing and available in the EXO Desktop app on the releases page of the exo github repo. For the TV integration, you'll need to input the IP of your WebOS-enabled-TV and for the Tapo smart plug integration, you'll need to input the username and password of your Tapo account as well as the IP address of your device.
The agent can be triggered by saying "Hey EXO" or just "EXO", then you can ask it to do things like play a YouTube video on your TV or turn on a smart plug.
Function calling is the key mechanism that allows language models to interact with the physical world. Without it, models are limited to generating text based on their training data.
Normally, a language model generates text based on its training data. For example, if you asked it to tell you the weather right now it would either hallucinate an answer based on its training data or tell you it doesn't know.
With function calling, the language model can call a function to get the weather right now.
This capability transforms a static language model into an interactive agent that can request realtime data or perform actions in the real world.
Since the model runs on your own hardware and all the software is open-source, you can choose which functions the LLM can request and what those functions do exactly.
This approach points toward a future where AI can serve as a trusted extension of ourselves—an EXOcortex that enhances our capabilities while respecting our privacy and security requirements.
We have built the first two integrations: WebOS TV and Tapo Smart Plug. We are working towards a full EXOcortex, where you can ask EXO to do anything you want and trust that it will act in a way that is aligned with you. For that, we need more integrations, so we're asking the community for suggestions for function call integrations. This could be anything from sending emails to controlling your coffee machine - we want your suggestions. Visit our Discord community to join the discussion or make a suggestion by clicking the button below:
Request a New Function Call