Popular on s4story
- One Man's Harsh Quest for Redemption in Britain's Post-Apocalyptic Wasteland: New Thriller Out Now - 104
- Resident Inspect Joins Property Meld Nexus Network with API Integration
- RAATV Premieres Original Reality Series "The Access Index: Jackson" June, 19
- RAS AP Consulting Advances to RFP Stage in Heidelberg Materials' SAP Vendor & Customer Master Data Modernization Initiative
- Umbrella Becomes First FinOps Platform to Support AWS Billing Transfer Onboarding
- HRC Fertility's Dr. Christo G. Zouves Appointed to San Mateo County Medical Association Board of Directors
- Virginia Moving Company Nearly Doubles Customer Calls in Two Weeks After Switching to CARL — the Bold New Alternative to WordPress
- Presidential Acceleration of Psychedelic Therapies Enters a Defining Moment as Federal Policy, FDA Alignment & Breakthrough Neurotechnology Converge
- "Fearless and Free": Long Beach Pride 2026 Celebrates Resilience, Family, and Multicultural Connection
- As Graduation Season Approaches, 'Find Your Gold Thread' Helps Students Align Career with God's Calling
Similar on s4story
- The AI Direction Deficit: TripleTen Study Finds Staff Get Told to Use AI — But Not Trained to Use It
- All About Technology Celebrates 25 Years of Bridging Detroit's Digital Divide
- iatroX surpasses 500,000 clinical queries and expands specialist exam coverage
- MSBG Corporation Acquires GridWatch US Telemetry Automation System
- TAYP Expands Athlete Exposure Platform Beyond Georgia With New Push Into Virginia and the 757
- The Millennium Alliance Appoints Former Adweek Executive Eric Hayden Shakun as Chief Financial Officer to Accelerate Next Phase of Growth
- $4.8M in Contracted AI Revenue with Projections of $30M Over 6-12 Months for Diversified AI Software and Platform-Based Services Provider XMax Inc
- Larry R. Wasion's Jump Gate III RoadMaker Blends Cutting-Edge Sci-Fi with High-Stakes Space Exploration and Complex Technologies
- SpeedyIndex Rolls Out Automated API for Mass URL Verification, Solving the Backlink Blind Spot for SEO Agencies
- DLT Resolution, Inc. (Stock Symbol: DLTI) Expands Into the $224 Billion Life Settlements Market While Accelerating Telecom Growth Across Canada
Stream Releases Open-Source AI Agent That Reads Your Face and Adapts How It Speaks
S For Story/10692670
BOULDER, Colo. - s4story -- Built on Vision Agents with Anam and Inworld to demonstrate emotionally aware, video-first AI
Stream released an open-source AI agent that responds to a user's facial expressions, gaze, and engagement in real time. The agent, called Crashout Buddy, is live at visionagents.ai.
The era of the floating orb is over. Most voice agents today are blind. They convert speech to text, run it through an LLM, and read the response back in a flat tone regardless of whether the user is laughing, frustrated, or close to tears. Built on Stream's Vision Agents framework in collaboration with Anam and Inworld, Crashout Buddy watches the user's face and shapes both what the agent says and how it says it. When the user goes quiet, it notices. When they look like they're about to lose it, it softens.
How It Works
The agent runs a multimodal perception stack on Stream's global edge network. MediaPipe tracks 52 facial blendshapes at 8 fps to classify emotion, gaze, and engagement. That signal is injected into the LLM (Gemini) on every turn, which steers Inworld's TTS-2 voice model using natural-language direction such as [say warmly with light, easy energy]. Anam renders a photorealistic, lip-synced avatar. Deepgram handles speech-to-text.
More on S For Story
The same pattern (facial state, rich agent context, expressive voice, lip-synced avatar) suits apps in dating, coaching, recruitment, tutoring, and customer support.
Key capabilities include:
Availability
The full project is open source. Try the demo at visionagents.ai, read the guide on the Stream blog, or explore the code at: https://github.com/GetStream/Vision-Agents
Stream released an open-source AI agent that responds to a user's facial expressions, gaze, and engagement in real time. The agent, called Crashout Buddy, is live at visionagents.ai.
The era of the floating orb is over. Most voice agents today are blind. They convert speech to text, run it through an LLM, and read the response back in a flat tone regardless of whether the user is laughing, frustrated, or close to tears. Built on Stream's Vision Agents framework in collaboration with Anam and Inworld, Crashout Buddy watches the user's face and shapes both what the agent says and how it says it. When the user goes quiet, it notices. When they look like they're about to lose it, it softens.
How It Works
The agent runs a multimodal perception stack on Stream's global edge network. MediaPipe tracks 52 facial blendshapes at 8 fps to classify emotion, gaze, and engagement. That signal is injected into the LLM (Gemini) on every turn, which steers Inworld's TTS-2 voice model using natural-language direction such as [say warmly with light, easy energy]. Anam renders a photorealistic, lip-synced avatar. Deepgram handles speech-to-text.
More on S For Story
- Janese Parker Releases Explosive Urban Fiction Novel "The Coldest Girl in D.C."
- SRK Collective Media Group Launches with a Modern Approach to Media, Authority Building, and Cultural Visibility
- MSBG Corporation Acquires GridWatch US Telemetry Automation System
- Digital Care Industry Transformation
- From Luxury Editorial To Lyrical Romance: Editor-in-chief Releases Novel
The same pattern (facial state, rich agent context, expressive voice, lip-synced avatar) suits apps in dating, coaching, recruitment, tutoring, and customer support.
Key capabilities include:
- Emotion, gaze, and engagement classification with hysteresis to prevent flicker
- Natural-language voice steering in 100+ languages via Inworld TTS-2
- Photorealistic lip-synced avatar via Anam's CARA model
- Proactive re-engagement when the user drifts off-camera or goes quiet
- Composable processors running at independent frame rates
Availability
The full project is open source. Try the demo at visionagents.ai, read the guide on the Stream blog, or explore the code at: https://github.com/GetStream/Vision-Agents
Source: Getstream.io
0 Comments
Latest on S For Story
- The Simplest Small Business You're Probably Not Thinking About
- San Francisco Writer Wins Webby Award, Internet's Highest Honor, for Website Based on her Novel
- EDC Weekend Comedy Special Featuring Don Barnhart & Friends — Use Promo Code FRIEND for 50% Off
- N Y S E: OTH Off The Hook YS Is Building a Vertically Integrated Marine Empire — And Investors Are Starting to Notice
- Concierge Title Agency Merges with Independence Title, Inc. to Deliver an Expanded Concierge Closing Experience Across South Florida
- Grow My Security Company Launches Next-Generation Website and Expands Strategic Marketing Solutions for the Security Industry
- $4.8M in Contracted AI Revenue with Projections of $30M Over 6-12 Months for Diversified AI Software and Platform-Based Services Provider XMax Inc
- Michelangelo's Great Secret Hiding in Plain Sight
- From Blank Page to Published Book
- Virginia Marchese's Paradox: A Nation Still Deciding Who Belongs Examines Race, Migration, Law, and America's Unfinished Struggle for Equality
- Larry R. Wasion's Jump Gate III RoadMaker Blends Cutting-Edge Sci-Fi with High-Stakes Space Exploration and Complex Technologies
- New anthology collects letters to our younger selves, providing kindness we all need
- American Mensa and Davidson Institute Join Forces To Strengthen Support for Profoundly Gifted Youth
- SpeedyIndex Rolls Out Automated API for Mass URL Verification, Solving the Backlink Blind Spot for SEO Agencies
- KLEKT Announces Appointment of Jay Kimpton to Board of Directors
- Michigan Attorney General Closed FGM Licensing Investigations Months Before Federal Case Ended, Records Show
- Mensa Foundation Event Reframes Brain Health for Every Age
- New from Regal House Publishing, Into the Night Woods, a boy's heroic effort to save his best friend
- DLT Resolution, Inc. (Stock Symbol: DLTI) Expands Into the $224 Billion Life Settlements Market While Accelerating Telecom Growth Across Canada
- Indie Author Releases Dark Superhero Comedy and Suspense Tale