OpenAI’s new Realtime API lets developers add realistic conversations to their apps
OpenAI announced several new features for app developers at its DevDay conference. The company is now offering tools to integrate AI-generated voices and fine-tune GPT-4o with images.
The new “Realtime API” lets developers add six AI voices to their apps. These voices are different from those used in ChatGPT. To avoid legal issues, developers can’t use third-party voices.
OpenAI showed off a travel planning app using the Realtime API. Users could talk to an AI assistant about a London trip and get quick responses. The API can also add restaurant suggestions to maps.
The technology works for phone calls too, like placing orders. OpenAI doesn’t automatically disclose it’s an AI voice, leaving that up to developers for now.
Ad
New functions for GPT-4o and cost savings
Other Updates:
- Developers can use images to fine-tune GPT-4o
- New prompt caching to cut costs and speed up responses
- “Model distillation” to improve smaller models like GPT-4o mini
- Doubled rate limit for the new o1 model
OpenAI says its prompt caching works automatically, potentially saving up to 50% on tokens. “Stored completions” let developers save model interactions on OpenAI’s platform for later fine-tuning. The company also released new evaluation tools.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
- OpenAI has introduced new features for developers, including integrating realistic AI voices into applications and fine-tuning GPT-4o with images. The aim is to make interaction with AI systems more natural.
- The Realtime API offers six AI voices to choose from and can be integrated into applications such as travel planning apps or phone calls. OpenAI leaves it up to developers to disclose the use of AI voices.
- Other new features include immediate caching to reduce costs, model distillation to optimize smaller models, and new evaluation tools. OpenAI also doubles the rate limit for the o1 model.
