How to build an AI phone assistant with AWS and GPT-4

How to build an AI phone assistant with AWS and GPT-4

Why businesses want the AI phone assistant

Forget old-school phone menus: leverage an AI-powered assistant with AWS and OpenAI GPT-4

Modern businesses often struggle with outdated call management systems, forcing customers to navigate frustrating phone trees. “Lisa” is an AI-powered phone assistant fully working prototype that offers a glimpse into the future of communication.

Built using AWS services and OpenAI GPT-4. Our prototype understands callers, routes them efficiently, and simplifies interactions.

I am sure it won’t be long before we are reminiscing about the “good old days” of endless phone menu trees: press 1 for existing customers, press 3 for credit cards, press 7 to dispute a charge… and on and on!

Key benefits of an Amazon Connect AI-powered inbound phone assistant

“Lisa” is designed to address the pain points of traditional phone systems while delivering measurable benefits for businesses. Here’s what makes her stand out:

1. Simplified customer experience: no more endless phone menu options frustrations

Our AI-based working prototype eliminates the need for multi-level tedious menus. Callers can describe their needs in plain language, and AI handles the rest. This reduces customer frustration and improves call outcomes.

2. Faster and smarter call routing

Using natural language processing, “Lisa” interprets caller intent and connects them to the appropriate team or solution instantly, minimizing delays and misrouted calls.

3. Scalable and cost-effective technology for businesses

Built entirely using AWS tools like Amazon Connect, Lex, Lambda, DynamoDB, and OpenAI’s GPT-4 API, “Lisa” is a scalable and cost-effective solution suitable for businesses of all sizes.

AI-Powered Phone Assistant in less than 90 seconds

Overcoming AI phone assistant challenges

While “Lisa” demonstrates incredible potential, there are areas for refinement:

  • Latency: Response times may vary due to API and service dependencies.
  • Natural language recognition accuracy: Amazon Lex may occasionally misinterpret certain phrases or words.
  • AI limitations: Complex, ambiguous or misinterpreted requests still need a backup strategy with human intervention.

Despite these challenges, our ai-powered solution showcases how existing technology will revolutionize business communication.

The readily available technology behind “Lisa”

Our AI working prototype is powered by:

  • Amazon Connect: For managing call flows and orchestrating conversations and other AWS and external components.
  • Amazon Lex: To process caller speech and convert it into text.
  • AWS Lambda: For serverless computing tasks that glue systems together.
  • Amazon DynamoDB: For flexible storage and retrieval of the valid (and dynamic) options.
  • OpenAI GPT-4 API: For interpreting caller needs, follow up questions and responses.

These tools work seamlessly together to create a smart, automated assistant.

Solution diagram of Naonis' AI-powered inbound phone assistant

Hear “Lisa” in action

Get a glimpse of “Lisa’s” capabilities in our extensive demo video. See how she transforms real-world scenarios like CRM integration inquiries and consulting requests with follow up questions, intelligent routing and responses.

The AI-Powered Business Phone Assistant You Can Build with AWS and OpenAI GPT-4

AI Business Transformation

Key takeaways

  1. “Lisa” is a fully working proof of concept: Built entirely using accessible tools like AWS and OpenAI GPT-4, showcasing AI’s potential for smarter business communication.
  2. Improved customer experience: Eliminates frustrating traditional phone menu trees with natural language understanding.
  3. Efficiency in action: The video showcases the handling real-world scenarios, such as CRM integrations and SaaS consulting requests, with quick and intelligent routing.
  4. Scalable and flexible: The solution highlights how existing tech can transform businesses without large-scale development investments.
  5. Future ready: While not yet production-ready, “Lisa” lays the groundwork for future reliable AI-driven call management systems.
  6. Get inspired: This solution is about thinking outside the box. It is a starting point to exploring how AI and existing cloud technologies can improve your customer experience. Let’s discuss how it could work for you.

Get Started with AI-Powered Solutions

Do you have specific AI needs? Do you have an idea that you would like to experiment with? Contact us to explore how AI-powered solutions can improve your business efficiency.

Start Your AI Journey


Video transcript

Introduction: Building the AI inbound phone assistant

How can you create and prototype “Lisa,” your helpful AI-powered phone assistant that efficiently screens and directs your inbound business calls?

Using only readily available AWS services including Amazon Connect, Lex, Lambda, and DynamoDB.

For AI, we are going to use OpenAI’s GPT-4 API.

The goal is to create a helpful AI-powered phone assistant that efficiently directs inbound business calls to the right team.

We use Amazon Connect to manage the call flow and orchestrate the conversation.
Amazon Lex processes the caller speech and converts it into text.
Once we have the text, OpenAI’s API analyses the meaning of the conversation to determine how it aligns with the valid options from DynamoDB.

The system continues refining the process until it identifies the best match.

To be honest, it’s not perfect.
At times it can be slow due to services latency, Amazon Lex occasionally struggles to understand what’s being said, and the AI has its quirks.

But as a proof of concept, it’s a solid start.

“Lisa” in action

Now let’s see our assistant “Lisa” in action.
Quick note: we ran several tests.
For this demo, we mostly focused on the positive examples to show what “Lisa” can do when things click.


Call Example 1: CRM and Accounting Integration

Caller: We are looking for custom integrations between our CRM, Salesforce, and QuickBooks Online Accounting to speed up our invoicing. Do you offer this type of service?

“Lisa”: Thank you. Let’s contact AI, it will take a moment.
We will be glad to help you with system integrations with SaaS applications.


Call Example 2: Direct Contact with Enrico

Caller: Hey Lisa, thanks for taking my call. I would like to speak with Enrico if he is available. Can you help?

“Lisa”: Thank you. Let’s contact AI, it will take a moment.
We will be glad to help you with: talk directly with Enrico.


Call Example 3: HubSpot and Salesforce Integration

Caller: I would like some custom system integration between HubSpot and Salesforce. Can you help with that?

“Lisa”: Great. OK, let’s contact AI. It will be a moment.
Great, we will be glad to help you with: system integrations with SaaS applications.


Call Example 4: Tray.ai Integration Services

Caller: Hello Lisa, we have been using the integration platform called Tray.ai for a while now. I am interested in augmenting our internal team in the US with Tray integration services. Is that something you offer at Naonis?

“Lisa”: Thank you. Let’s contact AI, it will take a moment.
We will be glad to help you with: Tray implementation and consulting services.


Call Example 5: Workato Implementation Guidance

Caller: We just purchased the integration platform called Workato and we’re looking for a little bit of guidance in terms of implementation, but also on the business side. Can you help?

“Lisa”: Great. OK, let’s contact AI. It will be a moment.
Great, we will be glad to help you with: Workato implementation and consulting services.


Call Example 6: Tray.ai Consulting

Caller: I’m looking for a consultant to help with integration platforms and, in specific, using Tray. Can you help?

“Lisa”: Great. OK, let’s contact AI. It will be a moment.
Great, we will be glad to help you with: Tray implementation and consulting services.


Call Example 7: Automation and Integration Needs

Caller: Hello there, I’d like some help with integration and automation as our processes are quite inefficient, and we are losing customers because of it.

“Lisa”: Thank you. Let’s contact AI, it will take a moment.
We will be glad to help you with: system integrations with SaaS applications.


Call Example 8: Automation and CRM Duplicate Records

Caller: Hello there, I’d like some help with automation as our processes are quite inefficient, and we are losing customers because of it.

“Lisa”: [Inaudible: speeded up audio exchange]

Caller: [Inaudible: speeded up audio exchange]

“Lisa”: [Inaudible: speeded up audio exchange]

Caller: Our main CRM system is HubSpot where we have a number of duplicate records. We also use Outreach, Gong, Lusha, Zoominfo, and Chilli Piper to name a few. So, I’d say we need help with custom system integrations and automation.

“Lisa”: Thank you. Let me contact AI.

Unfortunately, we could not determine the exact reason for your call.
This is the summary of what we understood: The caller expressed a need for help with custom system integrations and automation due to inefficient processes that are causing customer loss, specifically mentioning their HubSpot CRM and duplicate records.

We are now going to transfer you through to a human.


Call Example 9: Sugar CRM Integration

Caller: Hello Lisa. I’ve seen on your website that you offer integrations between Sugar CRM and other business applications. We have the latest Sugar CRM cloud version but have limited internal technical capabilities. Can you help?

“Lisa”: Thank you. Let’s contact AI, it will take a moment.
We will be glad to help you with: Sugar CRM integrations.


Closing Remarks

Alright, that’s great! We found a matching option for this test call.
Now I am going to hang up. But before that, if you liked the solution, I’d love it if you shared it with your friends and colleagues.

Let them know what we can do for them with integrations, automation, and AI.

This solution is powered by AWS, Amazon Connect, Lambda, Lex, Polly, and OpenAI GPT’s API.

You can find more about us at naonis.tech, which is spelled: n a o n i s . t e c h

Cheers and have a great day.


Final Thoughts

There you have it, that’s “Lisa’s” prototype. It’s built entirely with off-the-shelf tools and one AI prompt. A simple yet effective approach.

While we are not quite ready to call this a production-ready solution, this experiment shows what’s possible with today’s technologies. There’s still room to refine, optimize, and adapt, but the foundation is there.

For now, “Lisa” serves as a glimpse into what could become a reliable AI-driven assistant in the near future.

Thank you for watching and stay tuned for more insights from Naonis. Feel free to reach out at naonis.tech.

AI Business Transformation

Might also be interesting

Automate sensitive data removal in AI & CRM systems securely

Automate Personal Identifiable Information (PII) Removal!
A step forward to securing Your sensitive business data.

My top 6 reasons why CRM projects still fail!

My top 6 reasons why CRM projects still fail!

40 years after CRM was invented…
and the majority of implementations fail!

Increase ROI with reusable iPaaS components for automation

Do you know what is the killer iPaaS app based on what we are doing?

All trademarks mentioned on this page are the property of their respective owners. The mention of any company, product or service does not imply their endorsement.