A Virtual Assistant for Business

The goal of the Virtual Assistant project is to build an automated assistant that is natural and intuitive to use, and that can be an alternative to a personal human assistant. One of our primary goals in developing this assistant will be to create a natural, easy to use interface that is available when and where a user needs it. To do this we will build and incorporate technologies to enable intelligent and sophisticated voice-based and multimodal interaction on a variety of mobile devices. Currently, the virtual assistant supports access by telephone, with a speech interface to daily course-of-business data and function. Function includes setting up meetings, making phone calls, taking messages, etc. Data includes message content such as email, voice mail, fax, and personal information such as calendaring, address book, and to-dos. An important component resulting from the messaging requirements is a Universal Message Store, a single repository for all communications. In parallel to this component is an interface that allows for a single mode of interaction with the message whether it is voice mail, e-mail or fax. Phone-based access is enabled through speech recognition and text-to-speech. Recognized speech is processed using natural language understanding which allows a user to speak arbitrary phrases within the domain. We currently support a working prototype with about fifty users.

The Virtual Assistant (VA) is comprised of a number of components. The first is a VA For Callers that in its initial implementation serves a simple voice mail system. The second is a set of tools and functionality that augment the desktop to easily enable access to and creation of audio messages and perhaps reading of text messages. The last and most signficant component is the VA For Subscribers (or simply Virtual Assistant, as this is the primary application), which provides the phone-based interaction with business data and function. Each is described below in more detail.

Virtual Assistant for Callers (e.g. Voice Mail): The initial VA For Callers is a basic voice mail answering system, but instead of storing the voice mail content with the server, it is stored in the user’s (i.e. person being called) primary messaging store. It is delivered to the messaging store through SMTP in a WAV attachment that can be accessed from the desktop or the virtual assistant. Information about the source of the call is displayed in the sender field so that the voice mail can be delivered as if it originated from the caller, similar to e-mail. This enables simple replies to voice-mail either at the desktop or from the Virtual Assistant. Over time, we envision other features including call-screening, find-me/follow-me and calendar-based greetings.

Desktop Tools: The desktop tools, currently built on the Notes client, will provide the ability to play audio messages (including voice mail), create, reply and forward audio messages, and intelligently play back mixed text and audio messages resulting from replies and forwards. Audio messages and voice mail will be presented in the inbox along with and distinguished from other messages including text and FAX messages.

Virtual Assistant for Subscribers: The Virtual Assistant is the application through which a user is provided phone-based access to their business data and function. The ability to access and create e-mail, voice mail, fax, calendar, personal address book and todo entries is provided. Although DTMF (i.e. touchtone) will be enabled at various places in the application, the primary mode of interaction with the virtual assistant will be through voice recognition of conversational phrases and text-to-speech. The user will be able to say such things as "Do I have any new messages?", "What is on my calendar tomorrow", "Reply to that one", "Do I have any voicemail from John Doe?". These phrases will not be specific phrases that the user must learn in order to successfully use the system, but instead will be the phrases that have occurred to the user as appropriate for the request. To enable this free-form type of interaction we will incorporate a natural language engine which can map and parse phrases into formal representations that can be interpreted by the application. The final application should provide an intuitive and easy to use interface to course-of-business data.

Virtual Assistant Demonstration:

For More Information Contact: