Dialogflow V2: Unity Plugin Proposal

• Project Description
• Team Members
• Define the problem
• Address Greater Landscape
• Deliverables
• Timeline
• Documentation
• Accessibility
• Mentoring
• More about team
  • Ilana Pecis Bonder
  • Alice Sun
  • Lin Zhang
• References

Project Description

The Dialogflow V2 Unity Plugin is a continuation from Api.ai: Unity Plugin, which made it easy to integrate the Dialogflow(Api.ai) natural language processing API into your Unity project.

Team Members

• Ilana Pecis Bonder
• Alice Sun
• Lin Zhang

Define the problem

• Api.ai: Unity Plugin is no longer working in Dialogflow V2, due to the difference in the way the basic system works (Dialogflow API v2 versus v1).
• Dialogflow API V1 will be shut down in one year – on October 23rd, 2019.
• Our goal will be figuring out ways to build similar plugin that works in Dialogflow V2 system.

Address Greater Landscape

Opening new possibilities to involve voice-based conversational interaction in VR and AR environment.

Deliverables

• Feature: listening and sending voice data from Unity to the Dialogflow V2 service for recognition and processing
• Doc: show and explain how to use it (including STT process, if required)

Timeline

This project will be completed over 5 weeks (Nov 6 - Dec 11). Describe a timeline in detail below.

Week 1

• Defining project schedule and make proposal presentation
• Reaching out mentors
• Meeting with Nicole @1PM, Nov 8
• Dialogflow V2 Unity Plugin Hackathon Nov 11

Week 2

• Analyzing the authentication process in Dialogflow V2
  • Authentication via Google Cloud Service Account JSON key
  • It will give 401 error without adding the generated “client_email” in JSON file into the Dialogflow project authentication panel:

• Adding the “client_email” to "INVITE NEW PEOPLE" will give access to the Dialogflow project:

• JSON Web Token (JWT) for Google Clout Platform in Unity
• How can we automate this process?

Week 3

• Audio Request Support: Passing audio file directly via V2 API call

• Using Audio Source in Unity to capture audio from a microphone

Week 4

• Testing with a text call "hello", which received fulfillment response of "Hiiiii Unity~~~" from "Default Welcome Intent" in Dialogflow

Week 5

• Using Audio Source instead of text query
• Creating example!

Documentation

• Initial documentation including basic intro, proposal and brief process will be updated in README.md, while instructional contents will exist as a GitHub Page.

Accessibility

All documentation will be done with considerations of the W3C Web Content Accessibility Guide.

Mentoring

• Nicole He: Dialogflow Mentor
• Igal Nassima: Unity Mentor

More about team

Ilana Pecis Bonder

• Creative Technologist and UI/UX Specialist
• Interactive Telecommunications Program (ITP)
• Portfolio

Alice Sun

• Graphic & interaction designer
• Interactive Telecommunications Program (ITP)
• Blog

Lin Zhang

• Front-end developer
• Interactive Telecommunications Program (ITP)
• Blog

References

• Api.ai: Unity Plugin
• Dialogflow API v2 versus v1