Genie : a data-efficient platform for programmable, privacy-preserving virtual assistants
Abstract/Contents
- Abstract
- In recent years, virtual assistants have entered millions of homes worldwide. Yet, the current technology for virtual assistants is limited: to fully realize the promise of virtual assistants, we need to make development affordable and respect user privacy. This will allow supporting use cases beyond the common ones, and unblock sensitive applications like health care and finance. This thesis introduces a novel way to develop conversational agents and virtual assistants, with a lower data acquisition cost and better support for privacy. The methodology is based on an extensible programming language as an intermediate representation, the use of training data synthesis instead of expensive manual annotation, and a distributed architecture with communicating assistants. To represent the user's intent, we propose ThingTalk, a novel programming language expressing what the assistant can do. Natural language is translated to ThingTalk code directly using a contextual neural semantic parser. ThingTalk is designed to improve translation and to make it easy to acquire training data. The contextual semantic parser, which encodes a succinct formal representation of the dialogue, improves the robustness of dialogue agents compared to existing dialogue tree approaches. ThingTalk is more precise than traditional intent and slot representations, and can represent 98% of a reannotated MultiWOZ test set. Additionally, the design of ThingTalk, based on separating domain-independent constructs from domain-specific primitives, improves the extensibility to new domains. To overcome the cost of annotating data, we propose to train a contextual semantic parsing model on data automatically obtained from an abstract state machine and a library of domain-independent templates. The model is then fine-tuned on a small amount of manually annotated data. On the challenging MultiWOZ dataset, we find our model and training strategy yield a new state-of-the-art turn-by-turn accuracy of 78%, with only 2% of the typical amount of manually annotated training data. From our results, we have developed a set of tools, called Genie, to help developers build conversational agents at a lower cost using data synthesis. This thesis also demonstrates how ThingTalk and the synthesis of training data can be used to build a virtual assistant with programming capabilities beyond the simple commands supported by common assistants. We propose Almond, the first assistant to support compositional trigger-action commands. A prototype of Almond was well-received in initial user studies, and we also show that our training strategy with synthesized data improves the accuracy of the Almond semantic parser by 14% over a state-of-the-art baseline. Lastly, to address the privacy problem stemming from the centralized architecture adopted by commercial assistants, we propose a novel privacy-preserving architecture where the public skill repository is separate from the personal assistant; the latter can run on a device of the user's choice. Sharing of data between different assistants is possible using a distributed protocol where assistants exchange programs expressed in ThingTalk. Fine-grained access control on each request is expressed in natural language and enforced securely using satisfiability modulo theories. Our experiments suggest that fine-grained access control is useful and that our system can support 85% of use cases collected from crowdworkers.
Description
| Type of resource | text |
|---|---|
| Form | electronic resource; remote; computer; online resource |
| Extent | 1 online resource. |
| Place | California |
| Place | [Stanford, California] |
| Publisher | [Stanford University] |
| Copyright date | 2021; ©2021 |
| Publication date | 2021; 2021 |
| Issuance | monographic |
| Language | English |
Creators/Contributors
| Author | Campagna, Giovanni |
|---|---|
| Degree supervisor | Lam, Monica S |
| Thesis advisor | Lam, Monica S |
| Thesis advisor | Landay, James A, 1967- |
| Thesis advisor | Manning, Christopher D |
| Degree committee member | Landay, James A, 1967- |
| Degree committee member | Manning, Christopher D |
| Associated with | Stanford University, Computer Science Department |
Subjects
| Genre | Theses |
|---|---|
| Genre | Text |
Bibliographic information
| Statement of responsibility | Giovanni Campagna. |
|---|---|
| Note | Submitted to the Computer Science Department. |
| Thesis | Thesis Ph.D. Stanford University 2021. |
| Location | https://purl.stanford.edu/xf984xk2938 |
Access conditions
- Copyright
- © 2021 by Giovanni Campagna
- License
- This work is licensed under a Creative Commons Attribution Share Alike 3.0 Unported license (CC BY-SA).
Also listed in
Loading usage metrics...