Building an Intelligent Goal-setting Coach: Transformer-based Evaluation and Suggestion of Written Goals
Abstract/Contents
- Abstract
- The structured practice of writing learning goals has been shown to be beneficial for both cognitive and noncognitive skill development (Evans and Hardy, 2013; Ridley et al., 2010). Furthermore, these effects are increased when the goals follow the SMART framework, meaning they are Specific, Measurable, Attainable, Relevant, and Time-bound (Lawlor, 2012). However, the implementation of such programs in schools is challenged by teachers’ lack of availability to provide feedback (Rao, 2021). Therefore, the goal of this project is to use Natural Language Processing to automate two parts of the coaching process– evaluation and suggestion– to save teachers time when implementing a goal-setting program. Firstly, for evaluation, because attainability and relevancy vary based on the writer, I focus on the other three facets, and experiment with four models to evaluate written goals for whether they are Specific, Measurable, and/or Time-bound. Specifically, I experiment with the following systems: one that uses separate vanilla RNNs with linear classification heads for each task, one that uses a single vanilla RNN base that feeds into three task-specific output heads, a system with three separate BERT models with fine-tuned linear classification heads, and finally, a single fine-tuned BERT model base with three output heads, utilizing hard parameter sharing. From these experiments, I conclude that the single BERT base model with three output heads is most advantageous because of its performance and its parameter cost savings and reduced training time. Secondly, as a proof of concept for suggestion, I utilize GPT-3 with few-shot learning and demonstrate the ability for the model to adapt goals to make them Time-bound, with a success rate of 82%. Lastly, I contribute a novel dataset of 1,000 written personal goals, each labeled for whether they are Specific, Measurable, and/or Time-bound. These three pieces hopefully add to the development of automated goal-setting coaching, to expand possibilities for program implementation in schools.
Description
| Type of resource | text |
|---|---|
| Date modified | December 5, 2022 |
| Publication date | July 22, 2022; June 3, 2022 |
Creators/Contributors
| Author | Rao, Neel |
|---|
Subjects
| Subject | Edtech |
|---|---|
| Subject | Transformers |
| Subject | Goal Setting |
| Genre | Text |
| Genre | Thesis |
Bibliographic information
Access conditions
- Use and reproduction
- User agrees that, where applicable, content will not be used to identify or to otherwise infringe the privacy or confidentiality rights of individuals. Content distributed via the Stanford Digital Repository may be subject to additional license and use restrictions applied by the depositor.
- License
- This work is licensed under a Creative Commons Zero v1.0 Universal license (CC0).
Preferred citation
- Preferred citation
- Rao, N. (2022). Building an Intelligent Goal-setting Coach: Transformer-based Evaluation and Suggestion of Written Goals. Stanford Digital Repository. Available at https://purl.stanford.edu/mf158zk6617
Collection
Master's Theses, Symbolic Systems Program, Stanford University
View other items in this collection in SearchWorksContact information
- Contact
- neelvrao@stanford.edu
Also listed in
Loading usage metrics...