Chat-bots: Promise or Potential (Commerce)

Dr Dulani Jayasuriya from Accounting and Finance in the Business School at the University of Auckland developed a chat-bot to aid instructors and students to navigate a large undergraduate course. You will find this useful if teaching large cohorts, working on rethinking engagement, or have an interest in making learning more accessible.

Given the wider role chat-bots play in customer service sectors in many industries, Dulani wanted to incorporate that technology in order to digitally transform education. The chat-bots are programmed using Snatchbot software. Using machine learning and artificial intelligence allows bots to draw context from a conversation and reply in a way that is personal, engaging, and conversational.

Background

How can time and attention be re-focused to help students in need of support other than that of an administrative nature? Many staff and Graduate Teaching Assistant hours are spent answering overlapping administrative questions from multiple students. This time could be better spent on helping students engage and learn better. However, the need for increased engagement especially during social distancing and Covid-19 lockdowns and for rural students is increasing. Throughout Dulani’s years spent teaching in several countries, she noticed that due to the language barrier (students whose first language is not English), some administrative details are misunderstood, leading to lower engagement that hinders students’ progress in their studies.

Artificial Intelligence is changing the way students learn and absorb information thereby enabling educators to provide a personalized learning environment. A popular application of artificial intelligence is intelligent tutoring systems that provide a personalized learning environment to the students by analysing their responses and how they go through the learning content. Similarly, chat-bots with artificial intelligence technology can be innovative and enhance meaningful student interactions and collaboration. Dulani conducted a thorough literature survey on chat-bot use specifically for education and decided to create chat-bots to help her manage student queries and enhance interactions.

Design: challenges and process

Finally, And each task that a bot is able to handle instead of an educator allows us as educators more time to focus on our primary role: educating! Three chat-bots were developed for different purposes:

Chat-bot 1: Programmed to answer course Frequently Asked Questions (FAQ’s). To reduce the amount of time staff spend answering such questions so they can engage in more value creating activities with students. Chat-bot 1 also allows students to engage and obtain answers in real time.
Chat-bot 2: Programmed to answer specific concept-related questions commonly posed by students. To allow for deeper discussions to be directed towards teaching staff. Chat-bot 2 can highlight common mistakes made by students. This is also designed to decrease inequality among students who might not be able to attend office hours due to work commitments and to further encourage subject-related discussions among students.
Chat-bot 3: Draws from the lecture notes and provides page numbers and section headings from the lecture notes to students with visibility disabilities in large font. This particular chat-bot is also programmed to answer questions about grading and the links to the University of Auckland graduate profile. It is intended for students from diverse educational streams. disengaged students, off campus students and students with disabilities.

Figure 1. Chat-bot interface

Implementation and evaluation

The chat-bots will be implemented in Semester One in 2021 with approximately 420 students of F251, a second-year undergraduate course. Following their feedback, the plan is to test the chat-bots on around 160 students of F383, a third-year undergraduate course. After completing the proof of concept with regard to time savings and student feedback, the plan is then to propose incorporating this into Canvas, the learning management system for the University of Auckland. From a student’s perspective, this chat-bot can help them obtain answers anytime for vital course information, navigate through the course, be reminded of necessary assessment deadlines in a way that feels as if they are interacting with another individual.

What worked well?

Snatchbot was used due to its key benefits and also the facility to integrate with several key social media platforms used by students such as Whatsapp, Messenger etc. Key benefits of Snatchbot platform are:

Easy registration for free account (only e-mail validation needed)
Web based interface – no download required
Cloud based storage – no infrastructure needed
Intuitive interface for developing interactions and connections between interactions
Ability to limit responses to given options or allow free text input
Ability to embed videos and images within the responses
Ability to add delay between messages to represent a natural flow of conversation
Option to choose synonyms based on expected inputs for connections
Provision to extend conversation using NLP features
Option to export bot design for backup

What would be done differently?

The software has a limitation of pricing depending on how many queries are asked. Dr Jayasuria is in the process of applying for additional funding to expand the number of queries students can pose using this software. Key limitations of Snatchbot platform:

No provision for rich text based responses (fonts/colors/bold, etc.)
No option to display response in table layout
No visual interface for linking interactions
No option to host on-premise (if needed for privacy reasons)
Unexpected behaviour on other channels (driven by channel limitations.)

Alot more testing is needed to collect student feedback and refine the chat-bots. Keep a look out for Dulani’s upcoming paper: “To ask or not to ask? Chat-bots in Education” The University of Auckland, December 2020.

References

Crutzen, R., Peters, G.-J. Y., Portugal, S. D., Fisser, E. M., & Grolleman, J. J. (2011). An artificially intelligent chat agent that answers adolescents ́questions related to sex, drugs, and alcohol: An exploratory study. Journal of Adolescent Health, 48, 514–519. https://doi.org/10.1016/j.jadohealth.2010.09.002.
Gulz, A., Haake, M., Silvervarg, A., Sjod€ 􀳦en, B., & Veletsianos, G. (2011). Building a social conversational pedagogical agent: Design challenges and methodological approaches. Hershey: IGI Global. https://doi.org/10.4018/978-1-60960-617-6.ch006