PANGEA: Conversational AI for travelers

La Paz. Deep Learning. 2021

There are many digital services for travellers, but finding a specific answer can take time. That need inspired Pangea, a conversational AI web service designed to answer questions about travel destinations.

A traveler may have questions about a destination’s safety, customs, local dishes, music or places to visit. Consulting different sources takes time; the project aims to make that search easier.

To analyse text and answer questions, we use natural language processing (NLP) , a branch of Artificial intelligence (AI) which trains a computer to understand, process and generate (conversational) language.

Problem description

The problem is the time needed to find answers about a destination: food, accommodation, activities, transport, healthcare facilities, culture, music or the political situation, among other topics.

Objective

Develop, using Artificial Intelligence, a conversational question-answer web service for travelers applying NLP through the application of a deep learning model.

Techniques implemented

The techniques complementary to the resolution of the problem are presented, as any Deep Learning model requires data to be fed.

Data search

Conversational Artificial Intelligence Pangea answers questions using web text as context. To find pages related to each query, we used the Python library Google Search, which employs Google search engine as a source of information to provide the URLs of the required websites.

Data capture

Once we had the URLs, we used Web Scraping to obtain the literal content of said sites, that is, the different paragraphs and texts on each site. This technique collects web information through code. We used the library Beautiful Soup , available on PyPI.

Model selection and evaluation

Bert model: It is a bidirectional transformer encoder, which learns to interpret the language.

When additional layers are added to the Bert model and it is trained with a specialized purpose or task, a Bert model is obtained that solves a specific task.

We used models already fine-tuned for question answering with data from SQuAD, the Stanford Question Answering Dataset.

We obtained both models through Hugging Face Transformers. Each takes a question and a context to process and analyze it in order to return the answers that best fit the question.

The choice of the best model was given based on the results:

BERT in English: bert-large-uncased-whole-word-masking-finetuned-squad
Bert in Spanish (Beto): distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es

Table 1. Model Evaluation

We selected an example to compare both models’ answers to the same question, asked in their respective languages (Fig. 1).

Figure 1. Bar graph — Representation of Bert and Beto Model responses to the same question.

System Workflow

The user enters a question, and the system uses Google Search to find up to five related URLs. It then extracts paragraphs from those pages and supplies them to the model as context. The model processes the context and question and returns an answer; the answer and source URL are recorded, and both are displayed in chat message format, figure 2.

Figure 2. Own elaboration

Analysis of results (Bert vs Beto)

Pangea returns answers alongside their source URLs. In our tests, the model Beto achieved a higher proportion of correct answers. This comparison applies only to the small sample of questions used.

Conclusion and recommendations

The conclusions obtained after the development and resolution of the general objective are:

In a test of 17 questions, the Spanish model returned 64.7% correct answers, compared with approximately 30% for the English model. The questions concerned Bolivia. The team believes the availability of local information in Spanish may have contributed to the difference; the test does not establish that one model is generally better.
The Web Scraping technique correctly provided the necessary web content so that the BERT model could answer the questions adequately.
Since the service (Pangea) was deployed for a moment, the use of 2.5 GB of RAM was recorded with a delay of approximately 30 seconds while the model answers the question.
The Web Service that was offered for a few moments managed to capture the curiosity and amazement of users due to its minimalist design and interesting way of interacting.

On the other hand, the recommendations in this regard are:

The BERT model used was fine-tuned with the SQUAD dataset, which has a partially general domain question-answer format, so there are greater expectations with a specific tuning for the tourism area.
More careful source selection could improve answer quality. Google locates pages, but does not guarantee that their content is accurate or up to date.
Explore additional models to support other languages.