Simplifying MBTI question using Deep Learning with feature importance
Introduction
People really need to get some guidance for their study and career choices. MBTI gives the ability to determine which job is appropriate for each individual based on their characteristics. Standard MBTI questionnaire consists of 60 questions that will determine the personality of the person. This is not efficient because people might get bored, and give the random answer that will lead to inaccurate results.
We want to create short questionnaires that give the accurate result so the people will get results faster. We will use a deep learning approach to solve the problem.
MBTI
MBTI is test that give the prediction about personality that leads into career choices. This will give 16 personalities and each personalities will give different job recommendation.
[Give the more explanation about the test and 16 personalities]
Machine Learning
Deep Learning
Feature Importance
Collecting Dataset
We will use the free online test to predict personality for correspondence. However given that the free online test is not disclose their algorithm to predict the outcome, we will use Machine Learning to create a model that can replicate the personality prediction. We will try a lot of machine learning algorithm using Auto ML.
So first before we are doing model creation, we need to collect the dataset using python code to try permutation of question and answer automatically from the website. The algorithm is simply loop until all the result is balance.
The python skeleton looks like this
We are managed to collect 2 million data point for the experiment. The dataset looks like this.
Model Creation
Basically from Machine Learning perspective this kind of prediction is classification problem. So we use multiple classifier that included in RapidMiner to get fast result. We can use Auto Model or Auto ML in RapidMiner to brute force all machine learning algorithm for model creation.
