VARK and Validity
Researchers may want information on the statistical validity of the VARK Questionnaire. Dr. Walter Leite from the Research and Evaluation Methodology program at the University of Florida has conducted a study of VARK. The paper provides evidence of the validity of the VARK for measuring learning preferences and also presents its limitations. The paper is published and the citation is:
Leite, W. L., Svinicki, M. & Shi, Y. (2010). Attempted Validation of the Scores of the VARK: Learning Styles Inventory With Multitrait-Multimethod Confirmatory Factor Analysis Models. Educational and Psychological Measurement. 70, 323-339.
The link for the article is: http://epm.sagepub.com/cgi/content/abstract/70/2/323
The abstract reads:
Validity: The authors examined the dimensionality of the VARK learning styles inventory. The VARK measures four perceptual preferences: visual (V), aural (A), read/write (R), and kinesthetic (K). VARK questions can be viewed as testlets because respondents can select multiple items within a question. The correlations between items within testlets are a type of method effect. Four multitrait-multimethod confirmatory factor analysis models were compared to evaluate the dimensionality of the VARK. The correlated trait-correlated method model had the best fit to the VARK scores. The estimated reliability coefficients were adequate. The study found preliminary support for the validity of the VARK scores. Potential problems related to item wording and the scale’s scoring algorithm were identified, and cautions with respect to using the VARK with research were raised.
Reliability: They also explain that Cronbach’s alpha would underestimate the reliability of the VARK scores, because Cronbach’s alpha assumes that all items are parallel measures of the construct, which is not true with the VARK. Therefore, they provide estimates of reliability based on confirmatory factor analysis. The reliability estimates for the scores of the VARK subscales were .85, .82, .84, and .77 for the visual, aural, read/write, and kinesthetic subscales, respectively, which are considered adequate given that the VARK is not used for high-stakes decisions.
Most of the data (below) is from the VARK database June to December, 2017.
NOTE: Where a statistic refers to a question, please use the downloadable VARK Questionnaire from the website. The questions and their options are randomized when presented online on the VARK website so question numbers on your copy may not correspond with those below. You may request data from our database for research purposes. Over one million online users complete the VARK questionnaire annually. One third of those, leave demographic data about themselves.
WHAT DOES VARK INDICATE?
Before we analyze the results from the VARK database it is necessary to examine the shape and structure of the questionnaire so that the correct statistical techniques can be used.
VARK is not a fully-fledged learning style. The words learning style are often loosely used to describe almost any attribute or characteristic about learning but technically the term refers to all the components that might affect a person’s preferences to learn. Some inventories report on a number of components in a style (such as motivation, surface-deep approaches to learning, as well as social, physical and environmental elements) and some personality inventories have learning characteristics as a part of their wider descriptions.
VARK deals with only one dimension of the complex amalgam of preferences that make up a person’s learning style. The VARK questions and their results focus on the ways in which people like information to come to them and the ways in which they prefer to deliver their communication. The questions are based on situations where there are choices and decisions about how that communication might take place.
It is important to say what VARK is not, so that other components are not perceived as being a part of VARK. VARK has little to say about personality, motivation, social preferences, physical environments, or intraversion-extraversion. The choice to limit VARK to modal preferences was made because that is where Neil Fleming had most success in assisting learner. Of course, changing the other dimensions affects learning, but it was the modal preferences that had the most direct application for more effective learning and from which learners said they gained most help.
THE RATIONALE FOR MULTIPLE CHOICES.
Multimodality was the expectation in the design of the questions. There are no correct answers. The modal preferences of people are seldom singular and we live in a multimodal world. In the majority of cases people will have preferences for a number of modes and they will use strategies associated with their preferences depending on the context or situation. For example they may choose a Read/write response because the situation is biased towards printed matter. Intuitively this makes sense, as we seldom act on the basis of input or output from only one mode. For that reason, multimodality (bi-, tri- or quad-) is likely to be the “normal” condition and single-mode preferences are likely to be less common. Those who have a mild, strong or very strong preference for a single mode are still multimodal because they will have three other scores. it is just that one of their preferences is a little stronger than the others. For example a person with VARK scores of 6 3 3 3 is said to have a single preference for VARK’s Visual but is, in fact, still multimodal, though not categorized as such by the VARK algorithm. Some modes, notably Kinesthetic, is itself, an amalgam of senses and could be said to be multimodal in the broadest sense of that word. For VARK usage it has a specific definition that should be understood if you plan to use VARK.
If multimodality is the expectation in life situations, we should allow for it in the structure of the VARK questionnaire and that is why respondents can choose more than one answer to each question. But clearly if everyone chose every answer for every question then VARK would provide few insights into their strategies for learning. Allowing for multiple choices, however, reduces the discrimination of VARK. So on one hand we say that multimodality is the norm, but on the other hand we are really interested in the relative strengths of particular preferences within individuals. It is the ability of VARK to allow multiple choices, yet point out a person’s preferences in their profile of four scores, that is its strength.
If the database indicated that respondents’ choices were distributed evenly across all options then it is likely that the questionnaire would provide less discriminatory information for its respondents – most would be all-four – V, A, R, and K. The options to each question are designed so that those with a particularly strong preference will choose the response that matches that preference even when the situation in the question stem is biased towards another mode. That is how VARK discriminates and for that reason the proportion of respondents choosing each option in a question is unlikely to be close to 25% for each question. It is more likely that one or sometimes two options in each question will be very attractive to most and that only those with a strong preference will choose a different option aligned with their modal preference(s).
Those who have a single-preference may continue to choose the weakest options despite the attraction of the dominant option (see later). An uneven distribution across the options is expected. Table One shows this feature in the proportions (percentages) for each question taken from the June to December 2017 database (n=176053).
Note: The table was calculated using all the choices (except VARK – all four) for all respondents for each question. Data on those who chose all four options for a question was excluded because it would only inflate the figures (below) by the same amount. Many respondents chose more than one option for some questions hence the excess over 100% for the Total column. For 7 of the 16 questions there is at least one mode with more than half the respondents. Visual (5) has the lowest percentage of choices in the 16 questions. Read/write and Aural have the lowest in four questions and Kinesthetic in none of the questions. Conversely, Aural and Read/write have four questions where they are the most popular choice, Kinesthetic has five and Visual has three.
TABLE ONE: Percentage choosing each option.
The table includes double counting because of the opportunity to choose more than one option in any of the 16 questions.
Percentage who chose this option as all, or part, of their answer.
Most popular option
Least popular Option
R and K
Excludes those who chose all four options to any question.
THE VARK PREFERENCES
As in life, VARK allows for multiple approaches and strategies for learning. Most learning takes place in an environment of multiple modes and it is probably impossible to learn or teach using one mode only. Multimodality is certainly the norm. This is similar to saying that everyone has a multimodal profile with some V, some A, some R and some K but within their profile, some may have stronger preferences for some mode(s). There are a number of ways to assemble the VARK preferences. The usual method is to distinguish 23 profiles as shown below.
TABLE TWO: THE VARK PREFERENCES
|Single preferences||Visual – Mild, Strong and Very Strong.|
Aural – Mild, Strong and Very Strong.
Read/write – Mild, Strong and Very Strong.
Kinesthetic – Mild, Strong and Very Strong.
|Bi-modal preferences||VA, VR, VK, AR, AK, RK.||6|
|Tri-modal preferences||VAR, VAK, ARK, VRK.||4|
|All four modes||VARK||1|
- Recently we have been subdividing those in the all-four VARK Profile (above) into three segments. Those in Type One tend to use their preferences separately. They examine the situation and choose the preference that suits it. They could be described as “context specific” or “flexers”! Others (Type Two) need to use all their preferences to get an understanding that suits their learning. It could be said that they are “context blind“. Although they take longer to “understand” something new, their understanding is deeper and they have more, and wider, perspectives. The graph below shows the proportions who are in Type One and Type Two and a smaller group who lie in the transition area between the two. Note that this distinction between Type One and Type Two would also exist in the Bimodal and the Trimodal profiles but, for clarity, we have not added them into this graph. Using this categorization would mean there are 25 different profiles generated by the VARK algorithm.
TABLE THREE: VARK Database June to December, 2017: Distribution of Preferences
VARK Type One
VARK Type Two
This table shows that, for example, there is some Visual in several profiles – it is evident in the three single Visual preferences Mild, Strong and Very Strong, and in partnership with other modes in VA, VR, VK, VAR, VAK, VRK and in VARK. Each mode is therefore represented in 10 different profiles, seven of which are overlapping with other modes. Those with a Multimodal set of preferences total 63.7%.
SO WHAT IS NORMAL?
The VARK database samples populations largely dominated by those in education (>80%) so it is not representative of a random group from a population. In the absence of a distinctive distribution, what does it mean if, say, Read/write options are chosen more often? It could be argued that the proportions in the table above, indicate biases towards those with a Read/write preference in our population, or, that the questionnaire measures what it measures, and that is all it does. But this is a circular argument. In questionnaires where only one option can be selected there is a balancing effect. Choosing one option precludes another so if one set of choices is popular, by definition there will be other less popular choices. If the VARK questions and options were rewritten to balance the proportions we would be merely reflecting an hypothesis that modal preferences are balanced within our society. The hypothetical distribution of the 6560 possible and valid sets of four scores (where at least 9 questions have been answered) is shown in the graph below.
The above statistical nicety may be an interesting phenomena but it is derived from a very contestable hypothesis.
The VARK statistics don’t help us decide, as they are a result rather than a cause. In the 2017 June to December database (n=170653), of the total options chosen, 22.0% were for Visual, 25.8% were for Aural, 23.7% were for Read/write and 28.0% were for Kinesthetic.
The content or face validity of VARK is the best source for resolving the argument about what is normal (see above). The strength of VARK is that the questions and options are drawn from real-life, learning situations and that people identify with the results that they receive. That is VARK’s strength. If a large proportion of people found that the questionnaire gave them results different from their own perceptions, or the perceptions of those close to them, that would be a reason to re-examine or reject some questions, or the whole questionnaire. The strength of VARK is shown by the increasing number of respondents who use it and who comment on its usefulness, and, the percentage of respondents who indicate that their VARK results match what they perceive as their learning preferences. In June to the end of December, 2017 this “Match” statistic was 69.5% of respondents who answered that question (n=100902) and the “No Match” was 4.0%. The remaining respondents (26.5%) chose “Don’t Know” and of that group, 72% were aged 12-25 years of age. It is not surprising that the younger respondents say that they don’t know how they learn? Of the “no Match group 71.4 were in that same younger age group.
If we really wanted to balance the results for V, A, R and K we should search for, or create a number of additional questions so that V, K and A are the dominant options more often! In our five-yearly reviews of the questionnaire (last completed in October 2017 although we improved some questions, not much changed! As a consequence there is no distinctive distribution of VARK scores and no typical VARK profile for the general population. What we can state is that the average scores for each mode (V, A, R and K) are 5.6, 6.6, 6.1 and 7.3. The graph below, for the respondents from June to December, 2017, shows the proportions for the major profiles.
REDUCING THE COMPLEXITY
It may be useful to identify the statistics for VARK data reduced to only four modes. This can be done in two ways shown in the graph that follows and in Table Four below. For the graph, we have collected data from those who have some Visual preference, all those with some Aural preference, some Read/write preference and those who have some Kinesthetic preference. For each of these we have shown those who have a single preference and in the next vertical column of the bars those who have some of each preference in a bimodal profile and finally all those who have some of that preference in a trimodal profile. Those who have some of each preference in a four-part VARK profile have been omitted as that would only add the same equal data and make no distinction to each of the bars.
To interpret the graph, for the respondents in the June to December, 2017, data, 37.5% of those respondents had Kinesthetic as some part of the description of their profiles i.e. they either had a Kinesthetic single preference from the VARK algorithm or Kinesthetic was part of their bimodal or trimodal profiles viz. AK, RK, VK, VAK, ARK or VRK.
TABLE FOUR: Groups and the percentage of the V, A, R and K. Options Chosen. June to December 2017.
This table has the percentages for the total number of options chosen for each of the four modes for various populations. The four statistics will sum to 100%. For example, 62113 Females, chose 22.3% of their options from the Visual choices, 24.6% from the Aural options, etc.
|Used VARK Before||22.8||24.5||24.1||28.7||8420|
|First Time User||22.4||25.1||23.8||28.7||93769|
|Age under 18||22.9||25.0||23.1||29.0||11946|
|Aged 19 to 25||22.4||25.1||23.3||29.3||23187|
|Aged 26 to 34||22.1||24.3||25.3||28.3||10684|
|Aged 35 to 44||21.5||23.7||26.9||27.9||7469|
|Aged 45 to 54||21.0||24.2||27.6||27.2||4849|
|Matches my perception||22.8||24.4||24.0||28.9||70073|
|Does not match my perception||21.4||26.4||24.2||28.0||4062|
|Don’t Know How I Learn||21.5||26.3||23.6||28.5||26767|
THE MATCH AND NO MATCH STATISTICS
In the tables below are the statistics for a question asked of all respondents after they have completed the questionnaire and have seen their results. They are asked if their results match their perception of how they learn. They have three choices: “Match“, “No Match” and “Don’t Know.” The “Match” statistic is currently 69.4% and the “No Match” statistic is 4.0% so a further 26.5% are in the “Don’t Know” category. We use this statistic as a regular check on whether VARK is offering a useful service. If a much larger proportion chose “No Match” we would be concerned. Some of the 100902 who responded to this question (June- December 31 2017) claimed that they did not know how they learned and their data is shown in the tables below with the highest group first.
TABLE SIX: “Don’t Know” Statistics
|VARK Category||Percent of “Don’t Know”|
|Percent of this VARK Profile in the Total Database|
|VARK (All modes)||42.0%||35.0%|
TABLE SEVEN: Age of the “Don’t Know” group
Of the Don’t Know group 73.2% were aged under 26.
|Age||Percent of “Don’t Know” Database|
|19 – 25||31.9%|
|26 – 34||12.7%|
|35 – 44||8.4%|
|45 – 54||4.7%|
Sixty percent (60.3%) were females which is similar to the database total.
TABLE EIGHT: “No Match” Statistics
A total of 4039 respondents chose this category, making up 4% of those who answered that question (n=100395). They were mostly in these VARK Profiles.
|VARK Category||Percent of “No Match” Database||Percent of Total Database|
THE DISTRIBUTION OF V, A, R AND K SCORES
The VARK website Standard algorithm calculates each respondent’s Profile based on their V, A, R and K scores from the questionnaire. The scores for each individual and mode vary from zero (0) to 16. The frequency of each V, A, R and K score for all respondents in June 1 to December 31 2017 (n=176053 is shown in the graphs below.
NUMBER OF OPTIONS CHOSEN
VARK has 16 questions with four options for each question so each of the four modes (V, A, R and K) can be selected 16 times. Because respondents may choose more than one answer for each of the questions the possible total number of answers for any single respondent is 64 (16 V, 16 A, 16 R and 16 K). For a valid entry in the database the minimum number of questions attempted has been set at 12. The most common number (mode) of options chosen was 16. (Note: not necessarily one per question.)
Questions 5 and 11 were the most difficult, (or inappropriate for some respondents to decide as they were the ones most often left blank.
THE APPROPRIATENESS OF THE OPTIONS.
In testing the latest version it was important to know which options were “working” and which were not. One possibility was to design options so that each would attract significant numbers of respondents as discussed above. This could have led to almost equal proportions of respondents opting for each choice – i.e. for each question there would be equal numbers of respondents choosing each option. Because VARK allows multiple answers to each question and because we wanted the questionnaire to discriminate between preferences the proportions vary. To test the questions we collect statistics on the percentage of respondents who chose an option that was included in their final profile description. For example, if all respondents with a single Read/write preference and all those who had Read/write as a part of their multimodal preference chose the Read/write option for a question, that would provide a 100% statistic. We called this test Loyalty. This also told us who was choosing the weaker options. If the weakest option was still being selected by those who had some preference for the mode represented in their VARK Profile, we were confident that the questionnaire and its options were working appropriately. The alternative hypothesis that a large proportion who had no preference for a particular mode were choosing that option would indicate that the option was wrongly worded or poorly selected.
Table Nine indicates the LOYALTY percentages.
TABLE NINE: LOYALTY Percentages
n=102350, June to December 2017
|Question Number||% with some V in their profile who chose a V option||% with some A in their profile who chose an A option||% with some R in their profile who chose an R option||% with some K in their profile who chose a K option.|
To read this table: For Question One, 50% of the respondents who chose the Visual option had some Visual in their final VARK Profile i.e. they were categorized as having a single preference Visual (mild, strong or very strong) or had a bimodal, trimodal or four-part preference with Visual as a component of it. (e.g. VA or VRK or VARK etc.) The higher the number the stronger the option was for those with that preference and the better the option. Where the statistics are strong across all four modes the better the question.
TABLE TEN: SINGLE PREFERENCE LOYALTY
N=102350, June to December 2017
|Question Number||% of Single Preference Visual respondents who chose a V option||% of Single Preference Aural respondents who chose an A option||% of Single Preference Read/write respondents who chose an R option||% of Single Preference Kinesthetic respondents who chose a K option.|
To read this table: For Question One, 63% of the respondents who had a single (mild, strong or very strong) Visual preference chose the Visual option in their response to this question. The higher the number indicates the option was preferred by those with that preference.
TABLE ELEVEN: Data for Other Languages
Only those with numbers greater than 1000 using the VARK site during the most recent period are shown. The percentage who chose the options (V, A R and K) has been rounded.