Case Study - Extending the use of unstructured data analysis technology (NVivo software) from qualitative researchers to the entire university community.

Abstract   

Since 2017, the author has been teaching NVivo software on campus, initially to PhD students and later to PG and UG students from SSEHS.  Originally, the focus was on teaching NVivo for conducting qualitative data analysis, which tends to be a minority in many scientific disciplines, but in 2020, the author shifted the focus to teaching NVivo for writing literature reviews (LR). This extended the value of learning this software to the entire university community, including both quantitative and qualitative researchers/students. Over the last 3-4 years, it was noted that some UG and PGT students from SSEHS using NVivo had produced high-quality literature reviews, streamlining the process and proving to be very beneficial for students with some learning difficulties, which lead the author to submit a research proposal to analyse the benefits of using this software for this new use. 

1. Background

• The author has been teaching NVivo software on campus (含羞草视频 and London) since 2017. Initially to PhD students (basic and advanced level) but later extended its teaching of NVivo fundamentals to PGT students and recently to UG students (SSEHS). 
• From 2017 to 2020 the focus of her teaching was on how to use NVivo for qualitative data analysis, main purpose of the software for which it was created, that applies to research that collects data using methods such as interviews, case studies, focus groups or ethnography. It should be noted that qualitative research remains a minority in many scientific fields.
• In 2020, as a result of the author's personal experience and feedback from PhD students (see section 5 below), the sessions focused more on teaching NVivo for literature reviews writing than for qualitative data analysis.
• Thus, the learning of this technology was potentially extended to the entire research community, quantitative and qualitative researchers/students. At this point, the author felt that a short version of the seminar could be very beneficial to PGT and UG students writing a LR for their dissertations, and some seminars were offered for the first time to PG and UG students at SSEHS, mandatory for the former and optional for the latter (in 2023-24, 166 PGT Sport Management students  attended  (module PSP114 Research Methods and Skills for Sport Managers), 80 PGT Sport and Exercise Psychology students (modules PSP510 Qualitative Research Methods and PSP003 Qualitative Research), while it was offered on a voluntary basis to 475 UG students (module PSC700700 Final Project), of which 35 attended an in-person session. 
• Over the last four years, it has been observed that some of the UG and PGT students who used NVIVO for their dissertations were able to produce high quality literature reviews. Several of them commented to the author that the process was much quicker and more efficient. It was also noted that it was very beneficial for students with learning difficulties such as dyslexia or ADHD, since some of the tutees experiencing such impairments commented on its value for their coursework. After teaching a short seminar (2-3 h), some students wondered why they had not learned it earlier in the academic year, as it would have helped them considerably for other assignments such as essays.
• As there is little academic research on the benefits of using NVivo (or any other CAQDAS software) for anything other than qualitative data analysis, a proposal was submitted to the TIA Awards at LU with the aim of gathering evidence on the benefits and setbacks of using NVivo for LRs, no matter the field or methods used. Some authors (Beekhuyzen, 2007; O'Neill et al., 2018; Rylee & Cavanagh, 2022) described the steps of the process based on personal experience or provided detailed guidelines for coding and analysing journal papers from a theoretical point of view (Bandara et al., 2015), but there has been little reflection on the benefits of using CAQDAS to conduct literature reviews based on a sufficiently large sample of participants.

2. Methodology

NVivo courses until 2020 were organised around learning the software for data analysis, with a short final section on how to apply it to LRs. From 2020 onwards, the emphasis of the courses aimed at PhD, PGT and UG students, was changed to focus from the outset on the application of NVivo for LRs, after considering that journal papers are some form of unstructured data.  Subsequently, the sessions focused on learning how to use this software for the unstructured texts found in papers and journals, applicable to all students on campus (100%) as opposed to the usual 12% of students who apply a qualitative methodology in their dissertations/research projects. Thus, all students doing a dissertation would benefit from the advantages of this software/technology to read, organise the papers, collate and evaluate the information and analyse and interpret it more efficiently and effectively. 

3. Issues

The main barrier encountered has been the lack of knowledge and reluctance on the part of a large part of the scientific community on campus regarding the value of using technology such as NVIVO, taking into account the low penetration of this type of software among staff, both those who identify themselves as qualitative researchers (many of whom continue to analyse qualitative data manually) and researchers who self-identify as quantitative, as there is a certain disdain for anything related to qualitative methodology, which is somehow considered "unscientific".  The author has encountered reluctance to support the expansion of technologies such as NVivo among colleagues and staff at all levels for different reasons: not knowing the benefits, not having used it before, not understanding how it works, seeming too complicated, not considering it necessary for UG students (when they may be those who need it most).

4. Benefits

Having personally used a mixed methods approach both in my doctoral thesis and in my current research has given me a good understanding of the limits and advantages of both quantitative and qualitative approaches, as well as the existence of some misunderstandings between researchers/staff from both "camps". This made me aware of the need to be persistent and patient in incorporating changes that bring the two approaches closer together, even more so when it comes to the introduction of technology. On the other hand, being aware of the low penetration rate of this type of technology among many staff members, has led me to explain to students that they need to be convinced of its value, as they may have to justify it (e.g. in their annual reports), as not many supervisors are familiar with the technology or do not actively recommend it.

5. Evidence of Success 

NVivo can assist the researcher in reading and interpreting a large number of journal articles and other texts (O'Neill et al., 2018) in a more effective and efficient manner, and this was reflected in the feedback from the Doctoral College workshops delivered in recent years (see comments highlighted in yellow): FEEDBACK Fundamentals of NVivo workshop feedback (Doctoral College) 2020-22.xlsx and FEEDBACK Moving On with NVivo workshop feedback (Doctoral College) 2020-23.xlsx

6. How Can Other Academics Reproduce This? 

There is a perception that NVivo is a rather complex and difficult software to learn, but my experience teaching it at various levels for qualitative data analysis (6.5 h blocks for PhD students and 3 h for PGT) has shown me that 2 or 3 h are sufficient to apply basic features of the software to LRs, if properly guided. NVivo is a software that is already available on campus and can be applied to LRs in any field, not just the social sciences, but there is a need for greater awareness. This year a project has been submitted to the TIA Awards (together with Dr Janine Coates) with the aim of gathering evidence on the benefits of using NVivo for LRs (measuring the outcome and its effectiveness), the results of which are intended to be publicised in a number of ways: creating a toolkit highlighting its benefits and linking them to current LR-related resources already available on campus, such as Study Skills Learn Module LUA001, English Language Support Learn Module LUA010鈥痑苍诲&苍产蝉辫;Dissertation and Project Success, from the Library and the Academic Language Support Service; presenting at the LU Learning and Teaching Conference; signposting resources available on the LinkedIn Learning platform, and in the future, creating short courses on this content.

7.Reflections

There have been some unexpected or collateral discoveries from teaching NVivo for LRs. Some students with dyslexia or ADHD, or even some international students for whom English was not their first language, told me that they wished they had learnt  it earlier, rather than in the last year or at the end of the semester, as it would have helped them greatly in organising their readings and collating information for other coursework, such as essays and reports, given the additional difficulty they have in handling unstructured texts. It therefore remains to be explored how to integrate NVivo with the tools already available for students with learning difficulties (or for non-native speakers) to help them with all types of coursework.

8. References

Bandara, W., Furtmueller, E., Gorbacheva, E., Miskon, S., & Beekhuyzen, J. (2015). Achieving Rigor in Literature Reviews: Insights from Qualitative Data Analysis and Tool-Support. Communications of the Association for Information Systems, 37. 

Beekhuyzen, J. (2007). Putting the Pieces of the puzzle together: Using NVivo for a Literature Review. New Zealand. 

O’Neill, M., Booth, S., & Lamb, J. (2018). Using NVivoTM for Literature Reviews: The Eight Step Pedagogy (N7+1). The Qualitative Report. 

Rylee, T. L., & Cavanagh, S. J. (2022). Using NVivo as a methodological tool for a literature review on nursing innovation: A step-by-step approach. Health Services and Outcomes Research Methodology, 22(4), 454–468.  

Thelwall, M., Nevill, T. (2021). Is research with qualitative data more prevalent and impactful now? Interviews, case studies, focus groups and ethnographies. Library and Information Science Research, 43.