Pros and Cons of using data analysis digital tools

 The use of qualitative research software to assist the data analysis process is now common. However, for someone who began research analysis using non-digital methods, it is important to sieve out the potential pitfalls of relying on software to complete the task.

This article's abstract,  written for researchers in nursing, provides a good summary of the pros and cons of using data analysis software. The distraction from the real work of analysis and the need to ensure there is depth in the analysis are the most important factors. Too often, the software encourages a quick sort of the data, rather than the many readings/iterations required in the past, to undertake manual data analysis. Some of the nuances and riches of data are lost, in the shift towards quick thematic coding. Therefore, it is always important to ensure the methodology selected for data analysis, fits well within the capabilities and potential of the digital data analysis tool. For many researchers, the institution provided tool may be the only choice apart from open source software. It is always important to match the research objectives to the types of methodology and supporting tools used to ensure the integrity of the findings.

Working with multiple data

Attended a free webinar this morning via QSR the company that sells nVivo. The topic was on public consultation research presented by Patrick O'Neil rom Lincoln University and Dr. Lyn Lavery from academic consulting. Introduced by Kate from QSR.

This is the last of a series of 3 on working with different data came about through the public consultation process Chch city council ran after Chch earthquakes. with previous webinars available on research preparation (for the unknown) and rapidly synthesising and analysing data.

The projects was focused on the Share an idea project used to inform the Chch city plan. 14thMay and series of interactions also used with all data collected by July. 2 teams involved with city council And overseas consultants. Coding structure was set up to deal with multiplicity and amount of data. Needed to ensure data collected could be collated to be meaningful. Topics set up and sub themes emerged. Important to have shared understanding of what sub themes mean to all researchers.

All data, regardless of type, coded to nodes and subthemes on nVivo. large amount of data had to be analyzed within a short time frame. Need to ensure all data was analysable. Data gathered through expo, website, council forms, workshops.

Website collected tweets (over 20,000), emails, group ideas. Council forms included letters, emails, voicemails. Expo included post it notes, pictures and drawings, focus group transcripts etc.

NVivo allowed word, excel files, PDFs audio, video and images with all could be coded to the same topics and sub themes. Plus 5 researchers could all work on data at the same time using nVivo server.

Examples of data types from expo then described.
1) Let's hear it used written information on a form - with 4 questions - what missed, what to retain, how to make Chch city better, what is most important About vision for Chch. Transcripts entered as answers to questions and then coded to sub themes.

2) Web tweets with 4 themes - move, life, space and market. Engaged people before expo and then others could see their own and others and provide feedback and stimulated more ideas. Above coded with tweeters age And suburb and tweets coded into sub themes.

3) Post it notes from expo generated 17,000. Entered into an excel file and manually coded to theme. Then text search used on excel files deployed to create nodes. Merged into existing nVivo data. 1/3 of post it notes then coded using coding density functions.

4) YouTube videos used to capture interviews up to 3 minutes with people who preferred to provide oral feedback. Video transcribed within nVivo and again coded to sub themes. Selected parts were transcribed by firstly doing a first watch and then only transcribing relevant parts.

Children's art to help children articulate their ideas. Photo of art uploaded to nVivo, regions of the 5) picture can then be coded by identifying on the picture and labeling.

Outputs also used nVivo visualization tools and can be run using queries to generate tag clouds ( word frequency with stop word list to remove unnecessary words) and word trees (to illustrate words that are searched and other things associated to each word).  Visualisations could be used to quickly display the most common ideas to inform good mass communication and quickly represent a particular topic and display a large amount of content.

All in a good opportunity to have an example of how to better use nVivo for qualitative analysis. Plus information on nVivo 10 which allows for social media data to be more easily used plus need for nVivo to be able to handle very large projects more efficiently.

Tools for analysing qualitative research data

Jane Hart is compiling a list of most useful / most used tools for research. Many of the ones listed are unfamiliar to me!! Most of these are search engines, although there are also bibliographical archiving tools. I have been trawling the databases the last couple of weeks. Basically doing the ground work for my Ako Aoteoroa funded ‘teaching craftsmanship’ research project. So it was a good opportunity to try out a few of these tools. I find Google Scholar to be a good way to source ‘key words’ & starting material which I can then use in the standard academic search database to find references pertinent to the topic I am researching. Due to the nature of the topic ‘new trades tutors & their perceptions of teaching’ I have to rely on journal articles. A few books are available on Google books but only as previews so I have had to request interloans via our helpful CPIT library.

I have also started data anaylsis, learning my way through nVivo. I need to use NVivo so that I am familiar with the software in order to be able to support other staff in using this research analysis tool. Will provide more feedback later in the year on how I feel about using nVivo to replace my usual mixture of iterative data analysis using ‘copy & paste’ & find features on Word, along with summarising, collating and reorganising themes / threads into tables. My first impressions are that nVivo imposes a way of doing (& thinking) which is slightly out of synch with how I usually approach my data. I need to have a handle on the ‘big picture’ & how other themes relate to the overall scheme of things. An article by Elaine Welsh (2002) presents both the advantages & disadvantages of using nVivo for qualitative analysis.

Helen Colley & Kim Diment’s article on ‘holistic research for holistic practice: making sense of qualitative research data’ is part of the UK’s teaching & learning research programme ‘ building research capacity’ site. Their interpretation and recommendations on qualitative data analysis does resonate with my approach to undertaking research. I am often after the ‘whole picture’ of why things happen & am constantly trying to work out how the various themes that immerge from the data ‘fits into the whole’. It is also important to view the data as a holistic ‘narrative’ rather than just focus on its atomistic parts. Telling the story which research participants try to impart via their involvement in interviews etc. is an important role of the research process. I need to distil rather than filter & at the moment, I feel a software tool like nVivo is causing me to filter & sieve rather than to distil, refine and sharpen themes. It could be because I am working solely with digital sources instead of a mixture of digital & hard copy. Will persevere for the moment with nVivo & evaluate what eventuates in a couple of months. I can always do a manual collation as well after the nVivo process to see if anything different comes through.

The TLRP site is one I constantly dip into for interesting, informative and mostly practical / applicable suggestions on doing research. The Teaching & Learning Programme itself has been a source of many articles and research reports of relevance to the work I am doing towards by PhD thesis. In particular the projects completed under the ‘further & post 16 education’, higher education, workplace learning, professional learning, lifelong learning and technology enhanced learning sections.