Centre collégial de développement de matériel didactique

3. Analysis, Limitations, and Ethics

This third and last board will help you identify both the manner in which you will analyse your data, and the limits which circusmcribe your whole study.

3.1 Types of Analysis

Listed here are a few of the more obvious choices for analysis.

Choices for data analysis are circumscribed by Decision Board 2 selections. For instance, if you chose field research, you will most likely be drawn to narrative analysis. Multiple types of analysis may be required. For instance, if you want to run an involved survey questionnaire with a representative sample, you may want to select test of statistical inference, correlation or regression and descriptive statistics as your types of analyses. Or if you are intending to do a case study on an e-business, you may need descriptive, thematic and narrative analyses.

Test of statistical inference

Descriptive and inferential are the two general types of statistical analyses in quantitative research. Descriptive includes simple calculations of central tendency (mean, median and mode), spread (quartile ranges, standard deviation and variance) and frequency distributions displayed in graphs. Inferential includes more complex calculations of statistical significance usually associated with probability-based analysis. A t-test is a typical example of inferential analysis.

Correlation/regression

Correlation measures the association between variables, usually as a numeric value signifying the degree to which changes in the values of a dependent variable (Y) increase or decrease in parallel with changes in the values of an independent variable (X). Linear regression analysis can be used to make short-range predictions, but the associations are only as strong as the arguments demonstrating their supposed relationship. Any set of values could be shown to strongly associate with another set of values, regardless of the senseless nature of the association. A typical example of a senseless (spurious) correlation is the strong association between ice cream sales and drowning deaths; another variable, hot temperatures, is actually impacting the association between ice cream sales and drowning deaths.

Descriptive

Descriptive analysis is the chameleon of research analysis: it can take on many forms, from descriptive statistical graphic displays and number summaries to involved interpretive accounts. It is concerned with the “what is” as opposed to the “why” and involves drawing conclusions, discerning patterns and assessing the meaning and implications of the data/information.

After carefully observing a social science-related phenomenon or “text” or “body of knowledge,” a plausible, well-reasoned, descriptive account of the various meanings or interpretations of the data is produced. The analysis often takes into account the context in which the data were produced, who produced it and under what circumstances. Frequent and regular references to information sources typify descriptive analysis.

Thematic

Thematic analysis is one of the most popular types of qualitative analysis. It is also easy to use. It simply involves the skilled ordering of the findings into descriptive categories or themes around which most or all of the main elements of the data results can be presented. For a qualitative study on dumpster diving, for instance, the following themes could be used as descriptive categories in which to present the rich and detailed data: SUSTAINABLE LIFESTYLE, ANTI-CAPITALIST and COMMUNAL ORIENTATION.

Narrative

Narrative analysis is a form of inquiry based on a descriptive account of a group of people such as midwives or an extraordinary individual such as Nelson Mandela or the experience of surviving cancer, drawn from a collection of narrative accounts (diaries, letters, photos, poems…).

It values the particular and the subjective, lived experience in a workplace or in an unusual circumstance such as a natural disaster. The researcher analyzes the form, content and contexts within which the story unfolds, structured either chronologically or as critical incidents. The “narrative” emerges as a rich, detailed account that is unique to the subject(s) under analysis and specific to the researcher’s investigative talents.

3.2 Limitations

The limitations in the study design should be acknowledged. Whether these limitations are due to biases emerging from studying a small number of units or limited expertise, admitting them enhances the integrity of the proposal and credibility of the proposer. It is easy to overthink these aspects and write far too much about the following items, however. It is best to focus on a few key limitations and write about them briefly.

Bias

Bias generally refers to a systematically prejudiced or unbalanced treatment of data/information. Unfortunately, no study is without bias of one sort of another. The real challenge is to recognize the forms of bias that pose the most obvious threats and to recommend measures such as inter-rater reliability to rectify them.

The most obvious forms of bias are related to:

  • sampling (e.g., non-response bias or sampling bias)
  • instrumentation (e.g., faulty question wording or non-validated scale)
  • techniques of data gathering (e.g., experimenter bias and interviewer bias)
  • analysis (e.g., interpretive bias, recall bias and citation bias)

Reliability

Reliability refers to steadfastness of measures whether it is a tape measure or a questionnaire scale. This is often associated with respondent bias, when respondents do not answer truthfully. It is especially difficult to obtain reliable results for socially undesirable behaviour such as academic failure or binge drinking. Respondents are often hesitant to divulge the real facts to interviewers, or even to themselves. If you are studying variables that are vulnerable to such reliability threats, then admit the menace and recommend adopting measures to diminish the threat (e.g., place threatening questions later in the questionnaire or phrase questions in less threatening ways).

Time

Calculating how long it will take to process your data/information is no easy task. This can be especially challenging when you do not have years of research experience as a guide. Many researchers admit to underestimating the time it takes to move a project from the planning stage to the publication stage.

Find out how many weeks and hours within those weeks are available for the assignment and then try to build your DCA decisions around that information. Confer with your teacher if you have any concerns.

A general rule is to make sure your hypothesis or thesis is as focused as possible and to have a very clear sense of what your method options are, so you can choose from several possible plans of action. It is easier to build on a modest proposal if it proves too narrow in scope than to narrow down an ambitious proposal.

Cost

The costs college students are expected to incur while conducting course-related research work are related to standard course costs such as books and paper. Exceptionally, some incidental costs may be involved, such as transportation to a location for observation or an interview.

Most college research is desk-based research, however, and requires little in the way of expenses. For this reason, budgets are not usually required for research proposals at the college level. Time estimates and resource availability are the real currencies for research proposals at the college level.                                                                    

Validity

Often confused with reliability or bias, validity is a difficult concept to understand. Validity refers to how well the overall design and techniques measure the phenomenon that you expect to find out more about. Using a survey questionnaire that asks college students what they think of gang violence is not a valid measure of whether gang violence is becoming more violent. College student opinion on gang violence is irrelevant (an invalid measure of gang violence) to the research question. A valid way of measuring gang violence is to find data and information collected by professionals and agencies that are charged with taking account of incidents of gang-related violence.

Skills/knowledge

Even if you know a lot about the topic, your research knowledge and skills may be limited in certain respects. It is customary for a novice-level researcher to admit to not fully understanding certain analyses or the subtleties of a particular theory. It is better to admit than to conceal.

Available resources

As a college student, you have access to a wide variety of college resources: the library collections, computers, software, professors, reference librarians, fellow students and learning centre specialists. You also have access to your own personal resources, including your own network of friends, family, leisure/work colleagues, government services, neighbourhood facilities and the like.

Taken together, these represent a vast store of resources that can be leveraged for your research proposal. When building your method plan, take these resources into consideration. The key here is to demonstrate that you have specific resources at your disposal to advance the plan. Admit to shortcomings in your resources and address potential ways to offset them.

3.3 Ethical dimensions

Whereas human-subject ethics is restricted to research involving direct contact with human subjects, non-human-subject ethics involves all types and forms of research, whether directly involving human subjects or not. Research cannot be conducted without due regard for authors’ rights and the fair and proper use of data/information.

Human-subject ethics

Voluntary and informed consent are the pillars of ethical treatment of human subjects involved in research. Permission forms and ethical review board approval must be obtained before experiments and surveys can be administered.

Non-human-subject ethics

As members of a community of scholars, researchers are ethically bound to uphold the fair and proper use of data/information:

  • To provide fair and accurate accounts of the data/information used
  • To credit sources using recognized professional conventions
  • To respect rights of authors and sources of information in private collections
  • To avoid overgeneralizing or exaggerating claims beyond what the data/information warrants