SAGE Research Methods Video

ANCOVA

Video Title: ANCOVA

Originally Published: 2016

Publication Date: Sep. 30, 2016

Publishing Company: SAGE Publications, Inc.

City: Thousand Oaks, United States

ISBN: 9781506359168

DOI: https://dx.doi.org/10.4135/9781506359168

(c) SAGE Publications Inc., 2017

HERSCHEL KNAPP: Welcome to practical statistics for nursing using SPSS. This video shows how to process the ANCOVA test. You can watch the entire video or use the time slider to navigate directly to any time point.

HERSCHEL KNAPP [continued]: The ANCOVA test is similar to ANOVA test. Before proceeding, it's recommended that you first view the video Ch 06 – ANOVA.mp4. In terms of setup and results, the ANCOVA test and the ANOVA test are quite similar.

HERSCHEL KNAPP [continued]: Remember that the ANOVA test assesses three or more groups to detect statistically significant differences between the pairs of groups, specifically the results of ANOVA test are based on the effect that the independent variable, the anti-hypertensive drug, had on the dependent variable, systolic blood pressure.

HERSCHEL KNAPP [continued]: The ANCOVA statistic allows us to include a potentially confounding variable into the model, which we expect may influence the dependent variable, such as smoking rate. The ANCOVA test then adjusts the results in the dependent variable accordingly.

HERSCHEL KNAPP [continued]: The ANCOVA test has two pretest criteria, homogeneity of regression slopes and homogeneity of variance, Levene's test. We'll check for homogeneity of regression slopes now. We'll see the results of the homogeneity of variance test when we run the ANOVA test. This example uses the dataset Ch 07 – Example 01 -ANCOVA.sav.

HERSCHEL KNAPP [continued]: This dataset contains three variables. Group is a categorical variable containing three values, drug A, drug B, and drug C. Systolic BP is a continuous variable that contains the systolic blood pressure of each participant at the conclusion of the study.

HERSCHEL KNAPP [continued]: And smoking is a continuous variable that contains the mean number of cigarettes that each participant smoked on a daily basis. To check for the homogeneity of regression slopes, click on Analyze, General Linear Model, Univariate. Move Group to Fixed Factors.

HERSCHEL KNAPP [continued]: Move Systolic BP to Dependent Variable. And move Smoking to Covariance. Click on Model, select Custom, move Group and Smoking to Model. Next, hold down the Shift key, and click on Group and Smoking.

HERSCHEL KNAPP [continued]: This will select both Group and Smoking together to signify the interaction term. And move them to Model. Click Continue, click OK, and it'll process. We look to the Test Between Subject Effects Table. If the P value for the group smoking interaction term

HERSCHEL KNAPP [continued]: is greater than 0.05, then this would indicate that there is no statistically significant difference in the regression slopes among the variables involved in this model, and the assumption of homogeneity of regression slopes would be satisfied. In this case, the P value is 0.028.

HERSCHEL KNAPP [continued]: Since this is less than or equal to 0.05, this indicates that there is a statistically significant difference between the regression slopes for systolic BP and smoking. This violation makes sense, as the covariate is somewhat atypical. The covariate in this model is smoking,

HERSCHEL KNAPP [continued]: the number of cigarettes that each participant smokes in a typical day. Since more than 90% of the participants are nonsmokers, this finding is not unexpected. Since this pretest criteria is not fully satisfied, this should be noted in the results section.

HERSCHEL KNAPP [continued]: To run the ANCOVA test, click on Analyze, General Linear Model, Univariate. Click on Model, select Full Factorial, click Continue, click Options, move Group to Display

SAGE (c) SAGE Publications Inc., 2017

SAGE Research Methods Video

Page 2 of 3 ANCOVA

Means 4,

HERSCHEL KNAPP [continued]: check the Compare Main Effects checkbox. In the Confidence Interval Adjustment pull down menu, select Bonferroni. In the Display Options, check Homogeneity Tests. Click Continue, click OK, and it'll process.

HERSCHEL KNAPP [continued]: To finalize the pretest checklist, we see that the homogeneity of variance test produced a P value of 0.791. Since this is greater than 0.05, this indicates that there is no statistically significant difference between the variances, hence this criteria is satisfied. Next, we look to the Test of Between Subjects Effects Table.

HERSCHEL KNAPP [continued]: The P value of 0.004 indicates that a statistically significant difference has been detected among the adjusted means for the groups. In the Estimates Table, we see the adjusted means for each group. These means have been adjusted to account for the smoking covariate.

HERSCHEL KNAPP [continued]: These figures will be useful when documenting the results. Finally, we look to the Pairwise Comparisons Table. This table is read in the same way as the Multiple Comparisons table produced by the ANCOVA post-hoc test. To identify the pairs of groups that are statistically significantly different from each other,

HERSCHEL KNAPP [continued]: we look for P values that are less than or equal to 0.05. We see that there is a statistically significant difference in the adjusted means between drug A and drug B and between drug A and drug C. This concludes this video.

SAGE (c) SAGE Publications Inc., 2017

SAGE Research Methods Video

Page 3 of 3 ANCOVA

6/28/22, 4:02 PM BUS-7105 v3: Statistics I (7103872203) – BUS-7105 v3: Statistics I (7103872203)

https://ncuone.ncu.edu/d2l/le/content/258948/printsyllabus/PrintSyllabus 1/5

Week 5

BUS-7105 v3: Statistics I (7103872203)

Hypothesis Testing: An Introduction to Various Parametric Applications

A variety of statistical tools can be used to investigate a hypothesis. These tools allow us

to compare an average score against a standard (e.g., the z-test).

Other tools allow us to compare the means of two groups. One is the independent

samples t-test that is used to explore mutually exclusive groups (e.g., treatment and control, men and women, etc.) across one dependent variable. You will have the

opportunity to conduct a t-test for the signature assignment.

A paired samples t-test (also called a t-test for related samples) compares two related

samples, or the same samples (subjects) observed at two different time points. A example

of the latter is a comparison of pre-test to post-test scores following an intervention.

Another set of statistics used for hypothesis testing includes analysis of variance (also

known as ANOVA). We use ANOVA to test the statistical significance of differences

among means of three or more groups across one dependent variable. For example, we

may wish to compare work engagement across three different age groups. For example,

we may hypothesize that people under 30 years of age (group 1) are less engaged in their

jobs than those from 31 to 50 years of age (group 2), while those over 50 years of age

(group 3) are the most engaged. ANOVA can be used to examine mean job engagement

scores across these age groups. You will have the opportunity to conduct an ANOVA test

for the signature assignment.

A repeated measures ANOVA is used to examine the evolution of a variable over several

time periods (i.e., longitudinal analysis) or more than two groups and how they differ on a

variable of interest.

Other tests that may be used to examine differences across three or more groups are:

Analysis of Covariance (ANCOVA). This test extends the ANOVA to provide a method

to control for variables extraneous to the test that may influence variance in the

dependent variable. These variables are referred to as covariates.

Multivariate Analysis of Variance (MANOVA). This test extends the ANOVA to

provide a method to include multiple dependent variables that are related.

6/28/22, 4:02 PM BUS-7105 v3: Statistics I (7103872203) – BUS-7105 v3: Statistics I (7103872203)

https://ncuone.ncu.edu/d2l/le/content/258948/printsyllabus/PrintSyllabus 2/5

Books and Resources for this Week

Cho, H.-C., & Abe, S. (2013). Is two-

tailed testing for directional research

hypotheses tests legitimate? Journal of

Business Research, 66, 1261-1266. Link

NCU School of Business Best Practice

Guide for Quantitative Research Design

and Methods in Dissertations Link

Multivariate Analysis of Covariance (MANCOVA). This test extends the ANCOVA to

examine multiple dependent variables while controlling for one or more covariates.

Each of these are omnibus tests. That is, they provide an overall test to determine

statistically significant difference among three or more groups however, they do not

specify what kind of differences exist among which groups. Post hoc comparisons are

performed to test the statistically significance of differences between group means

computed post (after) having performed the omnibus test. In SPSS, the researcher needs

to request post-hoc testing. Multiple post hoc tests are offered based on the assumption

of (substantially) equal variances (homogeneity of variance).

Last, we will discuss how we can use a tool called linear regression to make predictions

about how one variable may influence another. In Week 6, we will focus solely on the

correlation, which is an element of the regression analysis.

Be sure to review this week's resources carefully. You are expected to apply the

information from these resources when you prepare your assignments.

Reference:

Weiers, R. M. (2011). Introduction to business statistics (7th ed.). Boston, MA: Cengage Learning.

90.91 % 10 of 11 topics complete

6/28/22, 4:02 PM BUS-7105 v3: Statistics I (7103872203) – BUS-7105 v3: Statistics I (7103872203)

https://ncuone.ncu.edu/d2l/le/content/258948/printsyllabus/PrintSyllabus 3/5

Introduction to Business Statistics (7th

ed.) External Learning Tool

Knapp Ph.D., H. (Academic). (2016).

ANCOVA [Video]. SAGE Research

Methods Video Link

Knapp Ph.D., H. (Academic). (2016).

ANOVA [Streaming Video]. SAGE

Research Methods Video Link

Knapp Ph.D., H. (Academic). (2016).

ANOVA Repeated Measures [Video].

SAGE Research Methods Link

Knapp Ph.D., H. (Academic). (2016).

MANOVA [Video]. SAGE Research

Methods Video Link

Knapp, H. (Academic). (2017). Paired t-

test [Streaming video]. Retrieved from

SAGE Research Methods. Link

Knapp, H. (Academic). (2017). T-test

[Streaming video]. Retrieved from

SAGE Research Methods. Link

Strangman, L., & Knowles, E. (2012).

Improving the development of

student’s research questions and

hypotheses in an introductory business

research… Link

6/28/22, 4:02 PM BUS-7105 v3: Statistics I (7103872203) – BUS-7105 v3: Statistics I (7103872203)

https://ncuone.ncu.edu/d2l/le/content/258948/printsyllabus/PrintSyllabus 4/5

Week 5 – Assignment: Identify Analysis Tools in

Published Research Assignment

Due July 3 at 11:59 PM

This week you will locate three quantitative studies addressing a topic in your area of

specialization. At minimum, two different statistical tests should be represented.

For example, you might search the literature for studies in transformational leadership and

you may find two that used regression analysis and a third that used a t-test. For each

study:

1. State the null and alternative hypotheses (Hint. The authors will note the

alternative hypotheses, but you will have to infer the null as those aren’t typically

stated in published research)

2. Identify the statistical test used to determine statistical significance (e.g., t-test,

analysis of variance, multiple regression, etc.).

3. Identify the test statistic, note it, and explain what it means (e.g., t=3.47).

4. Identify the significance level used in each study

5. Identify whether or not the authors found support for their

hypotheses. Consider sample size and Type I and Type II error. 6. Explain the implications of each finding.

Identify whether or not the authors found support for their hypotheses.

Consider sample size and Type I and Type II error.

Explain the implications of each finding.

Length: 4 to 6 pages

References: Include a minimum of 3 scholarly resources.

Your paper should demonstrate thoughtful consideration of the ideas and concepts

presented in the course and provide new thoughts and insights relating directly to this

topic. Your response should reflect scholarly writing and current APA standards. Be sure

to adhere to Northcentral University's Academic Integrity Policy.

Upload your document and click the Submit to Dropbox button.

6/28/22, 4:02 PM BUS-7105 v3: Statistics I (7103872203) – BUS-7105 v3: Statistics I (7103872203)

https://ncuone.ncu.edu/d2l/le/content/258948/printsyllabus/PrintSyllabus 5/5

International Journal for the Scholarship of Teaching and Learning http://academics.georgiasouthern.edu/ijsotl/v6n2.html Vol. 6, No. 2 (July 2012) ISSN 1931-4744 @ Georgia Southern University 

  1

Improving the Development of Students’ Research Questions and Hypotheses In an Introductory Business Research Methods Course

Laurie Strangman University of Wisconsin-La Crosse

La Crosse, Wisconsin, USA [email protected]

Elizabeth Knowles

University of Wisconsin-La Crosse La Crosse, Wisconsin, USA

[email protected]

Abstract In an introductory research methods course, students often develop research questions and hypotheses that are vague or confusing, do not contain measurable concepts, and are too narrow in scope or vision. Because of this, the final research projects often fail to provide useful information or address the overall research problem. A Lesson Study approach was used to develop a new lesson that models the development of research questions and hypotheses and provides multiple opportunities for students to practice this skill. Two tools were also developed to help students navigate this process, and the learning outcomes of the lesson were clearly defined. To assess the effectiveness of this lesson 122 research proposals generated by student research teams before and after implementation of the new lesson were evaluated using a grading rubric based on the learning outcomes. There were statistically significant improvements in three of the five learning outcomes.

Keywords: lesson study, teaching research methods

Introduction

Many disciplines such as psychology, sociology and business include an introductory research methods class as part of the core curriculum of their program, although the approach to teaching the course can vary widely. Research methods may be taught as an exercise in addressing hypothetical situations or by studying research cases. Alternatively, portions of the research process can be isolated for students to practice; for example, students may write a research proposal or design a survey. McBurney (1995) employs a problem method where students are given a set of scenarios to analyze using a variety of research methods. Even when a project-based approach is taken, where a student completes a research project from beginning to end, there are still elements of the course curriculum that may vary. Much of the difference between approaches is dependent on how the project is started: faculty may assign the research questions to study, or students may develop the research questions themselves. Aguado (2009) guides the development of research questions by providing general topics from which to choose, and in this way sides with “control over choice” (p. 253). Alternatively, Longmore, et al, note that when students

International Journal for the Scholarship of Teaching and Learning http://academics.georgiasouthern.edu/ijsotl/v6n2.html Vol. 6, No. 2 (July 2012) ISSN 1931-4744 @ Georgia Southern University 

  2

choose their own topics, motivation and quality tend to improve (1996). However, this approach presents some unique challenges. Students may think that the choice of a topic is the same as defining the broad business or research problem and/or their projects lack focus.

At the University of Wisconsin-La Crosse, business majors are required to take an introductory research methods course entitled “Business and Economics Research and Communication”. Upon entering the course, all students have completed an introductory statistics prerequisite and the typical student is either a second semester sophomore or a first semester junior. Over the course of the semester, students complete a research project in groups by collecting and analyzing primary data. The final project is presented in both written and oral form at the end of the semester. The problems that students research are self-chosen and reflect either basic research on attitudes and behaviors or address an applied problem. For example, projects have been completed for local businesses, for governmental units, and for organizations and departments on campus. There are five broad common learning objectives associated with the course: (1) Develop the ability to define a research problem; (2) Recognize and use the appropriate techniques to collect data to address a research problem; (3) Interpret data using statistical analysis; (4) Develop the ability to effectively communicate research results both written and orally; and (5) Develop the ability to critically evaluate limitations, errors, and biases in research. How to help students achieve the first of these objectives is explored in this research.

The difficulty of teaching an introductory research methods class is well-recognized (Denham, 1997; Markham, 1991). Denham notes, “Research methods may be the most difficult course to teach at the undergraduate level” (1997). The complex nature of the course presents several challenges. It is difficult to lead students through a process of answering a question that may not be well defined and for which there are multiple research approaches (McBurney, 1995). McBurney notes that “Students tend to become anxious and sometimes dispirited when an instructor refuses to tell them the right answer” (1995). In addition to the abstract nature of research, students often have had little or no exposure to conducting research, or the thought processes involved. Markham notes that “High school work had not prepared students to think in terms of variables or hypotheses, and very few students had taken enough laboratory science or mathematics to allow much transfer of learning” (1991). Aguado notes that the deficiency in skills for conducting empirical research is present at both the undergraduate and graduate levels (2009). This lack of skills is magnified because the research process is best learned by doing.

Evidence of these challenges appears early in the semester, as students try to develop research questions that will address the overall business problem. While experienced researchers anticipate the challenges with this first step, students seem to encounter a particularly substantial hurdle in developing the initial direction of their research. Even when a research topic is chosen, they do not necessarily see how to frame their research questions and hypotheses (Ransford & Butler, 1982). When evaluating student work, we noticed that students often developed research questions and hypotheses that were vague or confusing in terms of language, did not contain measurable concepts, and were either too narrow or too broad in scope to generate valuable conclusions. Because of this weak start, data collection was often haphazard, with students realizing too late in the process that they wanted to learn something different than what the data would reveal to them. When this

International Journal for the Scholarship of Teaching and Learning http://academics.georgiasouthern.edu/ijsotl/v6n2.html Vol. 6, No. 2 (July 2012) ISSN 1931-4744 @ Georgia Southern University 

  3

occurs, the final research project provides neither useful information to address the overall research problem nor the information a decision maker requires to act upon.

There are several reasons why students struggle to produce research questions and hypotheses. First, problem definition is an abstract process. This produces a challenge for students because the mind prefers concrete knowledge (Willingham, 2009). Second, students understand new ideas and concepts by building on what they already know, specifically by seeing relationships with and making connections to knowledge they already possess. However, outside of taking surveys, students have very limited experience with research activities, so there is little about the process of defining a problem that is familiar to them. With little foundation on which to build, students often leave the classroom with a shallow understanding of the process of problem definition and knowledge that is only tied to the specific examples or context offered in class (Abrose et al, 2010).

Because their knowledge is shallow, students will have difficulty generalizing the information contained in a specific example and applying it to a completely new business problem. According to Willingham (2009), “We understand new things in the context of things we already know, and most of what we know is concrete. Thus it is difficult to comprehend abstract ideas, and difficult to apply them in new situations” (p.88). Van Gelder (2001) also notes that transfer of skills is a challenge, as skills developed in one context may not carry over to other situations. When presented with new ideas or concepts that are abstract in nature, students tend to focus on the more concrete surface details of examples without seeing the underlying structure of the problem.

To address the difficulties associated with transfer, Willingham (2009) suggests that instructors provide students with several different examples and that these examples be compared to one another. Ambrose et al. (2010) also note that “structured comparisons”, which involves comparing and contrasting different examples, problems, or scenarios, have been shown to aid in transfer (p. 110).

Once examples have been provided, students need multiple opportunities to practice using new knowledge and skills (Willingham, 2009). Practice is the only way to become proficient at any new skill, and it is practice and experience that separate the novice from the expert. Like the examples offered in class, this practice should expose students to a variety of situations. The examples need to provide students with the opportunity to practice transfer itself, by applying concepts to new contexts (van Gelder, 2001).

Ambrose et al. (2010) further suggest breaking an abstract process down into its component parts and offering students the opportunity to practice each of these component skills individually. As students become more proficient at the individual pieces of the research process this frees up space in working memory for higher level thinking. “Thus, with practice, students gain greater fluency in executing individual sub skills and will be better prepared to tackle the complexity of multiple tasks” (p. 105). While much of the literature about teaching an introductory research methods class considers the overall research process, there has been little emphasis on the first step: how to help students develop the ability to define a research problem. Yet this aspect is so crucial to the success of the students’ research project that it cannot be ignored. The purpose of this project was to create a lesson that would help improve students’ development of research questions and hypotheses. A Lesson Study approach was used as

International Journal for the Scholarship of Teaching and Learning http://academics.georgiasouthern.edu/ijsotl/v6n2.html Vol. 6, No. 2 (July 2012) ISSN 1931-4744 @ Georgia Southern University 

  4

the basis for our exploration. The process of Lesson Study involves a small group of faculty who collaborate to plan, teach, observe, revise and report on a specific class lesson. A backward design approach is used where faculty start by clarifying the goal of the learning process, and then work to design instructional experiences that achieve the goal (Cerbin & Kopp, n.d.). Emphasis is placed on making student learning visible in order to identify gaps in understanding.

Lesson Development

Defining Outcomes The first step in helping students learn how to develop their research questions and hypotheses was to more clearly define the outcomes or expectations. We collaboratively identified the six most important characteristics of a well-defined business problem in order to provide a solid foundation on which to build the research project. These characteristics are: (1) the scope or vision of the proposal encompasses the relevant variables; (2) the information is useful for decision making or addressing the overall problem; (3) the research questions are well defined; (4) the research hypotheses