MLV Poster Peuker

Context of the Inquiry

The course used for this inquiry is ENGR A161: Engineering Practices II. The main purpose of this course is to give engineering and geomatics students an introduction to computer programing and engineering problem solving using the software MATLAB®. The learning outcomes include being able to set up the solution to a problem using engineering solving methods and to be able to code MATLAB® in order to obtain a solution to the problem. Students should also be able to present their work in a professional manner. Additional learning outcomes include understanding how engineers use computers to solve problems, and to prepare students for future use of programming and MATLAB®.

This course is currently required for mechanical engineering (38% of enrollment AY12/13), civil engineering (35%) and geomatics (7%) students, to be taken during their first year of entering the engineering program. Other students who take this course are electrical engineering (11%)and computer systems engineering (2%) students who need this course because of a previous catalog requirement, and non-engineering students (6%), including those "trying out engineering", and other majors in which understanding and knowing how to use MATLAB® is helpful. Almost two thirds of the students enrolled during the AY12/13 were in their first or second year of college. Each semester, the total number of students who take ENGR A161 ranges from 65 to 120.

The prerequisites for ENGR A161 are College Algebra or Trigonometry (MATH 107 and 108) or Pre-Calculus (MATH 109). Student taking this course are expected to be familiar with and able to use math in order to solve posed engineering problems in ENGR A161. This course is a prerequisite in the mechanical engineering curriculum and knowledge of MATLAB® is needed for multiple upper level courses.

Course Artifacts

Focus of the Inquiry

In Fall 2012, three sections of ENGR A161 were taught in a lecture hall, and using PowerPoint lectures and showing code using MATLAB®. Lectures took the entire 75 minute class period. Examples were done during class, but no formal practice of using MATLAB® was done during class. In Spring 2013, three sections were taught in a computer lab where each student had access to MATLAB® during class time. Lecture time was restricted to a maximum of 20-30 minutes each class session and the majority of the time was spent doing in-class activities.

The focus of this inquiry is "does it make a difference?", specifically addressing the following questions:

  • Is the performance on midterms and finals affected by how material is presented in class, e.g., lecture, pre-readings, activities?
  • How is student confidence affected? Are students more confident in doing homework if they have a chance to practice in class first?
  • What helps students learn material—lecture, in-class work, homework, etc.—the most?
  • Were there any differences in student engagement and/or professor engagement?
  • Are students getting more out of the in-class time when lectures are shorter and they must practice what they are learning?

 

 

Course Artifacts

Course Design and Implementation

The course design focused on changing a lecture only course to a course in which the student would be expected to use MATLAB® every class session. To do this, the time devoted to lecturing was reduced to give adequate time for the in-class activities, which meant that some examples and advanced topics were not covered in a lecture. The students were given access to the full lecture notes (PowerPoint) through blackboard—even the parts that were not explicitly covered during lecture time. The activities focused on giving students a chance to practice what we just covered during the lecture time. Since this course is also intended to show students how to solve engineering problems, most in-class activities and homework problems were engineering "word problems" and required the students to use the engineering problem solving method in order to solve the problem. See the attached activities Ch6A2, Ch8A2, Ch12A2 for examples of the types of problems students had to solve.

Fall 2012 (Lecture Only)

Class was held twice a week in a lecture hall for 75 minutes each class session. Lectures were PowerPoint based and showed how code worked using MATLAB®. Typically one week was spent per chapter. Short, five point quizzes were given weekly to check basic understanding and attendance. There were a total of 13 quizzes, 11 homework assignments, and two projects. One midterm exam was given, as well as the final exam. All exams and quizzes were open book, open note, closed electronic devices.The midterm exam, final exam, homework, quizzes, and projects were each worth 20 percent of the total grade.

Spring 2013 (Lecture-Lab)

Class was held twice a week in a computer lab for 75 minutes each class session. Lectures were kept to a maximum of 20-30 minutes each class session, and some class sessions did not have any lecture time. The lectures were PowerPoint based and showed how code worked using MATLAB®. Typically one week was spent per chapter, except chapter 9, in which 1.5 weeks were spent. Every class session there was an in-class activity or quiz, and 21 of these activities were collected and graded. There were 10 homework assignments and 2 projects. The 11th homework assignment was replaced by an in-class oral presentation on project 2. One midterm exam was given, as well as the final exam. All exams and quizzes were open book, open note, closed electronic devices. The midterm exam, final exam, homework, activities/quizzes, and projects were each worth 20 percent of the total grade.

Attachments:

Syllabus Fall 2012

Syllabus Spring 2013

Sample activities:

·       Ch5A1: no lecture, only activity, which was I called “a self-guided tour through plotting with MATLAB®”, which was collected at the end of class

·       Ch6A2: second day on Chapter 6, no lecture, only the activity, which was collected at the end of class

·       Ch8A2: second day on Chapter 8, short lecture, then the activity, which was collected at the end of class.

·       Ch12A1: first day on chapter 12, students were told to read/study the lecture notes before attending class, there was no lecture, only the activity. The activity was collected the following class period.

·       Ch12A2: second day on chapter12, no lecture, only the activity, which was collected at the end of class


Course Artifacts

Findings

The following are the summary of the findings from this inquiry.

1.A majority (70-80%) of the students liked the organization of the course—short lecture and activities, and felt that the activities were very effective in helping the students learn the material.

2.The extra and guided practice in class gives students more confidence to do problems on their own.

3.If the time is spent in class practicing the material instead of just listening to a lecture, then the attempted homework grades will be improved.

4.The activities benefit the middle and bottom students the most, in terms of giving them more practice and in boosting their final course grade, though not enough to pass undeserving students.

5.Overall, grades were not drastically improved or worsened for the lecture/lab course compared to the lecture only course.

6.The instructor email load was reduced, giving the instructor more time to concentrate on other activities outside of class and office hours.

The pre-semester survey of students—about amount of programming and MATLAB experience—indicated that the students of both semesters had about the same demographics, as shown in Figure 1. Over 50% of the students do not have any programming experience and another 25 to 30% only have a little experience before they take ENGR A161.Seventy-five percent of the students do not have any experience with using MATLAB. These results are expected because ENGR 161 is meant to be an introduction to programming and MATLAB at the freshmen level, and the class is taught assuming that students have no prior experience.

 Figure 1 Students’ pre- semester experience with programming and MATLAB

Figure 1 Students' pre- semester experience with programming and MATLAB

The mid-semester feedback—given during week 4 of the semester—asked the students to judge the frequency and length of time spent in class on lecture, in-class activities that were turn-in for a grade, and those not turned-in, and the instructor working out examples. The results—shown in Figure 2—indicate that 70-80% of the students thought the organization of the class time was going well, and that the activities seemed to be a well-liked aspect to the course.  

 Figure 2 Perception of the frequency of activities during class time.

Figure 2 Perception of the frequency of activities during class time.


A survey given at the end of the course asked students how effective each course activity—lecture, in-class activities/examples, homework, quizzes, office hours, asking questions during class, and emailing the instructor—were in helping them learn the material. The results are shown in Figure 3. While each course activity was rated effective or highly effective by a majority of the students, the in-class activities and examples had the highest percentage rating them as very effective—over 60% and another 30% rated the in-class activities as effective for learning the material. Not surprisingly, the homework was rated as effective or very effective for learning the material by close to 90% of the students.  Over 80% of the students stated that the lectures were effective or very effective in helping them learn the material, which is an interesting result because research has shown that only X% of the population learns by listening, and lectures have been shown to be ineffective teaching tools. One speculation is that students think they need the lecture to learn because they do not spend time outside of class reading the text book or doing other effective studying techniques.

 Figure 3 Effective and in-effective activities for learning the material

Figure 3 Effective and in-effective activities for learning the material


Questions on the end of the semester survey asked students from both semesters to rate their confidence to using MATLAB and their confidence to write a computer program before the course and after the course. An additional question asked to the Spring 2013 students to rate their confidence in setting up the solution to an engineering problem before and after the course. The results are shown in Figure 4, 5 and 6. Both semesters showed that students gained confidence to not only use MATLAB, but also to program in general. A larger percentage of students in the lecture/lab course said they had some or a lot of confidence compared to the lecture only course—100% compared to 93%. Similar for confidence in programming—95% compared to 88%—a larger percentage of students in the lecture/lab course were some or a lot confident. In the spring 2013, the percentage of students who were some or a lot confident in their ability to set up engineering problems went from 26% before the course to 90% after the course. As expected, both types of courses improved the students' confidence to do the work taught in the course; however the lab/lecture course did show a larger improvement in student confidence, indicating that the activities in the course gave students more practice with the course material, increasing their confidence to do the work.

 Figure 4 Confidence in using MATLAB

Figure 4 Confidence in using MATLAB

 Figure 5 Programming Confidence

Figure 5 Programming Confidence

 Figure 6 Confidence to set up engineering problems

Figure 6 Confidence to set up engineering problems

In Figure 7, the average activity grade is compared to the average homework grade, both out of 10 points, for the Spring 2013 lecture/lab course. The grades are the averages of attempted and submitted assignments, meaning if a student missed class or did not turn in a homework assignment; the zero grade was not averaged in the values for the purpose of this plot. The majority (90%) of the students who were in class, did the activity and received a passing grade on the activity had an average of 7 or higher on the homework. In addition, 36% of the students from the lecture only course had attempted homework grades that were below 7, compared to the 10% in the lecture/lab course. One of the purposes of giving the activities was to give the students more practice and guided practice at programming before attempting the homework. The high homework grades attest to the success of the in-class activities for this purpose.

 Figure 7 Comparison of activity grade to homework grade for lecture/lab course.

Figure 7 Comparison of activity grade to homework grade for lecture/lab course.


Since the in-class activities were worth 20% of the student's grade, it was possible that students' final grades could be artificially inflated. This was not the case for this course. Figure 8 shows a comparison of the final course grades, with and without the activity grade included. Also on the plot is an line representing x=y. Grades on the line indicate that the activities did not have an effect on the final grade, grades below line indicate the activity grade had an adverse effect on the final grade and grades above line indicate the activity grade helped the final grade.  The boxes on the plot indicate the top, middle and bottom students. For the top—A and high B— students, the activity grade had a mostly neutral effect on the final course grades. This is expected because these students put in the necessary time to study regardless of the course structure. Mid-range students—low B and C students—saw a modest boost to their grade and in some cases, those on the grade cusps did see a grade improvement. The middle students may be those students who are not good test taker and giving them another method to show their expertise and experience is good for them to prove they know the material and pass the course. The bottom students—D and F students—saw the largest boost to their grade, but it was not enough to change a failing grade to a passing grade, therefore the activity grade did not artificially boost grades for students who did not deserve it. The greatest benefit of the activities and the extra practice is for the middle and bottom students. These are the students that we should strive to help the most and doing in-class activities is one way to engage these students and help them work through more examples and practice problems.

 Figure 8 Comparison of final course grades, with and without the activity grade.

Figure 8 Comparison of final course grades, with and without the activity grade.


On the final exam, one question which was used for both courses asked the students to write out the code to solve a problem and tested the student's knowledge of loops, and indexing. This is a tough problem and tests the students' ability to work out the logic in order to solve the problem. From the lecture only course, only 4% of the students received full credit on the problem, compared to 15% of the students in the lecture/lab course, which is a significant improvement. In the lecture only course another 13% of the students were able to figure some of the logic out and received 9 out of 10 points, while in the lecture/lab course it was 20% of the students. The extra time spent in class working on the activities helped the students practice do better on the culminating exam questions.

In Table 1, the averages of the grades—final course, final exam, midterm exam, homework and projects—are shown for both course types. The difference between the majority of the average grades is not statistically significant. The moderate improvements of the lecture/lab course over the lecture course are encouraging, but the grades are not a determining factor in the success of this research or not.  

 Table 1 Averages of grades for each course type


An unexpected result was that the number of emails from students asking questions about the assignment was reduced. The ratio of the emailed questions (counted) to the number of students in the courses is shown in Table 2, and was reduced from 0.80 to 0.75. More significant was the reduction of multiple email exchanges with the same student about the same problem. The ratio went down from 0.24 to 0.10.  The reduction in questions probably came from not only the students understanding the material better and being better prepared to do the homework, but also the time the professor had in class to walk around and answer questions from the students about the homework, making it unnecessary to answer questions by email later. In the lecture only course, many students asked questions after the lecture, but only those with laptops were able to effectively get their questions answered. While in the lab, everyone had access to a computer and could ask questions about their code.

 Table 2 Ratio of emailed questions from students to the number of students in the course.


Course Artifacts

Reflections

The following are observations about the difference between teaching using lecture versus lecture and in-class activities, and the results of the study.

1.  As the instructor, spending time in class helping students individually instead of lecturing was more fun. It also seemed that students enjoyed the time more as well. Students were not falling asleep during the lecture and were actually working on assignments that should improve their understanding of the material. By moving around the classroom and working with the students, I was able to catch and correct student misconceptions earlier than later grading the homework and exams.

2.  It was disappointing to not see an improvement in the exam scores, but this should not be the only measure of success of a teaching tool or not. Questions that were not answered include: were the students more efficient at their studying after doing the activities? What was the time the students spent outside of class for the lecture vs. lecture/lab course?I suspect that the students are better at the homework outside of class after doing the activities, which is supported by the amount of emails that I had to answer about the homework. In addition, the bottom student grades were improved. In any course, no matter the teaching style or technique, the top students will be top students and will be successful. By changing how we teach, we are able to help some of the middle and bottom students be more successful, and this is what we should be aiming to do as educators.

3.  Other data that would also be helpful in evaluating this course, such as tracking when and how often the course material is accessed on blackboard. This might give a clue as to when the students are working on their homework and studying and how long it takes to do the homework and projects.

4.  While students seem to want lecture, I do not believe it is the best method for teaching and learning. In the future, I will set up my courses just that the students are expected to study the material before class, and then work on problems in class that build on that base knowledge. The in-class activities were an easy method—in that the prep time was not very much to set them up— to have students practice problem solving in class. For someone who wants to start getting away from lecturing, this is a good place to start.

Course Artifacts

Faculty Contact

Jennifer Peuker
School of Engineering