Teaching Sustainability; How’d it Go?

In 2014 Dr Erin Redman and I taught a Sustainability course, Socio-Ecological Systems for Sustainability. We designed an original structured approach based on current research, implemented it at a Mexican University and collected some data to assess the effectiveness of this approach in achieving the stated objectives. Here I share some of our preliminary analysis of that data.

Within the higher education arena, Sustainability courses have proliferated.¹⁾This is particularly so in the USA but far less so in other countries such as Mexico where we were working at the time. These courses take many shapes and forms but our course had a very different premise than what you typically find because:

1. The course was not located in any department or degree program. Even at US Liberal Arts universities, while almost all courses are open for students to take as electives, they are generally located in some existing department. This arrangement means that Sustainability courses are typically taught by a disciplinarian (eg Anthropologist) with a focus on its relevance for his or her students.
2. The course was stand-alone. This was not the introduction to a minor, a track or a major. This course was designed with the idea in mind that it would be the only Sustainability course that these students would be exposed to.²⁾Contrast this with Intro to Biology which is designed for Biology majors but taken by lots of people such as pre-meds.
3. The course was totally unconstrained. There was little administrative support or attention and no preexisting models that had to be built off of. We therefore were able to design the course from first principals.

What did we hope to achieve with this course?

Introduction and Stand Alone Sustainability courses suffer from the severe challenge that the breadth of content and of methods is just too enormous. Unfortunately, the result is generally to try and cram in as much as possible in an endless stream of facts and a rapid fire from one topic to the next. Therefore, while we did hope that the students would be able to describe some Sustainability issues such as Climate Change, we did not focus on content-oriented learning objectives. Rather, our main objective was to get the students "excited, engaged and hopeful about Sustainability," and create the foundation within them to be Sustainable citizens of the world in their personal and professional lives (not just in the classroom).³⁾In a future article I'll go into more detail than you'll want about our approach and framework, I promise!

This squishy non-fact oriented stuff can be scary for the ivory tower. But lets think about ethics courses at Business Schools. If the MBAs pass the tests but turn out to be totally unethical executives would anyone say that the course was a success? We must have the same standard with our Sustainability courses.⁴⁾This is particularly true for those courses intended as electives or general education and may be the only Sustainability exposure for those students, such as our was. Sustainability courses should be fostering a Sustainable citizenry like Biz ethics courses should foster ethical executives.

How'd it work out for us?

At the end of the course we asked the students to take an online survey and got 76 out of 86 of them to respond.⁵⁾We used Google Forms to design and administer the survey. Credit was given for participating but answers were separated from names to maintain anonymity. Some of the key results:

• 95% "liked the course" or "liked the course a lot" while 100% of them said they would "probably" or "definitely" recommend it to their fellow students and all but one said they would "probably" or "definitely" want to enroll in a follow-up course.
• 96% thought that Sustainability was "important" or "very important" for their careers while 100% of them agreed that “Due to their participation in the course changed their opinion on the urgency of resolving sustainability challenges”
• Students rating themselves "very familiar" with the concept of Sustainability rose from 14% before the course to 62% afterwards.

Overall these results indicate that the students enjoyed the course and that it sparked a continuing interest in Sustainability. More importantly though, they see Sustainability as not something that stays in the classroom but as something that they will apply to their lives and their profession. They even feel like they learned some content knowledge too.

Oh but there is more

Too much is often read into these immediate post surveys. Two of their problems are 1) the participants know what responses you are hoping for 2) objectives are typically long term change but surveys are done at the very end of the intervention/course--too short to even measure short term change! We were able to piggy-back on another research project of ours⁶⁾Investigating household waste behavior in the municipality of Leon, a project funded by PAPIIT. to implement a survey about the students attitudes, values and behaviors around waste before the class, after the class and one year later.⁷⁾We are still in the process of collecting the one year later data. Obviously self-reporting surveys have their own limitations but it gives us another way to see what kind of impact this course had on the participating students.⁸⁾We'll definitely be sharing the raw data, survey format and R script of all the analysis...once I figure out how; eg GitHub?!

This first graph shows us the mean of student responses (response range 1-4). In red are the means from before the class and blue shows us the increase in the means of the responses at the end of the course. We can see that across the board there was an increase in the "Sustainability" of the answers though there was quite a bit of variability. We'll get into that down the road. Overall this seems like a good result in terms of impacting the students but a difference of means itself isn't enough to be sure the difference exists or it was just the nature of our sample or randomness.

To examine if the difference between the pre and post responses was real, I ran a paired T-test on each of the question sets.⁹⁾Being able to pair the responses before and after gives us more strength with a smaller sample, which is nice. The dots represent the best estimate of what the real difference was between the pre and post responses. The I shaped lines are 95% confidence intervals. This means that we can be 95% sure that the real difference between pre and post responses falls in this space. We can immediately see that there are 4 questions that we can't be truly confidence that the difference in the responses is positive. For the rest we can be fairly comfortable saying that the course we taught changed the students responses in a meaningful way.

I hope that I have whet your appetite...

From what we have looked at so far we can conclude that the Sustainability course that we taught did change the students by the end but we hope that we are going to be able to learn a lot more than that!

1. Long term: We are currently collecting responses from the students to see how they answer one year later. The response rate will certainly be much lower but I have hopes that we will be able to see something interesting.
2. Specific questions: The obvious next step is to delve into specific questions which had big changes and those that didn't. That should be a fun post coming up soon.
3. Regressions: Based on earlier work¹⁰⁾Here is the PDF since the journal paywalls it. Dr. Redman and I did on a similar survey we intend to look at the relationship between the knowledge and value questions and the behavior questions. Is this relationship different before and after and long term? Did changes in knowledge and values change behaviors?

Would love to hear ideas for analysis, or other studies that I should check out. 🙂