For those of you unfamiliar with the NSSE and the CCSSE instruments they are student engagement surveys; the NSSE for four-year institutions and the CCSSE for two-year institutions. Both can be used as benchmarking tools, diagnositc tools and monitoring tools in the effort to continue moving towards greater outcomes of student success. Our institutions', and my primary interest in considering an instrument such as the CCSSE is to collect student reported data on the kinds of experiences they are having while at our campus.
"CCSSE was launched in 2001, with the intention of producing new information about community college quality and performance that would provide value to institutions in their efforts to improve student learning and retention, while also providing policymakers and the public with more appropriate ways to view the quality of undergraduate education."
The NSSE and CSSE have been criticized over the last couple of years as perhaps not being able to help institutions meet the outcomes most important to them; namely retention and graduation.
Citing the lack of any statistically significant relationship between NSSE scores and either grades or graduation, the authors note, "NSSE was never designed to measure these types of student outcomes, although it would certainly be encouraging if measures of engagement had some relationship to the sorts of outcomes that are of concern to senior administrators and governance officials."
Even more pointedly the NSSE and CCSSE were questioned on their reliability and validity.
While it took a few months, staff from NSSE and CCSSE recently responded to these critques in an Inside Higher Ed post. The authors of the post, who happen to be the directors for CCSSE and NSSE, state that no instrument is perfect and that when administered correctly and analyzed deeply they can provide institutions with powerful data for consideration. They make the point that these instruments were never intended to be used as predicitve indicators - something critics have said give these instruments less value.
Our institution is, actually, contemplating the value of becoming a CCSSE school and there are a lot questions swirling around that, I agree, must be answered before forging ahead.
- What is the value that we as an institution place on understanding and measuring student engagement?
Critics of the NSSE and CCSSE state that placing value on student engagement will not get us as close to our goals of retention and graduation as will a stronger focus on student learning and academic rigor or time on task.
- What can an instrument such as the CCSSE or NSSE tell us that we don't already know and NEED to know in order to improve student outcomes?
- How do we as an institution balance accountability with continous improvemt? Can and should CCSSE or NSSE support both?
- Should a single instrument be the only way we gather data on the student experience?
- In a time of diminishing resources is a national survey the best use of institutional money and time?
- And, most importantly for me, how will use what we learn to create space for meaningful dialogue and perhaps change?
While this is an issue that I do have strong feelings about - I'd actually like to see our institution engage our campus community in the administration and later anaylsis of the CCSSE - an even large issue for me deals with how institutions view the role of assessment in understanding students and in measuring and monitoring outcomes. I suppose that a second blog post dedicated solely to assessment in higher education would be a better place to begin this discussion.
A good friend and colleague - much more experienced with assessment than myself - reminds me regularly that the key questions we as student affairs practioners and institutions outght to be continually asking ourselves are:
- What is our mission/purpose? What do you value?
- Do the activities and services being provided intentionally support your purpose and values?
- How do you know that the activities and services being provided are making a difference?
So, perhaps until another blog post I'll leave it at that. I'd be interested in hearing the varying thoughts, opinions and experiences people have had with large national student engagement surveys or assessment in student affairs.
Libby, I find this very interesting and would also be curious to hear more about the topic from colleagues who have more experience with these surveys. Because I am unfamiliar with NSSE, I was perusing their web site trying to figure out what kind of questions the survey asks to see if I could form an opinion about the usefulness of this tool. Based on what I found, I have to side with the criticism you noted, and I can see how the studies that were in the article do not find much of a correlation. The questions I found (http://nsse.iub.edu) seem to be very broad at a huge institutional level.
ReplyDeleteWhile assessment is a huge topic in general in higher education, I do think it is nice to assess services in student affairs at least to some degree. In my experience, this is sometimes hard because of the qualitative answers that are easier to collect than quantitative data.
One example I would share that I think provides good assessment is in the office I work in, the Office of Admissions at UNC. We hold dozens of events each year for prospective students, and all of these events have a participant survey with them. I am responsible for the on-campus event for transfer students. This will be the third year we have held the event so we have survey results for two years. Being that it is a newer program, I value the survey responses to see what works and doesn’t work, what students like and dislike in the program. This assessment is used to make the events better in the future.
While this is on a MUCH smaller scale than NSSE or CCSSE, maybe the smaller scale assessment of student affairs is a better model over all.
The questions Libby posted associated with assessment hit very close to home as my place of employment, the Student Activities Office at UNC, began evaluation our program and services this summer. We’re using the Council for the Advancement of Standards in Higher Education, better known as CAS. When we started the process, a few of us saw the task as being quite daunting. However, as our training progressed, we realized exactly what Libby’s colleague reminded her: we are answering these 3 basic yet fiercely important questions. In my opinion, any measures available that help define and answer those questions are valuable instruments. The knowledge gained from the data about student engagement would be beneficial to helping provide insight into student retention, possibly especially if the institution doesn’t place a high value on measuring engagement as related to degree completion. However, the knowledge gained by the staff and participants through whatever process the program evaluation takes is equally, if not more important to maintaining and increasing positive learning outcomes. In my case, we as an office have already learned much about our approaches to our goals and our interpretation of the mission statement. And that is invaluable to our success as a team.
ReplyDelete