How Colleges Address the Issue of Measuring Students’ Learning
June 5th, 2012 by Dr. Bruce Johnson
You’ve completed an assignment and earned a passing grade, but can you say that you have actually learned something? This is a question that educators, colleges, and regulators are also asking about college students – especially given the issues surrounding tuition increases, dropout rates, and the value of a degree for finding work in a struggling economy. It’s a question I’ve also asked as an educator – are my students really learning just because they write a paper or answer a discussion question, and more importantly, what does it mean to learn?
Schools rely upon traditional methods of assessment, completion rates, and grades to demonstrate that students have learned. As institutions of higher education respond to continued questions about the value of an education, the focus is directed towards knowledge acquired and benefits students receive as an indicator of learning. Because of the transformative and personal nature of learning from students’ perspective, it may be necessary to utilize a measurement tool that assesses knowledge and competencies.
The Questions about Learning
A thought-provoking statement about higher education in America was made in the article, Are College Students Learning?, that “most institutions have no meaningful way to measure the quality of their instruction.” What I find interesting about this viewpoint is that students are not dependent upon superior instructional delivery to ensure that they learn. Of course they will have a much more positive and productive experience if their instructors are committed to quality instruction; however, students are completing their studies and assignments without direct control by their instructors. Perhaps the article, Measuring the Value of a College Degree, provides a better perspective about the question of learning by indicating that “while a diploma is a signal of what an individual has learned, it is not necessarily indicative of the actual skills and expertise that individual is ready to apply.”
A book published in 2011, Academically Adrift: Limited Learning on College Campuses, by Richard Arum, a professor of sociology and education at New York University, and Josipa Roksa, a professor of sociology at the University of Virginia, raised further questions concerning students’ learning in college. A New York Times blog, How Much Do College Students Learn, and Study?, discussed the book and its related study. The authors studied 2,300 undergraduates and “concluded that 45 percent demonstrated no significant gains in critical thinking, analytical reasoning, and written communications during the first two years of college.”
While this book has gained attention and added to the conversation in higher education about learning in college, the results do not seem surprising to me. During the first two years of college I would concur that 55 percent of students do make a significant gain with the use of these skills because of a willingness to learn, and 45 percent are either well-equipped already or they are struggling to just get by. Utilizing these statistics it could be said that a majority of students do learn in college.
Standardized Testing for Colleges?
Another New York Times article, Trying to Find a Measure for How Well Colleges Do, discussed the possibility of national testing for colleges based upon the findings of a Department of Education report issued in 2006. Commissioned by the education secretary at that time, Margaret Spellings, the report indicated that “learning must be measured by institutions on a ‘value added’ basis that takes into account students’ academic baseline,” and that “the results must be made available to everyone to measure the relative effectiveness of different colleges and universities.”
The implication is that standardized testing may be an answer, similar to the No Child Left Behind mandate for primary education, which has not been well-received in higher education. The reason is that exams do not always measure learning because too often teachers teach to the test and students memorize what is needed to pass the test. True learning does not occur as the information memorized is only retained by students for a short period of time, and only through reading comprehension, along with repetition and authentic use, will the information be stored in long-term memory.
Higher Education Responds
A coalition of higher education groups, New Leadership Alliance for Student Learning and Accountability, has a mission and purpose “to move the higher education community towards gathering, reporting on, and using evidence to improve student learning in American undergraduate education.” A report was developed, Committing to Quality: Guidelines for Assessment and Accountability in Higher Education, which provides institutions with strategies to put into use for the assessment of student learning. Within this report it indicates that “those granting educational credentials must ensure that students have developed the requisite knowledge, skills, values, and attitudes that prepare them for work, life, and responsible citizenship. – and U.S. higher education must focus on both quantity and quality — increasing graduation rates and the learning represented in the degree” (p.3).
In February 2012, the Association of American Colleges and Universities (AAC&U) released A Change on Student Learning Assessment: An AAC&U Working Paper. The AAC&U works with higher education institutions to “raise the level of student achievement on key capacities—what we call the Essential Learning Outcomes—that are relevant to work and life in the 21st century.” These learning outcomes include “knowledge: inquiry and analysis; critical and creative thinking; integrative and reflective thinking; written and oral communication; quantitative literacy; information literacy; intercultural understanding; and teamwork and problem solving.” This is an important aspect of the learning process as courses are designed with learning outcomes or objectives as the starting point.
These learning outcomes were developed not just from random beliefs about learning but from the needs of employers. As an example, the AAC&U learning outcome of information literacy was developed based upon a need for employees in the workforce to “use information from a number of sources and to be able to prepare reports that interpret quantitative and qualitative information.” In the AAC&U report it discussed a survey that was conducted in 2010 by Hart Research, which asked employers to rank their priorities for skills sought from college graduates. The top skills listed included:
• Effective oral/written communication: 89%
• Critical thinking/analytical reasoning: 81%
• Knowledge/skills applied to real world settings: 79%
• Analyze/solve complex problems: 75%
• Connect choices and actions to ethical decisions: 75%
• Teamwork skills/ability to collaborate: 71%
• Ability to innovate and be creative: 70%
Learning outcomes or learning objectives are also called competencies because it refers to an expectation of knowledge acquisition or mastery of the course content and skill set development. The use of objectives serves to establish accountability for both instructors and students. Instructors are accountable for providing resources, developing activities, and utilizing instructional techniques that reinforce completion of these goals. Students are accountable for their work product and for demonstrating progress made towards meeting the objectives. It seems to be a much more effective measure of college-level learning to assess a uniform set of students’ competencies than to provide a standardized test that focuses strictly on subject matter knowledge gained.
What Learning Means to Students
It is easy to point to colleges and state that students are not learning because there isn’t a standardized system of measurement. However, students are transformed by the learning process simply as a result of being introduced to a structured or formal classroom environment. While everyone learns informally through their life experiences, it is an unstructured and often unfocused process. Once students learn self-discipline, along with enhanced reading, writing, and research skills – they are often transformed and become lifelong learners.
The article, What Does College Teach?, discussed How College Affects Students, a “review of thirty years of research on college learning” by Ernest Pascarella and Patrick Terenzini. The results found that “simply going to college, any college, makes a major difference in a person's psychological development: students come away with improved cognitive skills, greater verbal and quantitative competence, and different political, social, and religious attitudes and values.” While students may not memorize the textbook theories and all related subject matter content, they have gained skills and increased their capacity to learn because they were introduced to the process of formal learning.
This now brings me back to the original questions posed in this post, which were this: are my students really learning just because they write a paper or answer a discussion question, and more importantly, what does it mean to learn? The answer is that yes, students learn because of their involvement in class. However, what has been learned may not always be fully measured through a test that assess knowledge gained, as the learning process is gradual and requires development of supportive skill sets. Even core competencies may not be fully developed and utilized until students have progressed through several classes in their degree program.
As institutions of higher education look for indicators of learning beyond grades and completion rates, the answer may lie in the use of uniform core competencies and learning outcomes that guide the development of courses and degree programs.
Photo © Images.com/Corbis