Solving Two Out of Three Problems with SLO

As promised here is the post on why “S” in SLO stands for Spurious, the “L” stands for Loathsome, and the “O” stands for Odious.

Let us start with the biggest problem with SLO the Spuriousness of the data. The standard methodology in statistic is when collecting data for comparison a single set of data should be collected. One should not compare apples to oranges. The current SLO assessment allows each instructor or group of instructor to decide how to measure the outcome. Depending on how it is measured and who is doing the measuring you will come up with different results. The only way around this problem is to standardize the outcome and the assessment tool. This works well when assessing outcomes in the vocational fields such as nursing and auto technology where the outcomes have concrete results. However, it does not work in fields where there is disagreement about the conclusion of the study. Example from economics include should taxes be raised or lowered. The answer to this question depends on your perspective. If you are a long-run thinker you will concentrate on the deficit and debt left behind which will hamper growth. If you are a short-run thinker you will concentrate on creating jobs to pay the bills. The choice depends on your current position and your personality. One can argue that we can write an outcome that measures this exact point. (if you continue reading you will see that faculty are already doing this!)

But how will we assess this point when emotions are involved. How useful are such assessments? Did the student miss the question because they could not separate their personal beliefs from what was being studied in a rational manner versus they did not get the concept completely. We could break the question down to study it at the granular level but then this can not be done in a simple manner. I as Economics instructor when assessing students to assign grades go through this process. (I told you we the faculty are already assessing our students!) Basically the faculty are being asked to duplicate their work and this leads to the “L” in SLO which stands for Loathsome. It is loathsome in that we are forced to duplicate our assessment and take time away from instruction.

The Administrators and non-experts in order to make the process less Loathsome for the bureaucracy and easier for the non-experts to understand the data we are asked to measure uses a simple assessment tool with Satisfactory/Unsatisfactory. So ease of Data collection and tabulation for bureaucrats leads to more work for the instructor and worst unreliable data. (Basically they are asking us to enter useless data so they can have another form that collects dust!)

Another field the assessment can be counterproductive is in fields where creativity is valued. Creativity and standardization are two different things. So Standardization which kills creativity leads to the “O” in SLO which stands for Odious. How are we to standardize our students’ tastes?

The only way around these problems, with the exception of creativity (The Odious Problem), is to adopt standardized tests created by a panel of experts and use them when assigning grades. If we are going to do this right lets Standardize the assessment. Basically, our grades and assessment scores will be one and the same. This solution is still odious due to the problem of Academic freedom. But the people who work in the intellectual gutters of universities, better known as Schools of Education, will come up with some other bright idea to solve this problem of academic freedom. I will confess that I am willing to teach to some standardized test such as CLEP instead of wasting my time reinventing the wheel by creating new questions, and wasting more time assessing students twice. The current set up asks the instructor to assess once for the grades they assign and once for the purposes of SLO. Next the current system asks us to waste even more time discussing it with other faculty who may or may not be in our field. These discussions usually lead to changing the assessment questions which leads to the problem comparing two different sets of data across time. This is basically comparing data for apples in one year to data for oranges in another.

I think the biggest obstacle to having a standardized test will be the Administration and K-12 teachers. Administrators will not like these standardized tests because they will see all of the students who will fail these standardized tests, thereby causing enrollment to fall and graduation rates falling as fewer students pass college level courses. The K-12 teachers will fear these tests because it will highlight the fact that our high schools are passing and graduating students who are not prepared for college.

In conclusion, if we are truly serious about SLO in the Social Sciences we should just buy the tests from CLEP for various disciplines (click here for a list of classes we can standardize) and use them to determine our students’ grades. We as faculty will teach to these tests and at the end of the semester our discussion will be based on truly reliable data, and not a waste of time discussing something spurious.