Assessment


15
Jul 10

Tornado Watch: Assessments for Student Retention

As a resident of tornado alley, there is a summer tradition of dusting off the Twister DVD while scanning the afternoon skies for possible wall clouds. The film takes place in Oklahoma, but was filmed near my current home in central Iowa. The story follows a team of meteorological students and scientists as they attempt to place weather sensors in the path of a tornado to measure readings inside of the storm. After many failed attempts, injuries, and even fatalities, our protagonists successfully launch the sensors and save humanity. Err, save their research. As the flick can also be caught at least three times a week on cable during the summer, I catch up on all of my favorite lines.

Jo: [cow flies by in the storm) Cow.
[cow flies by in the storm]
Jo: ‘Nother cow.
Bill: Actually, I think it was the same one.

Watching the segment as the sensors rise into the F-5 tornado and begin generating data, I am reminded of our students, particularly those in the first-year. If we could read their minds and extrapolate the whirlwind of thoughts and emotions, surely we could develop better methods for student success and retention. Fortunately, there are a variety of assessments to assist in this process.

The College Student Inventory™ (CSI) from Noel-Levitz allows students to answer questions regarding their strengths and challenges before they even arrive on campus. I ask my incoming students to complete this assessment after summer orientation and use the information to frame our beginning of the year 1:1 appointments. The student and advisor reports are handy for discussion and the group summary reports provide great information for planning our first-year seminar course and programming topics.

MAP-Works® offers a similar tool to discover student transition issues early in the semester. Students develop a personal profile based on their initial campus experience that is measured for potential barriers to success. A web-based report is generated immediately for students and faculty or staff advisors that compares with all first-year students on our campus. Campus resource services are suggested where needed.

The Myers-Briggs Type Indicator® (MBTI®) helps demonstrate theory that there are distinct patterns to individual psychological types even though persons exhibit these patterns in different ways. Helping students to understand their type preferences and how they affect personal learning styles provides a common ground for understanding differences and the transition to college. I provide an MBTI learning styles assessment for each student in our first-year seminar each fall. Students do not always grasp the type concept, but do find meaning from discussion of the transition to university style learning.

It is common knowledge among student affairs practitioners that students enter the college or university with varying degrees of emotional intelligence. Additionally, those familiar with retention issues will cite non-academic challenges as the frequent impetus for student attrition. Assessing emotional intelligence using the EQ-i® allows students to see potential areas for growth that may enhance adaptation and coping skills leading to academic achievement. I find the EQ-i particularly helpful for students seeking direction in their academic or life plan.

While no assessment tool can foresee every difficulty faced by our students on the path to graduation, I have found these tools to be helpful for communication, planning, and advising. Not a certified MBTI or EQ-i user? Check with your human resources office for recommendations.

Have you tried these assessments? Other tools you suggest?

Enjoyed Twister and need a good summer read? Check out The Stormchasers.


28
Jun 10

“The School of One” Mentality

Recently, I finally got around to listening to one of my favorite podcasts, Freakonomics Radio. A couple of weeks ago, they came out with an installment called “How Is a Bad Radio Station Like the Public School System?” (Click here for their blog entry about it). It focuses on customization and highlights a pilot program called The School of One being done by the NYC Board of Education. It’s a pretty cool pilot program that they have launched, and it focuses on providing students with a customized style of education, like (as discussed in the podcast) Pandora radio.

It got me thinking about Higher Education. Our in-class curriculum methods are just as outdated (I know there are professors out there that are more modern and being innovative) as K-12. However, if you step outside the box of a second, Higher Education does provide students with different ways to learn and the main people behind it are us, the Student Affairs Professionals.

We provide students the opportunity to learn in a variety of ways that leads students to develop transferrable skills, explore of their identity, and many other learning outcomes. We are the ones pushing to integrate tools such as social media in Higher Education. Since it is summertime, this is a great time to think about concepts like this. Here are some things to think about:

Dynamic Learning Outcomes: We tend to know (consciously or subconsciously) what outcomes we want students achieve during our daily interactions with them. However, in this day and age, students tend to know what they want and are much more consumer driven. I know this is typically a characteristic labeled on millennials but, based on my experiences working at an institution with non-traditional students, they are just as consumer driven.

We need to ask them, what do you hope to get out of this? Our learning outcomes should be dynamic instead of static. This will help you to also re-adjust outcomes if you misjudged the first time around.

Flexible teaching style: What if our style is not working for that particular student(s)? Our methods for engaging our students must be flexible so that we can adapt our skills to how our student(s) best learn. This will benefit both you and those that you are work with.

Open and Constant feedback: In order to achieve my first two points, you need to have open feedback with your students and it needs to constantly happen. Receiving feedback will help you to adjust your methods so that the process works for both you and your students. I think it can also help you to make sure your students understand your outcomes. Having an engaging talk about this topic may help your students to embrace the outcomes versus insisting that they need to change.

So, keep up the good work my fellow colleagues and continue to find ways to both innovatively engage and educate college students. Having students collaborate with us in their learning will reap lots of rewards.


25
Feb 10

Developing Respect for Student Affairs #sachat

I was dragged out of my mini-blogging retirement by tonight’s #sachat. I’ve heard this debate many times about how to leverage respect from administration for what folks in student affairs do. I wanted to share my opinion but was feeling a little stifled by the 10 million people involved in the chat as well as the 140 characters so I figured I would share my thoughts in longer form.

First, this discussion about developing “street cred” if you will for student affairs is essentially asking how you get administration (a nameless faceless entity, i suppose) to understand what we do. The inherent assumption is that they don’t understand or that they do understand but they don’t think it’s valuable. If we were going to stereotype the view of student affairs both within and without of student affairs, I believe that stereo type would be that we’re “in loco parentis” style baby sitters, we create play time for our kids, and we hold their hands and try to protect them from the big bad faculty. As with most assumptions, this is unfair. Some of the big bad faculty are ALWAYS looking out for students, and some within student affairs favor the approach of throwing students to the wolves over a touchie-feelie approach.

Personally, I find this argument about “admin doesn’t understand what we do” a little tired. If they don’t understand what they do, they’re no different from most people at high levels within organizations. They understand in theory what we do, but they don’t know in detail (isn’t this what that new TV show is about?). Here’s a realization that I think we all need: THEY DON’T NEED TO KNOW WHAT WE DO IN DETAIL. It’s not their job to know the details of our job, it’s our job to know the details of their job. I’m almost certain that most of us have only a basic idea of what our chancellors and VPs do on a daily basis even though we might like to think otherwise. To think that it’s their responsibility to understand every intimate detail of every job on campus is childish.

Second, without fail, the first answer to “how to gain respect” is assessment. Don’t think I’m about to slam assessment; I love assessment, and frankly I don’t think we assess enough. During my internship in grad school in student affairs assessment, the assessment reports that I looked at were PATHETIC and an absurd joke. Most departments in student affairs don’t take this seriously and most of these reports barely get looked at because they don’t actually assess anything that carries weight at a cabinet meeting. VPs might like reading anecdotes, but it’s hard to imagine a meeting at the VP level where an anecdote about how a students heart was touched is read; it’s not that they don’t care, it’s just that they have bigger things to handle. Another problem with assessment – as an old quote says “there are lies, damned lies and statistics”, sharing a bunch of stats does nothing because it’s so easily manipulated.

Third, I think student affairs speaks a different language than the rest of the campus. One of the main critiques you hear of faculty on research campuses is that they don’t have time for students because they’re doing research. However, I would frame it a different way – they’re doing research because that’s a huge part of their job – big difference. When we’re talking about “i had 15 students come to my workshop on getting along with my roommate” and they’re talking about 9 journal articles they got published, we’re really talking on two different planes.

So, what’s my point?

I think I’m essentially arguing for the progression of student affairs as a profession. We’re not that old, not nearly as old as the faculty profession and I think the discussion of “how to gain respect” evolves out of a desire to take our profession from something we can be proud of on the campus, not just in our staff meetings or in our student affairs christmas socials where we’re making christmas bags for our students with glue, yarn and glitter.

I have a couple suggestions, but I don’t think this is a comprehensive list.

  1. what does the faculty use to establish their value to a campus?  I argued on Twitter that faculty establish value by publishing, bringing in students (recruiting), research grants and revenue from research.  If I were to summarize these thematically, I would call it “money and prestige”.  My question is how can student affairs bring money and prestige to the campus through our work.  I think the answer will be different for each office so I’ll not make more specific suggestions.
  2. If we’re going to do assessment, it needs to connect directly with the mission of our office AND our division and meet scientific research standards.  Most of our assessment is a joke because we don’t know how to be research scientists.  Clearly this is time consuming, but if the profession is going to progress I think this is an area we clearly don’t bring our A game currently.  Do it right, connect with the standards of real research scientists or don’t do it.


22
Feb 10

On Citizenship

Greetings from data land with the Multi Institutional Study of Leadership results for my campus!

As my previous few posts have indicated, I’m strolling down a very long road with analyzing our rather large volume of data from this study.  Some changes in our Institutional Research department on campus have left me with more of a role in managing this step of the process than I had planned, so this is a little more than what I expected!

As some of you know, the MSL is structured around assessing leadership as structured around the values of leadership in the Social Change Model. (read about the theoretical frame for the study on the NCLP website!) This model offers values of leadership based on individual, group, and community values.

We were investigating the specific results along one of the values that is essentially important to our campus, Citizenship.  Our campus has invested a great deal of time and focus around building our commitment to a mission of service as indicated in our motto, “not to be ministered unto, but to minister.”  I’m working with a graduate student on our campus this semester on a project relating to community service involvement of men on our campus, so I immediately went to that area and looking at gender differences.

Rather than digging in to this aspect of the data, my eye strayed over toward the opportunity to look at other areas of campus involvement in addition to prior involvement in community service.  Instead, I went to look at the students who have cited prior involvement in campus programming organizations.

DING! DING! DING! We have a winner!

Students who cite prior involvement in programming organizations reported significantly higher results on Citizenship than our general student body. Their results are comparable to those from students who note prior community service involvement.

We knew this all along, didn’t we?!  Or did we?

I know that our programming board students are always among the most selfless students that I have the pleasure of working with.  I also know that their commitment to improving our campus has always been second-to-none after over 15 years of experience across a few institutions.

Knowing all of this, why have I never linked this to Citizenship before?  Conversations about citizenship on our campus typically only include discussions of service activities outside of our campus boundaries and various forms of political activism.  Yet, this study is telling me that the value of Citizenship is alive and well in our programming students to a comparable extent to our students involved in more typically termed “service” activities.  I am really not saying that all groups of students are meeting a “need” that is even comparable, but our students working in all of these capacities are telling me they understand Citizenship in a different way than our general student body.

If our field is emphasizing a focus on student learning and we see that learning about Citizenship is happening in campus programming, is there any hope of expanding a definition of service to include campus-based service?

I’m looking forward to your thoughts on this one.


13
Nov 09

Using Facebook Insights to Assess Your Facebook Page

Facebook Insights is an application available to all page
administrators.  It measures
exposure, actions, and behavior related to your page.  Insights gives you almost instant data––it is updated 12
hours after the end of each day. 
Available data includes: geographic and demographic information about
your fan base, a record of interactions (comments, wall posts, photo/video
views, “likes”), and the growth of your fan base over time.  All of this information is displayed in
the Fan Dashboard, and most of it can also be downloaded into a spreadsheet so
you can use the data in any way you see fit.

In Practice: How UW–Milwaukee University Housing Uses
Our Facebook Page

I created our Facebook Page
in July 2009.  After the first week of school, we were hovering under 100
fans.  Next, I harnessed the
influence of the Neighborhood Housing Office, who at that time had over 200
fans of their Facebook Page. 
Within two days of suggesting our page to all of their fans, we had over
200 fans as well.  By this point I
felt that our page had enough content and fan support/interaction to go
public.  So, when we launched our
new website on September 10, I included a prominent link to our Facebook Page
on the homepage.  Since then, our
fan base has grown at a modest pace of 3 pans per day, which should put us over
400 fans by the end of this semester. 
Facebook Insights gives me the tools to track this data easily,
especially if I take note of the dates when actions with potential impact on
our fan base occur (other page suggestions, new advertising strategy).

By downloading the Fan Demographic data from Insights, I can
see that 75% of our fans are between the age of 18-24 and 5% are 13-17 years
old.  So, it’s likely that 80% of
our fans are current or prospective students.  The remaining fans are likely staff members and
parents/family members –– yes, we have had a parent become an active fan of our
page!

Knowing who our fans are is not enough; to truly assess our
Facebook activities, I need to know if students are interacting with our
content.  From the Fan Interaction
data set, I can report on the number of total interactions with our content
(74), average number of interactions per post (1), total photo views (1711),
video plays (170), and comments on our content (30).  Astonishingly, our Facebook Page has had 3,875 page views
from 1,234 visitors since its creation. Soon, Insights will also allow page
administrators to see the Click Through Rate and Engagement Rate, which will
provide a clearer picture of how many fans are responding to your content.

Do you use Facebook Insights for assessment or reporting?  How have you used the data?


15
Oct 09

Sustainability in Student Affairs: What are you doing?

One of my personal areas of interest in sustainability.  This morning we had a staff meeting in another area of campus.  I left my reusable coffee cup at home asuming we would have ceramic cups at the meeting (sometimes our director splurges and provides us with drinks at these meetings), and I was saddened to find paper cups instead! I dejectedly used my first disposable cup since July!

This got me thinking about the things we are doing at Penn State to educate students about sustainability and whether or not they are actually making a difference in the long term. For example, two weeks ago was Penn State’s “Great Recycling Challenge.” This is a very exciting project where we encourage students to recycle as much as possible. Staff members from the Office of the Physical Plant actually audit the garbage for the week and determine which residence hall area recycles the greatest percentage of their recyclable trash. Students get daily updates on their recycling efforts, and there is a lot of friendly competition from the areas to see who can win the Great Recycling Challenge trophy. (For laughs, here I am with the trophy: http://www.flickr.com/photos/bck134/3945667667/ ) The area I supervise is the three-time defending champions . . . . but are my students more likely to recycle when they graduate? I don’t know (and I doubt it).

What sorts of sustainability projects are going on in the student affairs offices at your institution? How do you assess their effectiveness, and how do they contribute to student learning at your institution?


13
Sep 09

Introversion in student affairs

I had a conversation recently with our graduate intern about life as an introvert career counselor.  He's just starting his life as an introvert in the world of student affairs, and our conversation lead me to reflection of the earlier years of my professional life.

I told him to be aware and be intentional in his consideration of how his introversion is or is not an issue as he learns to navigate his professional life.  When I first transitioned from registrar-type work into my first counseling role it took me a while to understand my introversion in that generally-extroverted context.  In fact I think for about the two years I did a horrible job of balancing my needs as an introvert and the demands of the work.

As an academic counselor in a setting with ludicrously high advising ratios (try 5500 students to 4.5 counselors!), there was no down time, no time to re-energize, no time to contemplate or rejuvenate.  There was only go, go, go.  Next student, next student – typically 10 per day, in half-hour increments.  My evenings became more solitary.  My outlook, considerably more grumpy.

It took me years to understand why I was (and am) more productive in the morning than in the afternoon; why many of my ideas wouldn't be pursued because they didn't feel fully formulated until after the brainstorming session had ended; and why I preferred not to go out to lunch with my colleagues each day.

Now, not every student affairs position will demand this hyper level of interaction and outward-facing energy.  But, they all will have varying degrees of the same, and I believe that even for lower traffic student affairs offices there is a relatively steep on-ramp for the new introverted professional.

New professionals need to understand that these things are okay, while simultaneously learning that being in student affairs sometimes means putting on your "extrovert mask," even when it doesn't feel natural (it became entirely comfortable for me over time, I'm happy to report).  Here are a few other tips for the introverted new student affairs professional:

  • Don't feel guilty about needing time to yourself.  If that means foregoing lunch with colleagues or occasionally skipping out on that office social, do it.  But of course, you must strike a balance to make sure that you don't accidentally develop a curmudgeon label.
  • If you know the subject of a particular meeting, make notes and write down some of your thoughts ahead of time. It may help you to feel like you can participate more actively, having thought through the issues ahead of time.
  • Find the ways that are most accommodating for you to "become" an extrovert when it is needed.  Some skills like public speaking or working a room may not be natural, but you need to make them become comfortable. So, practice, practice, practice (then sit quietly for a while!)

As a profession, and indeed as a culture, we tend to prefer extroverts.   Extroverts often make more-immediate impressions and many qualities associated with extroversion are thought of as positive in the workplace.  But, our students are represented across the introversion/extroversion spectrum, as should be our student affairs professionals who will work with them.

Fellow introverts, I'd love to hear your thoughts and perspectives.

Gary Alan Miller

Cross posted on my blog


22
Apr 09

Student leader learning?

As the semester comes crashing to an end, you may be wrapping up programs and events with student leader teams. I found this thoughtful list of questions from Tim Milburn over at studentlinc. This semester, instead of asking your student leaders to evaluate the experience or program, ask them to complete a self-assessment. My peer mentors are hired in the fall for a calendar year term of service, so I may have them complete the questions now and once again in December after their fall experience.

  1. What did I learn as a student leader?
  2. What will I need to remember from my student leadership year?
  3. Which interactions with others taught me the most about how to work with people?
  4. What do I know now that I didn’t know a year ago?
  5. What am I better at as a result of this student leadership experience?

  6. How would I describe my student leadership experience in 100 words?
  7. How am I better prepared for the next chapter in my story?
  8. What would I have done differently as a student leader?
  9. If I had one hour with a group of newly elected student leaders, what would I want to talk to them about?
  10. What mistakes did I make this year and what did I learn from them?

  11. What do I hope to be remembered for as a student leader?
  12. How could I have done better as a student leader?

Completing a self-assessment allows student leaders to depart the experience with a greater context for their contributions and learning. Over at studentlinc, Tim has shared a pdf of these questions and other great student leader information.

Do you have student leader evaluation or assessment ideas to share?


16
Apr 09

“FIRE-ing” the Canon? Are the foundations of our profession being assaulted, or are we the “barbarians at the gate?”

Though there has been an evolution of thought and significant changes in approaches over the years, the guiding principles outlined for the Student Affairs profession have always leaned heavily on the idea that learning happens both inside and outside the classroom, and that learning and development are inseparable from each other. Student affairs practitioners, therefore, have a responsibility, as educators, to provide meaningful opportunities for learning and development.
 
Learning Reconsidered expands on this, and argues that "Every resource on every campus should be used to achieve transformative liberal education for all students, and all colleges and universities are accountable for establishing and assessing specific student outcomes that reflect this integrated view of learning."
 
Sounds great, doesn't it?
 
But just like the idea that "it takes a village to raise a child," an idea that might warm the heart of some sends chills down the spine of others. One person's "Kum By Ya," and hand-holding around the campfire, is to another the battle cry of barbarians at the gate, set on knocking down the pillars of the academy and toasting smores over the smoldering ashes of the library's special collections.
 
Figuring out what "specific outcomes" would be appropriate seems to be the main front in this war between traditionalists (who believe that learning is the exclusive realm of academic affairs) and those who feel that learning needs to be reconsidered (us). It's like putting two economists from different schools of thought in a room together and asking them to explain the state of the economy and its future direction. The conversation will be long, circular, at times acrimonious, and though full of points and counterpoints, likely resulting in nothing resembling consensus. Both sides will likely declare at least a moral victory in having made their points, and the onlookers will still be confused and dazed.
 
It's no surprise, then, that some institutional efforts to define particular learning outcomes have been criticized, even vilified, and held up to intense public scrutiny. If you haven't heard of the Foundation for Individual Rights in Education (FIRE), chances are, you will.


FIRE's mission statement seems a clarion call to those who cling tightly to more traditionalist views of academia, against what they seem to perceive as mass indoctrination of students by politically motivated and left-leaning pseudo-academics. You can learn more about FIRE and some of their projects at their website. If you haven't read up on them, you really should.

As a profession, how should Student Affairs respond to these challenges? How can we develop outcomes that will be both generally accepted, and practical to implement? Is the "outcomes" movement tantamount to a warning shot against all the academy holds sacred, or just a loud declaration of relevance from those of us who have resided so long in the outer provinces? Are we destroying the academic canon, or are organizations like FIRE burning the library and blaming us for the smoke? Who are the barbarians? And who's really guarding the gates?


17
Nov 08

New in the Toolbox: Emotional Intelligence

My spare time this month has been used to prepare for and complete certification in the use of the Emotional Quotient inventory or EQ-i. Emotional-social intelligence is a cross-section of interrelated emotional and social competencies, skills, and facilitators that determine how effectively we understand and express ourselves, understand and relate with others, and cope with daily demands. Understanding and assessing EQ in business and leadership coaching is common and research indicates that that the tool is equally useful in the academic setting with an 85% predictor rate for college success. I look forward to building expertise with this assessment and employing it to assist the transition of my first-year students.

The EQ-i is assessed through an online survey resulting in measurements of five areas: interpersonal, intrapersonal, stress management, adaptability, and general mood. Fifteen subscales or facets provide dimension to these scale areas.

Ml_eqhead_2EQ-i will join the College Student Inventory and MAP-Works in the toolbox of assessments that I rely upon for identifying issues challenging students in those first few crucial weeks of college. The College Student Inventory provides me with timely and strategic information on my students prior to their enrollment. Most importantly, it allows me to identify those with high need for student service intervention. MAP-Works is offered to students in the third week of enrollment and is a new complement to our campus retention initiatives. It aggregates student perception upon arrival and integration to the institution. Both surveys are great mediums for creating relationships with new students.

What’s in your student retention toolbox?

  • Tags

    Graduate Students blogging Twitter Tips students Social Media career Conferences ideas professional development Higher Education Job Search how to This and that technology faculty YouTube nacada Advice academic advising Best Practices acpa #saplacement video social justice NASPA NACADA Tech #sachat engagement RSS student affairs technology The Placement Experience administration interviews assignments Orientation Poll candidates student activities life purpose Student Affairs leadership facebook Placement leadership development