Assessment


12
Oct 11

‘Perspectivizing’ Your Online Engagement Data

Okay, I made up that word, but before our etymologist readers split for the hills, hear me out as to what it means.

When it comes to social media, I’m a data person and every month I run an analytics report to measure what’s working, what’s not working, and the overall growth of a couple Facebook Pages I help manage.

One particular Page has over 20,000 “Likes” on Facebook. A poll or link generates a ton of responses on their Page. If I post that same link on another Page that is 1/10th the size, the response rate is extremely low.

Without “perspectivizing” the data, the smaller Page would probably feel unhappy about the online engagement in comparison to the larger Page. But if we calculate the average level of online engagement per Fan (ALE), we find a different story.

The 20k Fan Page generates a .096 ALE whereas the 2k Fan Page generates a .153 ALE.

Applying an ALE score helps both big and small players really see what’s going on with their accounts.

Here’s how to calculate the ALE of your account…

  1. Log into your Facebook Page Insights.
  2. Change the date to the date you want to calculate.
  3. On the “Overview” page scroll down to “Post Feedback” and record that number.
  4. Click on the “Users” page and scroll down to “New Likes.”
  5. Toggle “New Likes” to “Total” and hover over the end of the time period and record that number.
  6. Divide the “Post Feedback” number by the “Total New Likes” number to determine your Page’s ALE number.

Once you’ve calculated your ALE, please share it in the comments below so we can see a community norm.


3
Oct 11

“Please Complain.”

That line is something that every syllabus should include in its first few lines. Not in the context of, “This is too hard.” or “This is too much work.” but rather in the context of “you aren’t challenging/stimulating/engaging me.”

In any classroom the facilitator should be engaging the participants in the learning, as opposed to lecturing and breathlessly spraying knowledge into the air. The latter is akin to throwing spaghetti against the wall, hoping some of the noodles stick. To take the analogy further, lecturers will show off the noodles that stick (A’s for those who learn by listening) while brushing the ones that don’t stick (C’s for those who learn better by doing) under the rug.

What brought me to this topic is a perfect storm of an insightful Seth Godin blog post (also titled, “Please complain”) and a blog post by one of my students who innocently touted, “My classes are way too easy… They are kinda pathetic in a way.” Upon approaching the student, I encouraged her to complain and let me (as one of her educators) know that the classroom that I’m running isn’t challenging, engaging, or stimulating her thinking. I’m not sure if she was more stunned by the fact that someone was actually reading what she was putting on the internet or by the fact that I was empowering her and encouraging her to follow through on her willingness to be challenged. In any case, we’ll see if she follows through.

My point is this, in a higher education environment that is increasingly focusing on costs, revenues, returns-on-investments, and customer satisfaction – who are we to not encourage and empower our students to complain if they are unhappy with their academic or co-curricular experiences? We are here to facilitate their learning, to create safe environments for their self discovery through successes and failures, and to ask questions such as “how would you make it better?”

In the end however, it’s up to us. We can either challenge, support, and engage or we can keep throwing spaghetti at the wall and hope enough of it sticks to make a difference in the appetites of our hungry learners. Seth Godin says it best:

“Acquiring and processing user feedback is a choice. If you want people to speak up, be clear and mean it. If you don’t, don’t pretend.”

Is this a mentality of higher education becoming consumer focused? Is this the mentality of a generation feeling entitled enough to tell an educator when they are teaching in ways that are no longer efficient or applicable?

I hope you feel challenged, engaged, and stimulated enough to comment and continue the discussion.

 

 


15
Jul 10

Tornado Watch: Assessments for Student Retention

As a resident of tornado alley, there is a summer tradition of dusting off the Twister DVD while scanning the afternoon skies for possible wall clouds. The film takes place in Oklahoma, but was filmed near my current home in central Iowa. The story follows a team of meteorological students and scientists as they attempt to place weather sensors in the path of a tornado to measure readings inside of the storm. After many failed attempts, injuries, and even fatalities, our protagonists successfully launch the sensors and save humanity. Err, save their research. As the flick can also be caught at least three times a week on cable during the summer, I catch up on all of my favorite lines.

Jo: [cow flies by in the storm) Cow.
[cow flies by in the storm]
Jo: ‘Nother cow.
Bill: Actually, I think it was the same one.

Watching the segment as the sensors rise into the F-5 tornado and begin generating data, I am reminded of our students, particularly those in the first-year. If we could read their minds and extrapolate the whirlwind of thoughts and emotions, surely we could develop better methods for student success and retention. Fortunately, there are a variety of assessments to assist in this process.

The College Student Inventory™ (CSI) from Noel-Levitz allows students to answer questions regarding their strengths and challenges before they even arrive on campus. I ask my incoming students to complete this assessment after summer orientation and use the information to frame our beginning of the year 1:1 appointments. The student and advisor reports are handy for discussion and the group summary reports provide great information for planning our first-year seminar course and programming topics.

MAP-Works® offers a similar tool to discover student transition issues early in the semester. Students develop a personal profile based on their initial campus experience that is measured for potential barriers to success. A web-based report is generated immediately for students and faculty or staff advisors that compares with all first-year students on our campus. Campus resource services are suggested where needed.

The Myers-Briggs Type Indicator® (MBTI®) helps demonstrate theory that there are distinct patterns to individual psychological types even though persons exhibit these patterns in different ways. Helping students to understand their type preferences and how they affect personal learning styles provides a common ground for understanding differences and the transition to college. I provide an MBTI learning styles assessment for each student in our first-year seminar each fall. Students do not always grasp the type concept, but do find meaning from discussion of the transition to university style learning.

It is common knowledge among student affairs practitioners that students enter the college or university with varying degrees of emotional intelligence. Additionally, those familiar with retention issues will cite non-academic challenges as the frequent impetus for student attrition. Assessing emotional intelligence using the EQ-i® allows students to see potential areas for growth that may enhance adaptation and coping skills leading to academic achievement. I find the EQ-i particularly helpful for students seeking direction in their academic or life plan.

While no assessment tool can foresee every difficulty faced by our students on the path to graduation, I have found these tools to be helpful for communication, planning, and advising. Not a certified MBTI or EQ-i user? Check with your human resources office for recommendations.

Have you tried these assessments? Other tools you suggest?

Enjoyed Twister and need a good summer read? Check out The Stormchasers.


28
Jun 10

“The School of One” Mentality

Recently, I finally got around to listening to one of my favorite podcasts, Freakonomics Radio. A couple of weeks ago, they came out with an installment called “How Is a Bad Radio Station Like the Public School System?” (Click here for their blog entry about it). It focuses on customization and highlights a pilot program called The School of One being done by the NYC Board of Education. It’s a pretty cool pilot program that they have launched, and it focuses on providing students with a customized style of education, like (as discussed in the podcast) Pandora radio.

It got me thinking about Higher Education. Our in-class curriculum methods are just as outdated (I know there are professors out there that are more modern and being innovative) as K-12. However, if you step outside the box of a second, Higher Education does provide students with different ways to learn and the main people behind it are us, the Student Affairs Professionals.

We provide students the opportunity to learn in a variety of ways that leads students to develop transferrable skills, explore of their identity, and many other learning outcomes. We are the ones pushing to integrate tools such as social media in Higher Education. Since it is summertime, this is a great time to think about concepts like this. Here are some things to think about:

Dynamic Learning Outcomes: We tend to know (consciously or subconsciously) what outcomes we want students achieve during our daily interactions with them. However, in this day and age, students tend to know what they want and are much more consumer driven. I know this is typically a characteristic labeled on millennials but, based on my experiences working at an institution with non-traditional students, they are just as consumer driven.

We need to ask them, what do you hope to get out of this? Our learning outcomes should be dynamic instead of static. This will help you to also re-adjust outcomes if you misjudged the first time around.

Flexible teaching style: What if our style is not working for that particular student(s)? Our methods for engaging our students must be flexible so that we can adapt our skills to how our student(s) best learn. This will benefit both you and those that you are work with.

Open and Constant feedback: In order to achieve my first two points, you need to have open feedback with your students and it needs to constantly happen. Receiving feedback will help you to adjust your methods so that the process works for both you and your students. I think it can also help you to make sure your students understand your outcomes. Having an engaging talk about this topic may help your students to embrace the outcomes versus insisting that they need to change.

So, keep up the good work my fellow colleagues and continue to find ways to both innovatively engage and educate college students. Having students collaborate with us in their learning will reap lots of rewards.


25
Feb 10

Developing Respect for Student Affairs #sachat

I was dragged out of my mini-blogging retirement by tonight’s #sachat. I’ve heard this debate many times about how to leverage respect from administration for what folks in student affairs do. I wanted to share my opinion but was feeling a little stifled by the 10 million people involved in the chat as well as the 140 characters so I figured I would share my thoughts in longer form.

First, this discussion about developing “street cred” if you will for student affairs is essentially asking how you get administration (a nameless faceless entity, i suppose) to understand what we do. The inherent assumption is that they don’t understand or that they do understand but they don’t think it’s valuable. If we were going to stereotype the view of student affairs both within and without of student affairs, I believe that stereo type would be that we’re “in loco parentis” style baby sitters, we create play time for our kids, and we hold their hands and try to protect them from the big bad faculty. As with most assumptions, this is unfair. Some of the big bad faculty are ALWAYS looking out for students, and some within student affairs favor the approach of throwing students to the wolves over a touchie-feelie approach.

Personally, I find this argument about “admin doesn’t understand what we do” a little tired. If they don’t understand what they do, they’re no different from most people at high levels within organizations. They understand in theory what we do, but they don’t know in detail (isn’t this what that new TV show is about?). Here’s a realization that I think we all need: THEY DON’T NEED TO KNOW WHAT WE DO IN DETAIL. It’s not their job to know the details of our job, it’s our job to know the details of their job. I’m almost certain that most of us have only a basic idea of what our chancellors and VPs do on a daily basis even though we might like to think otherwise. To think that it’s their responsibility to understand every intimate detail of every job on campus is childish.

Second, without fail, the first answer to “how to gain respect” is assessment. Don’t think I’m about to slam assessment; I love assessment, and frankly I don’t think we assess enough. During my internship in grad school in student affairs assessment, the assessment reports that I looked at were PATHETIC and an absurd joke. Most departments in student affairs don’t take this seriously and most of these reports barely get looked at because they don’t actually assess anything that carries weight at a cabinet meeting. VPs might like reading anecdotes, but it’s hard to imagine a meeting at the VP level where an anecdote about how a students heart was touched is read; it’s not that they don’t care, it’s just that they have bigger things to handle. Another problem with assessment – as an old quote says “there are lies, damned lies and statistics”, sharing a bunch of stats does nothing because it’s so easily manipulated.

Third, I think student affairs speaks a different language than the rest of the campus. One of the main critiques you hear of faculty on research campuses is that they don’t have time for students because they’re doing research. However, I would frame it a different way – they’re doing research because that’s a huge part of their job – big difference. When we’re talking about “i had 15 students come to my workshop on getting along with my roommate” and they’re talking about 9 journal articles they got published, we’re really talking on two different planes.

So, what’s my point?

I think I’m essentially arguing for the progression of student affairs as a profession. We’re not that old, not nearly as old as the faculty profession and I think the discussion of “how to gain respect” evolves out of a desire to take our profession from something we can be proud of on the campus, not just in our staff meetings or in our student affairs christmas socials where we’re making christmas bags for our students with glue, yarn and glitter.

I have a couple suggestions, but I don’t think this is a comprehensive list.

  1. what does the faculty use to establish their value to a campus?  I argued on Twitter that faculty establish value by publishing, bringing in students (recruiting), research grants and revenue from research.  If I were to summarize these thematically, I would call it “money and prestige”.  My question is how can student affairs bring money and prestige to the campus through our work.  I think the answer will be different for each office so I’ll not make more specific suggestions.
  2. If we’re going to do assessment, it needs to connect directly with the mission of our office AND our division and meet scientific research standards.  Most of our assessment is a joke because we don’t know how to be research scientists.  Clearly this is time consuming, but if the profession is going to progress I think this is an area we clearly don’t bring our A game currently.  Do it right, connect with the standards of real research scientists or don’t do it.


22
Feb 10

On Citizenship

Greetings from data land with the Multi Institutional Study of Leadership results for my campus!

As my previous few posts have indicated, I’m strolling down a very long road with analyzing our rather large volume of data from this study.  Some changes in our Institutional Research department on campus have left me with more of a role in managing this step of the process than I had planned, so this is a little more than what I expected!

As some of you know, the MSL is structured around assessing leadership as structured around the values of leadership in the Social Change Model. (read about the theoretical frame for the study on the NCLP website!) This model offers values of leadership based on individual, group, and community values.

We were investigating the specific results along one of the values that is essentially important to our campus, Citizenship.  Our campus has invested a great deal of time and focus around building our commitment to a mission of service as indicated in our motto, “not to be ministered unto, but to minister.”  I’m working with a graduate student on our campus this semester on a project relating to community service involvement of men on our campus, so I immediately went to that area and looking at gender differences.

Rather than digging in to this aspect of the data, my eye strayed over toward the opportunity to look at other areas of campus involvement in addition to prior involvement in community service.  Instead, I went to look at the students who have cited prior involvement in campus programming organizations.

DING! DING! DING! We have a winner!

Students who cite prior involvement in programming organizations reported significantly higher results on Citizenship than our general student body. Their results are comparable to those from students who note prior community service involvement.

We knew this all along, didn’t we?!  Or did we?

I know that our programming board students are always among the most selfless students that I have the pleasure of working with.  I also know that their commitment to improving our campus has always been second-to-none after over 15 years of experience across a few institutions.

Knowing all of this, why have I never linked this to Citizenship before?  Conversations about citizenship on our campus typically only include discussions of service activities outside of our campus boundaries and various forms of political activism.  Yet, this study is telling me that the value of Citizenship is alive and well in our programming students to a comparable extent to our students involved in more typically termed “service” activities.  I am really not saying that all groups of students are meeting a “need” that is even comparable, but our students working in all of these capacities are telling me they understand Citizenship in a different way than our general student body.

If our field is emphasizing a focus on student learning and we see that learning about Citizenship is happening in campus programming, is there any hope of expanding a definition of service to include campus-based service?

I’m looking forward to your thoughts on this one.


13
Nov 09

Using Facebook Insights to Assess Your Facebook Page

Facebook Insights is an application available to all page
administrators.  It measures
exposure, actions, and behavior related to your page.  Insights gives you almost instant data––it is updated 12
hours after the end of each day. 
Available data includes: geographic and demographic information about
your fan base, a record of interactions (comments, wall posts, photo/video
views, “likes”), and the growth of your fan base over time.  All of this information is displayed in
the Fan Dashboard, and most of it can also be downloaded into a spreadsheet so
you can use the data in any way you see fit.

In Practice: How UW–Milwaukee University Housing Uses
Our Facebook Page

I created our Facebook Page
in July 2009.  After the first week of school, we were hovering under 100
fans.  Next, I harnessed the
influence of the Neighborhood Housing Office, who at that time had over 200
fans of their Facebook Page. 
Within two days of suggesting our page to all of their fans, we had over
200 fans as well.  By this point I
felt that our page had enough content and fan support/interaction to go
public.  So, when we launched our
new website on September 10, I included a prominent link to our Facebook Page
on the homepage.  Since then, our
fan base has grown at a modest pace of 3 pans per day, which should put us over
400 fans by the end of this semester. 
Facebook Insights gives me the tools to track this data easily,
especially if I take note of the dates when actions with potential impact on
our fan base occur (other page suggestions, new advertising strategy).

By downloading the Fan Demographic data from Insights, I can
see that 75% of our fans are between the age of 18-24 and 5% are 13-17 years
old.  So, it’s likely that 80% of
our fans are current or prospective students.  The remaining fans are likely staff members and
parents/family members –– yes, we have had a parent become an active fan of our
page!

Knowing who our fans are is not enough; to truly assess our
Facebook activities, I need to know if students are interacting with our
content.  From the Fan Interaction
data set, I can report on the number of total interactions with our content
(74), average number of interactions per post (1), total photo views (1711),
video plays (170), and comments on our content (30).  Astonishingly, our Facebook Page has had 3,875 page views
from 1,234 visitors since its creation. Soon, Insights will also allow page
administrators to see the Click Through Rate and Engagement Rate, which will
provide a clearer picture of how many fans are responding to your content.

Do you use Facebook Insights for assessment or reporting?  How have you used the data?


15
Oct 09

Sustainability in Student Affairs: What are you doing?

One of my personal areas of interest in sustainability.  This morning we had a staff meeting in another area of campus.  I left my reusable coffee cup at home asuming we would have ceramic cups at the meeting (sometimes our director splurges and provides us with drinks at these meetings), and I was saddened to find paper cups instead! I dejectedly used my first disposable cup since July!

This got me thinking about the things we are doing at Penn State to educate students about sustainability and whether or not they are actually making a difference in the long term. For example, two weeks ago was Penn State’s “Great Recycling Challenge.” This is a very exciting project where we encourage students to recycle as much as possible. Staff members from the Office of the Physical Plant actually audit the garbage for the week and determine which residence hall area recycles the greatest percentage of their recyclable trash. Students get daily updates on their recycling efforts, and there is a lot of friendly competition from the areas to see who can win the Great Recycling Challenge trophy. (For laughs, here I am with the trophy: http://www.flickr.com/photos/bck134/3945667667/ ) The area I supervise is the three-time defending champions . . . . but are my students more likely to recycle when they graduate? I don’t know (and I doubt it).

What sorts of sustainability projects are going on in the student affairs offices at your institution? How do you assess their effectiveness, and how do they contribute to student learning at your institution?


13
Sep 09

Introversion in student affairs

I had a conversation recently with our graduate intern about life as an introvert career counselor.  He's just starting his life as an introvert in the world of student affairs, and our conversation lead me to reflection of the earlier years of my professional life.

I told him to be aware and be intentional in his consideration of how his introversion is or is not an issue as he learns to navigate his professional life.  When I first transitioned from registrar-type work into my first counseling role it took me a while to understand my introversion in that generally-extroverted context.  In fact I think for about the two years I did a horrible job of balancing my needs as an introvert and the demands of the work.

As an academic counselor in a setting with ludicrously high advising ratios (try 5500 students to 4.5 counselors!), there was no down time, no time to re-energize, no time to contemplate or rejuvenate.  There was only go, go, go.  Next student, next student – typically 10 per day, in half-hour increments.  My evenings became more solitary.  My outlook, considerably more grumpy.

It took me years to understand why I was (and am) more productive in the morning than in the afternoon; why many of my ideas wouldn't be pursued because they didn't feel fully formulated until after the brainstorming session had ended; and why I preferred not to go out to lunch with my colleagues each day.

Now, not every student affairs position will demand this hyper level of interaction and outward-facing energy.  But, they all will have varying degrees of the same, and I believe that even for lower traffic student affairs offices there is a relatively steep on-ramp for the new introverted professional.

New professionals need to understand that these things are okay, while simultaneously learning that being in student affairs sometimes means putting on your "extrovert mask," even when it doesn't feel natural (it became entirely comfortable for me over time, I'm happy to report).  Here are a few other tips for the introverted new student affairs professional:

  • Don't feel guilty about needing time to yourself.  If that means foregoing lunch with colleagues or occasionally skipping out on that office social, do it.  But of course, you must strike a balance to make sure that you don't accidentally develop a curmudgeon label.
  • If you know the subject of a particular meeting, make notes and write down some of your thoughts ahead of time. It may help you to feel like you can participate more actively, having thought through the issues ahead of time.
  • Find the ways that are most accommodating for you to "become" an extrovert when it is needed.  Some skills like public speaking or working a room may not be natural, but you need to make them become comfortable. So, practice, practice, practice (then sit quietly for a while!)

As a profession, and indeed as a culture, we tend to prefer extroverts.   Extroverts often make more-immediate impressions and many qualities associated with extroversion are thought of as positive in the workplace.  But, our students are represented across the introversion/extroversion spectrum, as should be our student affairs professionals who will work with them.

Fellow introverts, I'd love to hear your thoughts and perspectives.

Gary Alan Miller

Cross posted on my blog


22
Apr 09

Student leader learning?

As the semester comes crashing to an end, you may be wrapping up programs and events with student leader teams. I found this thoughtful list of questions from Tim Milburn over at studentlinc. This semester, instead of asking your student leaders to evaluate the experience or program, ask them to complete a self-assessment. My peer mentors are hired in the fall for a calendar year term of service, so I may have them complete the questions now and once again in December after their fall experience.

  1. What did I learn as a student leader?
  2. What will I need to remember from my student leadership year?
  3. Which interactions with others taught me the most about how to work with people?
  4. What do I know now that I didn’t know a year ago?
  5. What am I better at as a result of this student leadership experience?

  6. How would I describe my student leadership experience in 100 words?
  7. How am I better prepared for the next chapter in my story?
  8. What would I have done differently as a student leader?
  9. If I had one hour with a group of newly elected student leaders, what would I want to talk to them about?
  10. What mistakes did I make this year and what did I learn from them?

  11. What do I hope to be remembered for as a student leader?
  12. How could I have done better as a student leader?

Completing a self-assessment allows student leaders to depart the experience with a greater context for their contributions and learning. Over at studentlinc, Tim has shared a pdf of these questions and other great student leader information.

Do you have student leader evaluation or assessment ideas to share?

  • Tags

    career Student Affairs 9/11 Community candidates Orientation Social Media Tips The Placement Experience engagement student affairs technology Graduate Students residence life technology how to student activities education Twitter Education Theory Education and Technology facebook Conferences Personal professional development leadership development Student Engagement Theory Placement Job Search video reflection leadership Uncategorized ideas Advice students interviews NASPA Higher Education Poll acpa higher ed #sachat college freshmen This and that Student Engagement

  • WP SlimStat