Meetings 2.0

"Audience Response" is a misnomer

If you plan meetings or speak at them, you've used an "Audience Response System" (ARS) that enables a speaker to ask the audience a multiple choice question, and show the results live.   You may have also discovered that once the novelty wears off, this experience can fall a little flat.  Surprisingly, I haven't found much on the web that discusses effective uses of ARS in meetings (although here's a related article on its effective use in classroom settings)  So I thought I'd share some of the guidelines we use at nTAG to help craft a satisfying ARS experience.

The key point is that the technology should be called "Speaker Response", not "Audience Response"  Often, when I attend meetings, I see a speaker put up an ARS question, show the results, say something like "OK then...", and move on to his/her next bit. The  power of the technology is not that the audience's choices dynamically affect a giant, dancing bar graph in the front of the room, however (although that is very cool). It's that their choices effect the speaker.   So joke questions -- or questions with one good answer -- that don't inform the speaker in any way or change his/her focus should only be used in limited doses.  Here are some other ideas: 

1. Let the audience vote on what they want to hear the speaker talk about.  This is an easy one to do, because all speakers have "bonus" content they can cover if there is interest.
2. Use quiz-type questions to assess audience understanding, and tailor content appropriately.  This can be done pre-test style, to gauge an audience's understanding coming in, or post-test, to see if they "got it", or need some additional help.
3. Use questions to reveal pluralistic ignorance -- the phenomena people wrongly believe they are alone in having a particular opinion or character trait that is actually more widely held.  For example, Business Networking Guru Diane Darling and I did a workshop where we asked people to rank themselves between "Extreme Introvert" and "Extreme Extrovert".  Many  people were surprised to discover that they weren't an introvert in an extrovert's world, as they had feared -- they were an introvert in an introvert's world.  Of course, the anonymity afforded by ARS helps people pierce pluralistic ignorance by expressing their opinions. 

Certainly, I'm just scratching the surface here.  Please  speak up about your experience with these types of questions, or other suggestions for better ways to structure ARS.

Who Were the Players at EmTech 08?

What types of people do you talk to at an event that make you feel "Wow, I got a lot of value out of networking there?"  Based on a preliminary Event Catalyst Analysis of EmTech 08, Speakers and VIPs seem to be the "Network Nuggets" that people prized.  Conversely, interactions with sponsors or members of the elite Tech Review 35 did not drive networking satisfaction (and I'm not bitter about never having been chosen, back when I was <=35). 

Data digression: We determine this by regressing people's survey response about their networking satisfaction against data about what types of people they exchanged digital cards with.   This gets around the attribution problem -- not to mention the grammatical problem -- of trying to ask people "What types of people that you networked with did you get value from?"

Of course, the trouble at EmTech -- and at many events we do -- is that the speakers don't circulate enough.  If you check out the MixRay from EmTech below, you can see that speakers were 3X more likely to interact with another speaker than one would expect by random chance and they were 30% less likely to interact with a regular attendee.  So at next year's EmTech, we need to do more to get  popular speakers like Scoble to circulate -- since we know this adds real value for attendees. 

And by the way... I reported that last year, women networked at the event more than men -- as in, they exchanged more cards.  It was still true this year.   I need to explore this further with other event data sets.

*** Special thanks to the folks at Tech Review for being willing to be so open about their event.  All data is good data when it comes to making an event better.

The Two Types of People at EmTech 08

There seem to be two types of people at the MIT Emerging Technologies 08 conference going on now.  The type who use PCs, watch FOX news, eat junk food, drink beer, and like Angelina Jolie versus the type who use Macs, listen to NPR, eat organic, drink wine and like Meryl Streep. The full details are in the table below.  Are these two different types of geeks?

Dimension Type 1 Type 2 Computer PC Mac News FOX NPR Food Doritos Organic TV Brady Bunch Partridge Family Coffee Dunkin Donuts Starbucks Actor Angelina Jolie Meryl Streep Drink Beer Wine

These two types of people emerged from some survey data people are entering in to their nTAGs to help them network. I did some multidimensional scaling on the live survey data -- about 100 people had answered the questions at the time -- and produced the following image.  The algorithm attempts to locate two answers close together if the same people who chose one were also inclined to choose the other.  I added in blue lines between answers that were particularly well-correlated, and used these to determine the above clusters.

The freaky thing is I fit the PC stereotype almost entirely -- except I'm an NPR person.  Maybe I should switch to FOX. Do you fit one of these stereotypes?

Are Men or Women Better Networkers?

Are men or women better networkers?  As my son used to say to every question when he was around 5, "The answer is 'Nobody Knows'" (I think he heard that at a Museum of Science Planetarium show).  Of course, when you're older, you realize the correct response is "That's not a well-formed question."  What I do know is that at last year's Emerging Technologies Conference @ MIT, women used their nTAGs to exchange more business cards than men did.  It wasn't by a huge margin -- they exchanged about 1.5 more cards per person on average -- but it was statistically significant (p<.01).   You can also see from the social network diagrams below that women were more "central" at the event.  nTAGs are being used at this year's EmTech conference, and it starts today.  I'll keep you posted on whether this phenomenon persists across years.

The blue dots in the above diagram are men.  Look at those disconnected doormats in the outer circle.  Why did they even both coming?

The blue dots in the above diagram are women.  When you compare the diagrams, you can see the women are clustered more tightly toward the center, reflecting their greater centrality at the event.  The raw numbers reveal when you calculate all the "shortest paths" in the social network between each pair of people at the event, the average woman was on 772 shortest paths -- the average man was on just 487 of them.   Ergo, the women are playing a larger role in connecting the community.  By the way, the above diagrams are based on data collected by the nTAGs on who standing in front of whom (ostensibly talking) -- they're not tied to actually exchanging business cards. 

Do you agree? Do you think we'll see this again this year?   The answer is...

Face-time vs. Facebook

Having spent some time on Facebook the past few weeks, I'm feeling there is something subtly broken about it.  It's not just that it's the arch-nemesis of the unoffical "Saving Face" movement -- at least in the eyes of Dentyne (and I have stomped on one or two intimate face-to-face moments by reaching for my Facebook iphone App).  It's that it doesn't make it easy to know what other people in my circle know and see what they see -- it doesn't have good support for building a Meeting Of the Minds (MOM), aka a shared understanding.  Or worse, it creates the illusion of a shared understanding where there isn't one -- a shared misunderstanding.   Creating a MOM is  the thing that face-to-face is really good at.  Facebook has a ways to go here. 

Let me get specific:  There's a  cool feature on Facebook that has gotten a lot of attention called a "News Feed" -- it keeps you updated on all your friends' activity on Facebook: who they "friended", what groups they've joined, what links they've posted, etc.  Here's a snapshot of my current newsfeed.  Apologies to any friends who feel their privacy has been violated, but the content  seems pretty  innocuous to me.

So now I know that my elementary school chum Cyndi Headley isn't wearing flannel today because she overheated in Tucson yesterday, Jr. High pal Levi Phillips is now married, and my graphic designer colleague Josh Silverman joined some group called CIRCUS.  Years of reading "news group" type posts might also make me think that Cyndi, Levi and Josh now know these things about each other, but that would be a grievous mistake.  This feed is prepared just for me -- no one else is seeing just what I'm seeing.  It turns out that Levi and Cyndi are linked on Facebook, so they probably know about each other's posts, but Josh doesn't, and I'm left not knowing what other people know.  Now, when Josh later makes some comment about flannel, I might mistakenly interpret it as a reference to what Cyndi said, or I might say something to someone else that will be misconstrued because they don't know what I think they do.  The social knowledge fabric has been torn. [Rereading this, I'm realizing its another version of the age-old rant against the "Daily Me" personalized newspaper, which of course has become ubiquitous]

Now, you might say "This is your fault -- be a better reader", and that's a fair point.  People with more Facebook experience have probably learned to make better judgments about the social knowledge of who has seen what.  However, the medium will also need to grow to provide more cues about this.  This is one of the things that face-to-face does so well.  Check out this picture of an audience at a performance:

In an instant, the woman on the left and the man on the right have seen the same performance, seen the other's reaction, and seen the other see their reaction.  They have effortlessly established the content of the peformance and their feeling toward it as mutual knowledge.  It sounds abstract, but I think we've all experienced the power of it when laughing at a funny moment at a performance when we were drawn to look over at a friend (or soon-to-be friend) to establish the reaction as mutual.  Yes?  Online media isn't there yet.  Not clear if it ever will be -- after all, they have seemingly perfect virtual reality on the Holodeck in Star Trek: The Next Generation, and yet they still risk life and limb to actually  cross the universe to meet people in the flesh.  There's just something about face-to-face...

Saving Face

Lots of good blog talk on how as travel costs & headaches become more unbearable, and online discussion tools get better and more ubiquitous, meetings need to get better in order to clear the bar.   For example, Seth Godin says there better be a lot of engagement and interaction to make face-to-face meetings worthwhile;  Corbin Ball insists audience members need to engage directly with speakers in real time.   Because of the threat that face-to-face meetings might drown under a tidal wave of impossible coordination and logistics,  I think we should call this project "Saving Face."  It has two steps:

1. Figure out the powerful activities / behaviors / engagement patterns  that are unique to face-to-face meetings
2. Figure out how to make more of that stuff happen in face-to-face meetings.

I'm going to tackle  #1 in this post, and save #2 for the next one.  For #1, I'd say there's general agreement on the richness of the "lower layers" of  face-to-face communication.  For example:

1. Richer conversation stream:  The existence of gaze awareness, gesture, vocal range, and conversational props (things we can point to and talk about) in face-to-face environments all contribute to a much more nuanced conversation
2. Low latency:  The simultaneity of face-to-face means that conversational turns can literally take an eye-blink (or an eye wink), allowing for faster convergence around a subtle idea than can happen online. 

There's less agreement about why those nerdy communication features really matter on a human level, however.  Kathy Sierra says in a thoughtful post on the SXSW conference that face-to-face is uniquely good at creating inspiration.  Live performers would argue that face-to-face has an intimacy, immediacy, and a theater-of-cruelty ability-to-confront that communication at a distance lacks. 

For me, the key  human-level thing supported by Face-to-face's richer lower layers is its ability to foster a true "Meeting Of the Minds" -- let's call it a MoM because it's just about that important.  A MoM is a joint state between people who share an understanding of something, and feel that they share it.  Academics call it a Shared Understanding.  It's the feeling you have when you've just discovered someone loves the same obscure movie you do for the same obscure reasons.  It's the feeling you have when a comedian jokes about something that you thought was idiosyncratic to you, and the whole audience laughs.   It's the feeling you have after an argument with a group of people where you've finally uncovered an unspoken assumption you were making that was not shared by everyone. 

Face-to-face meetings are uniquely good for creating MoMs.  I would argue that it was the MoM at SXSW that was responsible for the inspiration that Kathy felt.  Meetings alone are not inspiring, but Meetings of the Mind are. 

Next post, I'll discuss my thoughts on #2: What you can do turn a meeting in to a MoM. Until then, please join in me outlining the "Saving Face" agenda.  Can we turn it in to a manifesto?  I've always wanted to be involved in a manifesto.

The Cure for Clubitis

The #1 disease at corporate events is "Club-itis"  -- the phenomenon where attendees travel across the world at tremendous company expense only to hang out with people from their same home office (or home team or job level).  Most of our clients' events suffer from this dreaded disease in one form or another, and it's often what motivates an event owner to get in touch with us.  For example, at an event we did recently, the owners were introducing their product to a few hundred high-potential customers they flew in for a luxe, three day, all-expenses-paid event.  In this case, they wanted to make sure their sales team spent a lot of face time with their customers, and didn't just "talk amongst themselves".  So we worked with them to incenti this behavior.  And how did it turn out? 

We use the visualization below -- which I think we should call a mixRay -- to diagnose Clubitis.  It shows the amount of interaction between different types of people at the event.  Check out the circle in the bottom row (labeled "Client") and 4th column (labeled "Staff - Sales") that says "+15%".  That means that there was 15% more interaction between clients and salespeople than would have been expected if people were just mixing randomly, which is  good news (for a more complete description about how to read the circles, see below).  Also good news is the circle in the 2nd row down and 4th column over that shows interaction within the sales group was "-33%" -- that is, there was 33% fewer interactionsh between two salespeople than one would expect if people were mixing randomly.  Finally, note from the number in parentheses below the bottom axis that each member sales staff interacted with an average of 36 other people at the event, which is almost 50% more than the next highest group (clients).  From all this, we conclude that sales people were not mixing randomly -- instead, they were making a concerted effort to interact with the clients (and also vendors, according to the data).  Bottom line: we successfully managed sales team Clubitis.

To understand how read the circles:  The black circle represents how much interaction one would expect by random chance and the colored circle represents how much actual interaction occurred.  For ease of comparison, the size of the black circles is held constant across the graph, and the colored circle radius is scaled relative to it.

Note that Staff Clubitis was rampant, but also benign: there were 170% (1.7x) more Staff to Staff interactions than would have been expected by random chance.  However, this didn't cause alarm -- staff working together is pretty normal at events. 
 

Great event. I’m not coming back.

At a recent user conference that used nTAGs, excellent sessions were a powerful event catalyst (or performance driver, in business intelligence terminology). Specifically, for every session an attendee rated as excellent, their stated overall satisfaction, awareness of product features, likelihood of sticking with the product and likelihood of adopting additional product modules all increased about a tenth of a point on average – a statistically significant amount. Those were some hot, hot sessions that meaningfully advanced most key event objectives -- except making people want to come back next year. That’s right: there was no correlation between attending excellent sessions and intending to return the following year.  So the irony is: one of the event objective that the organizers cared the most about – driving return attendance – was not impacted the by the part of the program they spent the most time on – the sessions.

The “unstickiness” of great sessions may be part of an emerging pattern. We’re seeing the same thing at other user conferences we’ve analyzed, including the one I discussed here. We’re working to collect better data about why this is, but if I I had to guess, I’d say it’s because attending a great session doesn’t provide any particular draw for coming back to the event next year (except the vague sense that there might be more great sessions). Specifically, there is no strong desire to come back to see the exact same great session next year. The same is not true for great people, however.

People will come back to the same event to see the same people year after year in order to refresh valuable connections.  The data reflects this. At the recent user conference, the more interactions attendees had with alumni (return attendees), the more likely they were to say they were going to come back. Of course, our clients – like most event owners – knew that their attendees cared about networking. However, before this analysis, they didn’t quite get that – in terms of deciding whether to come back next year -- this was all they cared about.

Here are a few event planning lessons we can draw from this:

• To avoid the “Great event – I’m not coming back” phenomenon, spend at least as much energy on creating networking opportunities for your attendees as you do creating a good session program
• Brand your sessions, as a way to make them stickier. Promote memorable speakers, and bring them back year after year. Repeat memorable formats every year -- like “Executive Shark Tank” and “Roadmap Roadkill” (ask me for more details on these) -- that your attendees will want to come back for.
• Survey your audience about your success on all your event objectives, and do the necessary analysis to reveal which catalysts drove each of them. If our client had asked about awareness of product features and overall event satisfaction without asking about likelihood of return, they would have missed a key event catalyst.

 

 

Taking Care of Newbies at User Conferences

Here's some proof that first-time attendees need help getting socially integrated at events, and a few suggestions about how to provide it.  The following two  social network diagrams are from a user conference using nTAGs that is completing its first day as I write this.  For those who haven't seen a social network diagram, all you need to know is:

1. Each circle is a person, each line connects two people who have spoken to each other
2. The graph algorithm draws people who have interacted closer to each other
3. The attendees that serve as the hubs/connectors of the event wind up getting drawn toward the center of the graph.

The following graph highlights the alumni -- those who have attended this event before -- in red.  So you can see that the alumni are relatively "central" at this event:

Contrast this with the first-timers,aka the newbies, shown in red below   Rather than clustering in the center of the graph, many of them are out on the periphery of the the social network, no doubt feeling disconnected.  If you want a more quantitative comparison, the alumni have an average  betweenness centrality of 43, while the newbies average is 28.

Based on this analysis, there are a few different things I'll recommend this client do ASAP at their event. 

1. Feed these results back to the attendees using a  "Community Mirror".  Often, just making a community's networking patterns public in real time is enough to gently shift them.  In this case, helping the alumni  see -- and see that others can see -- that they're marginalizing the newbies might be enough to get them to reach out to them more
2. Reward alumni for seeking out newbies,  for example, by giving them extra points with the nTAG Incent application.

Event Catalysts

Vladmir:  It's the start that's difficult.
Estragon: You can start from anything.
Vladmir: Yes, but you have to decide.
-- Waiting for Godot

So I've decided to start with event catalysts. 

Value for attendees at events is not evenly distributed.  For example, at a user conference, you don't learn about a product in a steady stream over the course of a few days.  Instead, there are long stretches of time where you learn nothing, punctuated by a set of moments where your knowledge lurches forward.   At nTAG, we call the interactions and experiences that trigger these advances event catalysts, and we try to help event owners create more of them at their events.  That involves two things: first, doing statistical analysis on data we've collected with nTAGs to figure out what interactions and experiences are associated with delivering value; then, using the nTAGs to "program" an event that creates more of those experiences for all attendees.    Here are a few examples:

At a sales training event, one of the event owner's objectives was to educate the sales team about advantages of their products over their competitors.   We discovered that the more time an attendee spent interacting with some external sales people who were at the meeting, the more those attendees said they learned about their products benefits versus the competition -- a statistically significant relationship.  This makes some sense:  the external sales people sold the company's products as well as competitors', and were apparently effective at communicating the salient differences to the internal sales folks they talked to.  Based on this, we advised the event owner to bring in more of these external sales folks for the next event, and to use some nTAG activities that would drive more interactions with them.

At another user conference, we discovered that interactions with people who had attended before (alumni) were powerful catalysts in making attendees -- both newbies and alumni -- want to come back the following year.  In fact, for alumni, interactions with other alumni were the only catalyst driving intent to return next year.  For newbies, the number of sessions they rated as excellent also correlated with their intent to return, but not so for alumni.  For them, it seemed to be  all about refreshing previously-formed connections (or at least networking with other alumni).  This led us to suggest several ways of enhancing the networking opportunities for next year's event.

I don't want the above examples to sound too clinical, so let me share some unabashed enthusiasm for what they represent.  Both examples offer the first steps toward hard proof of the value of social networking at events -- previously considered a "soft" benefit.  With data collected from the nTAGs, we were able to show that networking with certain constituencies drove the key objectives of the event owners.  This is crucial for making sure event owners preserve time for "off grid" interaction at events -- what happens outside the formal sessions.  If this is only considered a "nice to have" benefit, it will always be endangered by creeping "on grid" time.  I'll write more about off grid/on grid interaction later.  I'll also write more about the kind of "closed loop event improvement" that we're starting to demonstrate, with the tag's dual ability to help figure out event catalysts, and to effect change at an event.   

* Thanks to Paul Litvak for doing the statistical analysis on event catalysts.