Feedback - The Breakfast of Champions

In 1999 I prepared a 90-minute presentation for a security educator’s conference. It was my first professional presentation and I was eager to do well.  I worked on the presentation for several months—researching, analyzing, editing, and creating custom graphics.  The printed research comprised several feet of documents, covered in highlighter marks and handwritten notes.  I had rehearsed for days with a stack of note cards, a tape recorder, and a stop watch, and finally, I was prepared to face a live test audience: my mother. Slide after slide, she patiently listened. She laughed in the right places and didn’t yawn once.  When I was done, I asked her what she thought, expecting to hear praise for the quality and the hard work I had done.  She was quiet for a moment.  Then my wonderful, encouraging, you-can-do-it cheerleader mother said something like, “That’s very nice, dear, but it’s not useful.”  

My jaw dropped.  I had put so much into that presentation, and I was scheduled to deliver it the following week.  When I was able to speak, I asked for an explanation.  I had, she said, spent 95% of the presentation outlining the problem in excruciating detail, but only 5% offering solutions.  

It was like the classic joke about the adventurer who was lost while traveling in a hot air balloon.  She finally gets low enough to shout a question to someone on the ground: “Where am I?”   The person on the ground looks up and replies, “Oh, you’re about 300 feet up.”  The balloon rider then realizes that the person on the ground is a consultant because the information given was completely accurate, but useless.

Thanks to my most-loving critic, I reworked the entire presentation to focus on providing value—with the bulk of the time used to offer practical solutions. Because she did not leave me “in possession of my error,” the talk was well-received.

Another time that feedback helped improve something I was proud of happened when I received a frustrated email from someone working at a retail client where our security awareness posters were displayed. The poster showed a fox wearing a chicken suit asking a chicken for the coop password.  The poster’s tag line read: Social Engineers will say anything to get confidential information.

“We sat around the break room,” the woman said, “and tried to figure out what that poster means.  The best we could guess is that it’s referring to plants —  spies that management has sent to get information from the staff.”

“What? No.  It’s about social engineers—bad guys who want to steal your passwords and account information.”

The problem here is one described by Chip and Dan Heath in their book, Made to Stick: Why Some Ideas Survive and Others Die.  They introduce Elizabeth Newton, a Stanford student who earned a Ph.D. in psychology for a study where she assigned people to one of two roles: “tappers” or “listeners.” Tappers were given a list of twenty-five well-known songs, such as “Happy Birthday to You” and “The Star-Spangled Banner.” Each tapper was asked to pick a song and tap out the rhythm to on a table.

The listeners were supposed to guess the song.  In the experiment, 120 songs were tapped out. Listeners correctly identified 3 of the 120 songs. Before the listeners heard the tapping, Newton asked the tappers to predict the odds that the listeners would guess correctly. They predicted that the odds were 50 percent. The book shares the results: “The tappers got their message across 1 time in 40, but they thought they were getting their message across 1 time in 2.  Why?”

The answer is that when a tapper taps, the tapper is hearing the song in her head, but the listeners can’t hear that tune. They hear disconnected taps, “like a kind of bizarre Morse Code.”

“Tappers are flabbergasted at how hard the listeners seem to be working to pick up the tune,” say the Heath brothers. To the tappers, the tune is obvious. “The problem is that tappers have been given knowledge (the song title) that makes it impossible for them to imagine what it’s like to lack that knowledge. When they’re tapping, they can’t imagine what it’s like for the listeners to hear isolated taps rather than a song. This is the Curse of Knowledge. Once we know something, we find it hard to imagine what it was like not to know it. Our knowledge has ‘cursed’ us. And it becomes difficult for us to share our knowledge with others, because we can’t readily re-create our listeners’ state of mind.”

The security officer who chose that poster had chuckled over it. He knew what a social engineer was.  So did we (and now most people do), but at the time, the retail workers at his company had no idea.  The frustrated woman who gave me the feedback had another tip.  She noted that the poster didn’t have a call to action.  It didn’t say what to do about social engineers (whatever those really were).  We modified that poster based on her feedback and it’s still one of our most popular designs. 

Recently, we developed a course on the General Data Protection Regulation (GDPR), a new regulation that applies to all companies—anywhere in the world—who collect, process, or store any data belonging to EU citizens.  The course included an animated sequence with a song about data breaches.  (When you’re late for the train / And drop papers on the lane / Data breach! / When you copy data to the cloud / Turns out you shared it with a crowd / Data Breach! ♬)

Prospective customers eagerly viewed the demo, but few licensed the course.  Perplexed, we asked for feedback.  Most viewers said that they liked the course, but it just wasn’t right for them.  When we asked for more specific feedback, one client told us that the cartoon animation we used with the data breach song was just not “corporate enough.”  When we removed that song segment from the course, clients began to license the revised course.  

Our Favorite Breakfast: Feedback

Obtaining, listening to, and responding to feedback from clients is vital to developing useful products.  Feedback is our favorite breakfast.  Please share your thoughts about our products or security awareness in general. We love to hear what we’re doing right and what we could do better.

We invite you to:

• View a demo of our courses (Awareness for All Hands, awareness for IT Professionals, or GDPR 
• Receive a sample newsletter 
• Take a look at our security- and privacy-themed posters 

Then tell us what you think.

• Do the materials suit your organizational culture?
• Are the topics the ones that are most important to you?
• Is there something you need for security awareness that you don’t see here?