DDI (Development Dimensions International) is one of the many HR consultancy firms in the market that offers talent assessment solutions. One of their better known programs is the one-day assessment center test that puts applicants through a role-play under what they would label as "stretched" and "stressful" conditions. I recently was selected to participate in the assessment center program as the last and final selection stage for the Global-Asia Management Program administered by the Economic Development Board of Singapore (more on this program in my next post). Despite being skeptical about the true effectiveness of such a method in assessing an individual's qualities, I went ahead to prepare for my actual day assessment by putting in the hours to review the materials which were e-mailed to me about five days in advance. I was assigned to take on the persona of Marty Kane, the Area Director for North American Operations of a fictitious company, Global Solutions, Inc. This company offers a whole slew of robotics solutions across diverse industries.
Having read and thought through the required materials beforehand, I was confident that I will be able to nail the challenge on the actual day -- I knew what to expect at the assessment center; I had several strategic frameworks all written out to help me tackle different business issues; I understood the criticality of certain issues and the urgency of getting them all resolved as Marty Kane. I was all ready to jump right into the exercise.
On that morning, I arrived at the assessment center to find five other candidates who had also been shortlisted by EDB to participate in this exercise. Most of them were all from very well-known b-schools -- London Business School, MIT Sloan, Stanford, Oxford Said. (While I was acutely aware of how elitist Singapore is, I was pleasantly surprised to find myself in the unique position of being an underdog, given the lower ranking of my b-school.) Interesting start to the day. We were then quickly ushered into a conference room and were briefed by an immaculately-attired DDI representative on the string of events to take place. Nothing I didn't already know.
Shortly, we were asked to take up our positions in the room. Each room was deliberately furnished to resemble an office, with a laptop and other stationery on the desk -- except that this room came equipped with a videocam that points at you and surreptitiously captures every single movement of yours. I felt my privacy being intruded upon and felt at that instant, like a guinea pig about to be operated on in a science laboratory. Seriously, would any companies that treat their staff with integrity want to even allow them to be continually filmed for seven hours, all in the name of talent assessment? I frowned and quickly realized that my expression could have been recorded. I cast aside that thought and quickly got down to work.
I began to feel very uncomfortable and started to perspire profusely as I worked on my e-mails. No, not that the e-mails were intimidating, but it was simply the room -- there was little ventilation. I was in my executive jacket (hey, there were points for executive presence!), and the heat and humidity got the better of me. I refrained from wiping my perspiration thinking that this would portray me poorly but this didn't last long. Twenty minutes into the session, I decided to let go (screw those executive presence points) and just be myself. Which chief executive in the world would not perspire when it's hot? This thought sent me free, and I gained the momentum to tackle the e-mails as they came in.
Like what I had expected, there were about six e-mails sitting in my in-tray awaiting resolution. As the Area Director who was on the job on his first day, the number of e-mails sure made things seem urgent. I wondered to myself, how realistic is it for an person - regardless of designation - to be asked to look into critical issues on his/her first work day? Would any CEO worth their salt entrust a new director to handle a public relations fiasco that has severe repercussions on the share price of the company? Grudgingly, I went through all the e-mails and began to plan my day on how and which issues to work on first. Since my business plan presentation was after lunch, I reckoned that I'd be able to power through my business plan presentation during my lunch hours. I could then use the remaining time to focus on other outstanding issues.
The issues were plenty and ranged from HR issues (I needed to teach someone to coach a recalcitrant employee who used to be a star performer. How did that person even become a manager without knowing how to coach a subordinate?), to operational issues to power struggles (yes, another director obviously was already unhappy with Marty Kane, even before she had joined the company. What a b****). I paused for a minute and recalled the things that didn't work and leadership behaviors I loathed when I was working. I decided that if I were to be a leader, I would have to take a position on certain issues (especially since DDI had made the cases urgent by emphasizing that I would only have one day to handle all issues and would be uncontactable thereafter). I had to be very focused and clear in my communications -- and used this approach to tackle the issues as they came.
The e-mails - unsurprisingly - came in fast and furious and before I knew it, it was lunch time and I had to start working through my business plan presentation. I wanted to impress by inserting a graph but decided that I'd be better off using that time to craft my message. My overarching goal was to impress my supervisor within 15 minutes and get his backing for my proposal. I broke down my presentation to several key areas -- Market Analysis; Marketing Strategy; Financials; Operations; Execution Plan; Potential Risks. I was quite pleased that I had comprehensively formulated the problem (my b-school professor would be proud of me) and came up with a set of slides that touched on the key areas; I was equally excited to be making a presentation too. After one round of rehearsal, I was ready to welcome my assessor -- a snobbish lady who was to do the role-play with me. The entire presentation lasted for half an hour and the assessor asked some questions pertaining to my slides and at one point, I was a little taken aback when she asked questions on areas which I had already covered extensively in my presentation. (Maybe she was thinking about her lunch, I reckoned.) She also fired away some questions which she had prepared beforehand, which were not presentation-specific. While it wasn't a nerve-wrecking experience (trust me, hosting a dinner is more a million times scarier than this), I wasn't in the best of mood. Again, my attempt to show my enthusiasm during the presentation had only made me perspire even more profusely! I tried to maintain my composure and put on a grin, despite beads of perspiration running down my cheeks. Gosh, what is wrong with the ventilation of the building? I almost wanted to scream to literally let off some steam. But seeing the voice recorder (yes, they record every single thing about you) on the table, I stifled my scream and soldiered on.
After the business plan presentation, it was back to my e-mails - and more e-mails. At some point, I began to even laugh at how ludicrous the scenarios were. This ironically helped me relax and I actually felt good typing out my responses to each of those e-mails, even proffering advice based on what my real-life work experience. My day abruptly ended with an interview, with the same snobbish lady whom I met earlier. She started off asking me questions pertaining to how I went about organizing my day, who I would actively engage in the company, which issues were deemed most important to me (I could hardly remember all the issues and the names of everyone involved and I naturally fumbled to come up with a response. I quietly suspected that this question was more a test of my memory than anything else), before moving on to questions about how I have handled certain situations throughout my career. Honestly, I was surprised and irked when she started asking me those questions, many of which had already been asked twice in a video and a phone interview I have already had prior to coming to the assessment center. The feeling was akin to being in an operationally defunct hospital, with a patient having to answer the same questions asked by three different doctors. They probably have never heard of resource streamlining and optimization.
Next, I was moved to a conference room where I met up with the other shortlisted candidates. I was late by about 10 minutes (due to the interview) and I went in not knowing what to do. I quickly reorganized myself and realized that we had to - as a team - come up with some suggestions to the problems that the fictitious company was facing. It was interesting to see how different people behaved in a group setting, especially with an alpha male candidate clearly dominating the discussion. Different people had different opinions and I could not see the discussion converging. What a déjà-vu scene! I thought I should chip in and get everyone back on track -- and thought I did well by not dominating the discussion though the team failed to achieve any consensus at the end of the discussion. There were some DDI assessors in the room too, and we were judged for how we behaved and what we said during that 20 odd minutes.
The day soon ended and I was exhausted from all the perspiration. Mentally, it was not as challenging as I thought it would be -- well, it was all fictitious and there was nothing to stop me from coming up with out-of-the-box "solutions" since there was simply no way to assess whether that solution would be effective or otherwise. I was relieved that I could remove my jacket and breathe easy, and actually motivated by my interactions with those applicants from the better schools. This vindicated my view that students should not be judged solely based on the prestige of their schools. A great confidence booster !
About two weeks later, I heard back from the Global-Asia Management Program administrator -- I was rejected. Oh dang. What did I do wrong at the assessment center? I quickly fixed an appointment to speak with a senior consultant from DDI to get a better sense of how I was assessed. Surely, if I had successfully advanced to the final selection round only to flub it, the assessment center must have given me nothing but a less-than-favorable report? I couldn't wait to hear it from DDI -- and grill them on their methodology.
A senior consultant prefaced the feedback with several caveats. According to him, the results of the assessment were based the collective efforts of three DDI staff, with one reviewing e-mails and others giving their feedback based on what they saw and heard throughout the day. While there was a list of attributes that they checked against, I was only briefed on two key strengths and two key weaknesses. I received praise for being task-oriented (the evidence being that I was very clear in my instructions and in my e-mails. Yes, I have achieved my goal to avoid bad writing and dishing out nebulous instructions) and for being someone who is not afraid of taking on challenges and new responsibilities. I displayed a strong go-getter character and showed courage at tackling various tasks and was adamant in resolving them. (Perfect observation that matches my goal to be an enterprising innovation chief!). He also said that I was very analytical and could make great business analysis and judgement and was adept at making sense of large data sets. So what went wrong?
Going on to what DDI called "developmental areas" (a euphemism for weakness), I was told that I could do much more to "involve others in areas where I am able to ensure mutual outcomes"; I should also consider ways to better enhance my ability to "lead people to change". Wait, what was he saying again? And so he explained that I was authoritative (excuse me, isn't this how a leader should be like given the criticality of the situation?) and was not as effective as they would like me to be when engaging others to come up with mutually beneficial outcomes. He then went on to clarify that by "leading people to change", he meant that I was being prescriptive to only my staff. (I have a big issue with this -- why be so pompous as to string together power corporate words like "lead", "people", "change" when all they meant I was being prescriptive to my staff? This makes no sense.) I clarified that I reacted according to the circumstances of the case, based on my interpretation of the instructions given. I was bemused when he added that I had great people skills and was a strong communicator. Does this point not conflict with their observations of me being authoritative and prescriptive? Was there a misinterpretation somewhere? While I may seem like I am resisting negative feedback here, I would like to add that feedback given should be consistent and clear and supported with right evidence and observations. All they could be quote was an example through their 20-minute long observation and several "themes" they drew by reading my e-mails. Pathetic. Where's the robustness in their assessment?
My experience shows that HR consultancy companies that charge a few grands to allegedly provide a systematic solution to gauge staff potential are not worth their salt. It's all an exercise in futility and an utter waste of precious resources for all involved. Unless those companies could address the following issues, you - and your company - would do better to think twice before using an assessment center to evaluate your future hires.
1. Unrealistic. Each candidate is put through situations that purportedly resembles real-life situation. But really? Which company would put a new hire on the frontline on their first workday? Is a seven-hour assessment, based on the arbitrary evaluation by three strangers for a short time period enough to accurately gauge the qualities and effectiveness of a person? Is disallowing all forms of communications - except e-mails - representative of the modern, highly connected world? Are e-mails even a good proxy of someone's interpersonal skills? Is a short 20-minute discussion any effective at determining how someone performs at work? If not, how does DDI even achieves its goals to help companies "reduce hiring mistakes"? Given that behaviors are better assessed over an extended period of time, how does a short seven-hour assessment be any more reliable than a manager's evaluation of someone's performance and behavior over the course of, say, a year or several years? Would past performance not be a better indicator of future performance than the results from a make-believe environment that is anything but representative of the real world?
2. Inaccurate. There has been a lot of research done on how e-mails could give rise to misunderstanding. Surprisingly, this point was lost on DDI when one of their main judging criteria was e-mails. The assessors were looking out for "key themes" and a balance of several qualities in the e-mails candidates sent. Hardly anything was said about the content and quality of the ideas in the e-mails. If e-mails were the least effective way to resolve any HR issues, then why is a HR company using e-mails as a way to judge how a candidate would potentially interact with other people in the future? I find it incredulous that after all these years when I was lauded for my ability to lead teams to deliver solids results, I was sadly evaluated - based on eight e-mails I sent - for being unable to "lead people to change". Seriously, DDI?
3. Ambiguous. I thought there is a structured way to finally measure a person's professional abilities. This turned out to be a huge disappointment. The entire assessment center evaluation was based on the subjective perceptions and interpretation of three strangers. How much experience do those assessors have in working with top executives who run for-profit organizations? Are they the best person to partner any organizations in making hiring decisions, knowing that those companies are going to place a high emphasis on the outcome of their evaluation? Why did DDI - a company that has been around for 40 odd years - not even think of coming up with a scale to assess how strong or weak each quality is? Just like a strength can be a weak strength, a weakness can also be recessive. Definitely ample room for improvement here. The ambiguity of the evaluation methodology is a huge flaw.
4. Biased. One key bugbear many employees have is how biased a performance review process can be; in a similar vein, it is often hard to isolate the biasedness that recruiters tend to have when interviewing a candidate. While the process of putting candidates through a screening process like an assessment center may seem at first glance a way to finally overcome those biases, my recent experience showed that it is nothing revolutionary. Given that the assessment remains a highly subjective exercise by three other people who have their biases, prejudices, personalities, likes and dislikes, how is anyone to be certain that the results are reliable and free from biases? If not, how does it overcome those biases? Who's to say that there will be no evaluator who will dominate the discussion and drive the outcome of the final report based on his/her preconception? If the results are likely to be flawed, how then does it justify the integrity of its evaluation of any one person?
Such assessment centers may help complement the results of a manager's evaluation, since the manager has the advantage of already assessing their staff prior to receiving the results from the assessment center. (Still, any effective managers should trust their own judgements of their own staff rather than entrust the fate of their staff to a team of highly-paid strangers.) However, given how much these companies charge, and the attendant shortcomings of the entire process, I'm bemused why companies are willing to fork out huge sums of money to make hiring decisions, which if wrongly executed, will have a detrimental impact on a company's future strategic growth. Until such a time when the process could be made objective, assessment centers lack the credibility they need to develop into an accurate talent measurement resource.
