How to Analyse Employee Survey Comments Without Losing the Human Story

Employee engagement surveys produce two kinds of evidence: tidy ratings and the messy free-text comments that explain what those ratings actually meant. How you analyse employee survey comments — without flattening them into a word cloud or a few cherry-picked quotes — is a methodological question with direct consequences for the decisions an organisation then makes.

Why Open-Ended Survey Comments Carry Disproportionate Weight

Rating scales tell an organisation how big a problem is. Free-text responses tell it where the problem comes from and what it looks like in practice. A 3/5 score on “I feel supported by my manager” communicates dissatisfaction but not its source. The comment attached to that rating, describing, for instance, a team where one-to-ones are routinely cancelled because a line manager has absorbed dual responsibilities after a restructure, is what makes the problem actionable.

Qualitative data is sometimes described as the layer that gives quantitative findings their meaning. Braun and Clarke’s (2006) widely cited paper on thematic analysis argues that structured qualitative methods offer “accessible and theoretically flexible” access to participants’ perspectives, provided the analyst commits to systematic procedures rather than impressionistic summary. Research by Bakhshalian and Reddington (2017), set out in their book PEP® Dr: Unlocking the Puzzles of Peak Engagement and Performance, similarly emphasises that engagement is a contextual phenomenon: the same survey score can reflect very different underlying realities across teams, functions, or career stages, and the open-text comments are often the only place those realities surface.

Common Pitfalls When Analysing Survey Comments

Several analytical shortcuts routinely erase the very signal that open-ended questions are designed to capture.

Word frequency and word clouds. Converting comments into a bag of words identifies surface-level recurrence but ignores syntax, sentiment, and context. A word cloud that foregrounds “manager” cannot distinguish between “my manager is exceptional” and “my manager is the main reason I am leaving.”

Over-aggregation. Collapsing thirty heterogeneous comments into a single theme labelled “communication” produces a finding that is technically accurate and practically useless. Senior leaders receive a headline. They never learn that in one function the underlying problem is information overload, while in another it is the silence that follows restructuring announcements.

Confirmation in quote selection. When analysts approach free-text data with a conclusion already drafted, the comments become illustrations rather than evidence. Krippendorff (2018), in his foundational text on content analysis, describes this as a failure of inferential validity. The analysis no longer supports claims beyond the cases it has selected to illustrate.

Confidentiality stripping that removes meaning. Protecting anonymity is essential, but overly aggressive redaction can strip the contextual detail (team, tenure, recent event) that makes a comment interpretable. Better practice is to design analytical categories carefully enough that patterns emerge at a level of granularity that protects identity without erasing meaning.

Thematic Analysis as a Defensible Foundation

For most employee survey comment datasets, thematic analysis offers the appropriate balance of structure and flexibility. Braun and Clarke (2006; 2021) describe a six-phase process: familiarisation with the data, generating initial codes, searching for themes, reviewing themes, defining and naming themes, and producing the report. Each phase is iterative rather than linear.

Within this framework, two decisions matter most for whether the human story survives.

The first is the choice between inductive and deductive coding. Inductive coding allows themes to emerge from the data. Deductive coding starts from a pre-existing framework, such as the Job Demands–Resources model set out by Schaufeli and Bakker (2004) in the Journal of Organizational Behavior. In practice, survey comment analysis usually benefits from a hybrid approach. A deductive scaffold anchors findings to established engagement constructs, while inductive codes capture what the framework misses.

The second is the choice between semantic and latent coding. Semantic coding describes what is explicitly said. Latent coding interprets the underlying meaning. The human story tends to live at the semantic level: the specifics of what employees said, in something close to their own language. Moving too quickly to latent interpretation risks substituting the analyst’s narrative for the participants’.

A Practical Workflow to Analyse Employee Survey Comments

A defensible analysis of open-text survey responses typically follows a sequence that mirrors Braun and Clarke’s phases, adapted to the particular demands of organisational data.

Familiarisation. The first pass is slow and non-analytical: reading every comment, making marginal notes, and beginning to register the range of voices. Guest, Bunce, and Johnson (2006), writing in Field Methods, found that the bulk of thematic content in a qualitative dataset typically emerges within the first twelve transcripts. Survey data differs in that even low-frequency comments may carry high signal value. A single detailed account of bullying, for example, warrants serious attention regardless of how many other respondents raise it.

Codebook development. For larger datasets, or for surveys where findings will be revisited across years, a structured codebook supports consistency. MacQueen, McLellan, Kay, and Milstein (1998) set out a team-based approach in which each code is defined in writing, with inclusion and exclusion criteria, to allow different analysts to apply the scheme reliably. This is particularly valuable when the same survey is repeated annually and longitudinal comparisons are required.

Theme development and review. Themes are patterns of meaning, not aggregated code labels. They are developed and refined by testing them against the data. A theme that cannot survive a re-reading of the comments it supposedly captures needs to be split, merged, or reframed.

Preserving verbatims. Every reported theme should be illustrated by verbatim extracts, with enough context to be interpretable, and drawn from a range of respondents rather than the same three quotable voices. Reporting is where the human story is either kept or lost.

Tools: Supporting, Not Substituting, Judgement

Software such as NVivo, widely used in academic and applied research, supports the coding process by organising data, tracking code frequencies, and enabling cross-cutting queries. Survey platforms including Qualtrics and SurveyMonkey offer increasingly sophisticated text analytics, including sentiment scoring and automated topic clustering, which can accelerate familiarisation with very large datasets.

More recent generative AI tools can produce summary themes from thousands of comments in seconds. That speed is genuinely useful for getting an initial feel for a large dataset, but auto-generated summaries tend to produce plausible-sounding themes that are difficult to trace back to specific quotations, and that average out the distinctive voices a survey was designed to surface. Saldaña (2021), in The Coding Manual for Qualitative Researchers, argues that coding is a fundamentally interpretive act, bound up with the analyst’s judgement about meaning. Tools that remove the analyst from that loop tend to produce output that reads well but cannot be defended when challenged. A more responsible use of AI-assisted summarisation treats it as a first-pass familiarisation aid, followed by structured human coding of the underlying data.

Reporting That Honours the Data

The final step, how findings are presented to leadership, often determines whether the analysis has been worth the effort. Reports that consist only of headline themes and percentages lose the evidence base. Reports made up entirely of verbatim quotes overwhelm the reader instead. A stronger format pairs each theme with a concise definition, a quantitative indicator of its prevalence, two or three illustrative verbatims drawn from different parts of the workforce, and an explicit note of any counter-evidence or divergent accounts. The goal is to give decision-makers the material to act on, without implying a false consensus where none exists.

Frequently asked questions

Conclusion

Employee survey comments are among the richest data most organisations hold about their own workforce. Analysing them well requires treating open-text responses as qualitative evidence in their own right, not as decorative quotes attached to a quantitative report. A structured method, typically thematic analysis, combined with transparent coding choices, careful preservation of voice and context, and responsible use of technology, produces findings that hold up to challenge and can actually be acted on.

Organisations seeking expert support with survey design, comment analysis, and evidence-based reporting for senior leadership can draw on Elmira Bakhshalian’s workforce insight consultancy, which combines published research expertise with over a decade of applied work for clients including the NHS, the British Medical Association, and more than 150 UK organisations. Readers with an academic interest in the underlying methods may also find relevant resources through her academic research support service.