AI analysis of open-ended comments is the classification and summarization of free-text responses in employee surveys by topic, tone and the suggestions they contain, using language models and text analytics methods. Work that can take weeks by hand can be done in minutes. However, AI can miss context, irony, organization-specific terms and rare but critical comments; the soundest approach is therefore a process in which an expert builds the coding framework, AI performs the classification and a human validates the results on a sample.
- AI can classify thousands of open-ended comments by topic and tone in minutes; doing the same work by hand can take weeks.
- It can miss context, irony, organization-specific terms and rare but critical comments, so the results must always be validated by an expert.
- In the soundest process, a human builds the coding framework, AI does the classification and accuracy is measured through human review of a sample.
- Because comments may contain personal data, they should be stripped of identifying information before being sent to an external AI service.
Why AI?
Open-ended questions reveal the reasons behind the scores that closed questions show. But in an organization with several thousand employees, even a single open-ended question means thousands of comments. Reading and classifying these comments by hand takes weeks; in most organizations it is either not done at all or limited to reading a few striking comments.
AI is changing this picture in three ways:
- Speed: Thousands of comments can be classified in minutes, and the findings can be presented to management at the same time as the quantitative results.
- Consistency: The same rules are applied to every comment in the same way, with no fatigue and no differences in personal interpretation.
- Scale: Analysis by breakdown (which topic stands out in which unit?) becomes much easier than doing it by hand.
As a result, open-ended questions have started to take up more space in surveys. We discuss the broader context of this agenda in our article on employee engagement trends for 2027.
Where is AI strong, and where is it weak?
| What it does well | Where it falls short |
|---|---|
| Classifying comments into predefined topics | Understanding organization-specific terms, abbreviations and project names |
| Recognizing that a comment touches on more than one topic | Correctly interpreting irony, reproach and indirect expression |
| Distinguishing positive, negative and suggestion-type comments | Highlighting rare but critical comments (harassment, ethical violations, safety) |
| Making sense of short comments or comments with spelling errors | "Inventing" a finding that is not in the data, or exaggerating proportions in summaries |
| Summarizing long sets of comments | Knowing the organizational context in which a comment was written |
This table shows that AI is extremely valuable as an analyst's assistant, but risky when left on its own.
Methods used
- Keyword and rule-based classification: Assigning comments that contain words such as "salary," "pay" or "raise" to the "pay" topic. It is simple and transparent, but it cannot tell "I'm happy with my salary" from "my salary isn't enough," and because Turkish is an agglutinative language, keyword lists quickly grow long.
- Supervised machine learning: A model trained on a hand-labeled set of comments classifies new comments into the same topics. Accuracy is high, but it requires labeled data for every organization and coding framework.
- Large language models: Given topic headings and their definitions, a language model can classify comments, determine their tone and summarize them without any prior training data. It is the most common and flexible method today; however, its results must be validated.
- Topic modeling: Tries to find the topics that emerge naturally in the comments, without defining them in advance. It is useful in the exploration phase, but interpreting and naming the topics it produces is still the expert's job.
A human-supervised analysis process
- Clean the comments: Before the analysis, mask names, job titles, phone numbers, email addresses, project names and incident details that could identify a person. This step is necessary for both privacy and the impartiality of the analysis.
- Let a human build the coding framework: Read a sample of comments to define the topic headings and a definition for each one. Keep the headings as aligned as possible with the survey's categories. The AI's accuracy depends largely on how clear these definitions are.
- Let AI classify: Assign each comment to one or more topics and tag it as positive, negative or a suggestion.
- Validate with a sample: Have an expert separately classify a randomly selected group of comments (for example, 100–200 comments) and compare the results with the AI's. Clarify the definitions of topics with low agreement and repeat the classification.
- Count and cross-tabulate: Calculate how many comments mention each topic, their tone and the groups in which they are concentrated. Read these results together with the results of the closed questions.
- Check summaries and quotes: Make sure every claim in AI-generated summaries is backed by numbers. Quotes in the report should be employees' own (cleaned) words, not sentences rewritten by the AI.
- Have a human review sensitive comments: Comments that point to issues such as harassment, discrimination, ethical violations or occupational safety should not be left to automatic classification but reviewed separately by an expert. How these comments are handled should follow a process defined before the survey starts, without compromising anonymity.
We describe the general framework for analyzing open-ended comments in our article on open-ended employee survey questions, and the analysis as a whole in our article on how to analyze employee survey results.
Challenges specific to Turkish comments
- Agglutinative structure: The same word can be written in dozens of different forms ("yönetici," "yöneticimiz," "yöneticilerimizden": manager, our manager, from our managers). Language models handle this much better than rule-based methods.
- Missing Turkish characters: In comments typed on mobile devices, it is common to use Latin equivalents instead of "ı, ğ, ş, ç, ö, ü" ("isyuku cok fazla" instead of "iş yükü çok fazla," meaning "the workload is too heavy").
- Abbreviations and corporate jargon: Abbreviations for units, systems and projects may be misclassified if they are not clearly defined in the coding framework.
- Indirect expression: Criticism expressed through polite formulas or as reproach can lead to the tone being misjudged.
For this reason, the validation step is especially important for Turkish comments.
Privacy and KVKK
Open-ended comments are the part of a survey most likely to contain personal data. When using AI tools:
- Clean the data: Mask identifying information before sending comments to any external service.
- Choose the tool carefully: Prefer enterprise tools where data is not used for model training, the retention period is defined and data processing terms are set out in a contract. Do not upload employee comments to general-purpose tools designed for personal use.
- Check cross-border transfers: If the service's servers are located abroad and the data qualifies as personal data, KVKK's requirements for transfers abroad may apply.
- Be transparent: At the start of the survey, clearly explain to employees how open-ended comments will be analyzed and whether AI tools will be used.
This section is for general information only; consult your legal counsel for an assessment specific to your organization. For other topics related to anonymity and KVKK, see our article on anonymity and KVKK in employee surveys.
Questions to ask when choosing a tool or provider
- What method is used to classify comments? Who builds the coding framework?
- How is classification accuracy measured, and is it checked by an expert?
- How are our organization-specific terms and categories introduced into the system?
- How is personal data in the comments removed before the analysis?
- Where is the data processed, how long is it retained and is it used for model training?
- How are sensitive comments (harassment, ethics, safety) identified and handled?
- Are the quotes in the reports employees' own words, or AI summaries?
We compare the general differences between a research firm and survey software in our article research firm or software for your employee survey?
Frequently asked questions
How accurately does AI analyze open-ended comments?
Current language models are generally good at classifying comments into well-defined topics; however, accuracy varies with the clarity of the coding framework, the language of the comments and organization-specific terms. In every analysis, an expert should therefore review a randomly selected sample of comments and measure how well it agrees with the AI's classification.
Is it appropriate to upload employee comments to general AI tools such as ChatGPT?
Caution is needed. Comments may contain personal data such as names, job titles or details of incidents. Uploading comments to services that have no enterprise agreement, may use the data for model training or have servers abroad creates privacy and KVKK (Turkey's Personal Data Protection Law) risks. Comments should first be stripped of identifying information, and an enterprise tool with clear data processing terms should be preferred.
Can AI replace the human analyst?
No, but it greatly speeds up their work. While AI takes on repetitive tasks such as classification and summarization, building the coding framework, validating the results, interpreting the context and turning the findings into action remain the expert's job.
Is sentiment analysis useful for employee comments?
On its own, it is limited. Knowing that a comment is positive or negative is not enough to act on without knowing what it is about. Sentiment analysis becomes meaningful when combined with topic classification, for example: "The vast majority of comments about careers are negative."
