midData Mining

What is sentiment analysis?

Updated May 17, 2026

Short answer

Sentiment analysis is a natural language processing (NLP) technique used to identify and classify the emotional tone or opinion expressed in text, typically as positive, negative, or neutral. It is widely used in data mining to extract insights from large volumes of unstructured text such as reviews, social media posts, and customer feedback. Modern sentiment analysis systems often use machine learning or deep learning models to learn patterns in language and predict sentiment automatically.

Deep explanation

Sentiment analysis, also called opinion mining, is a data mining task that analyzes textual data to determine the attitude, emotion, or subjective opinion expressed by a writer toward a topic, product, service, or entity.

The goal is not only to understand what is being discussed, but also how the author feels about it. Since most business data exists in unstructured text formats, sentiment analysis helps convert large amounts of text into structured information that can support decision-making.

A typical sentiment analysis pipeline includes the following steps:

  1. Data collection
  • Gather text data from sources such as:
  • Product reviews
  • Social media posts
  • Customer support conversations
  • Survey responses
  • News articles
  • The collected data may be labeled manually for supervised learning or used without labels in unsupervised approaches.
  1. Text preprocessing Raw text usually requires cleaning before analysis. Common preprocessing steps include:
  • Removing HTML tags, URLs, and unnecessary symbols
  • Converting text to lowercase
  • Tokenization (splitting text into words or subwords)
  • Removing stop words when appropriate
  • Stemming or lemmatization
  • Handling emojis, slang, and spelling variations

Example:

Original text:

`` "The movie was AMAZING!!! I loved it 😍" ``

After preprocessing:

`` ["movie", "amazing", "love", "😍"] ``

  1. Feature extraction Machine learning models require numerical representations of text. Common approaches include:
  • Bag of Words (BoW): Represents text using word occurrence counts.
  • TF-IDF (Term Frequency-Inverse Document Frequency): Weights words based on how important they are within a document compared with the entire dataset.
  • Word embeddings: Represent words as dense vectors that capture semantic relationships.
  • Contextual embeddings: Models such as transformers generate representations based on surrounding words, allowing better handling of context.
  1. Model training and classification

Sentiment models can be built using different techniques:

Traditional machine learning approaches:

  • Naive Bayes
  • Logistic Regression
  • Support Vector Machines
  • Random Forests

Example:

```python from sklearn.linear_model import LogisticRegression

model = LogisticRegression() model.fit(training_features, training_labels)

prediction = model.predict(test_features) ```

Deep learning approaches:

  • Recurrent Neural Networks (RNNs)
  • Long Short-Term Memory networks (LSTMs)
  • Transformers such as BERT and its variants

Deep learning models generally perform better because they can understand complex language patterns and context.

  1. Sentiment classification

The output depends on the business requirement. Common classification levels include:

  • Binary sentiment:
  • Positive
  • Negative
  • Three-class sentiment:
  • Positive
  • Negative
  • Neutral
  • Fine-grained sentiment:
  • Very positive
  • Positive
  • Neutral
  • Negative
  • Very negative

Some systems also perform aspect-based sentiment analysis, where sentiment is associated with specific features.

Example:

`` "The camera quality is excellent, but the battery life is disappointing." ``

Aspect-level output:

`` Camera: Positive Battery: Negative ``

Challenges and Trade-offs

Sentiment analysis is difficult because human language contains ambiguity, context, and implicit meaning.

Common challenges include:

  • Sarcasm:
  • "Great, my flight was delayed for six hours."
  • A simple keyword-based model may incorrectly classify this as positive.
  • Negation:
  • "I do not like this product."
  • The word "like" alone suggests positivity, but the complete meaning is negative.
  • Domain dependency:
  • The word "unpredictable" may be negative for a car but positive for a movie plot.
  • Mixed sentiment:
  • A review may contain both positive and negative opinions.
  • Language variation:
  • Slang, abbreviations, multilingual text, and emojis require specialized handling.

Evaluation Metrics

Sentiment analysis models are evaluated using standard classification metrics:

  • Accuracy: Percentage of correctly classified samples.
  • Precision: How many predicted positive/negative results are actually correct.
  • Recall: How many actual positive/negative cases were identified.
  • F1-score: Harmonic mean of precision and recall.

For imbalanced datasets, accuracy alone can be misleading. For example, if 95% of reviews are positive, a model predicting everything as positive achieves high accuracy but provides little value.

Real-world example

A company wants to analyze thousands of customer reviews for its mobile application. Instead of manually reading every review, it uses sentiment analysis to automatically identify customer satisfaction trends.

Example reviews:

TypeScript
1. "The new update is fantastic. The app is much faster now."
2. "The app crashes every time I try to upload a photo."
3. "The interface is okay, but some features are confusing."

A sentiment model may produce:

TypeScript
Review 1 → Positive
Review 2 → Negative
Review 3 → Neutral/Mixed

A simple implementation using a pre-trained NLP model might look like:

Python
from transformers import pipeline


classifier = pipeline("sentiment-analysis")


reviews = [
    "The new update is fantastic. The app is much faster now.",
    "The app crashes every time I try to upload a photo."
]


for review in reviews:
    result = classifier(review)
    print(result)

The company can aggregate these predictions to track product quality, detect emerging issues, and prioritize improvements.

Common mistakes

  • * Treating sentiment analysis as simple keyword matching instead of understanding language context.
  • * Ignoring sarcasm, irony, and implicit opinions in text.
  • * Training a model on one domain and assuming it will perform equally well in another domain.
  • * Using accuracy as the only evaluation metric for imbalanced sentiment datasets.
  • * Removing important information such as negation words during preprocessing.
  • * Ignoring multilingual content and language-specific expressions.
  • * Assuming sentiment analysis can determine objective truth rather than subjective opinion.
  • * Failing to continuously retrain models as language usage and customer behavior change.

Follow-up questions

  • How does sentiment analysis differ from text classification?
  • What are the main approaches used for sentiment analysis?
  • Why do traditional sentiment analysis methods struggle with sarcasm?
  • What is aspect-based sentiment analysis?
  • How would you improve the performance of a sentiment analysis model?

More Data Mining interview questions

View all →