Entropy vs Gini Impurity in Decision Trees: Formula, Differences & Examples

Introduction: Should You Use Entropy or Gini Impurity?

Every decision tree has to answer the same question at every single node: which feature, and which split point, does the best job of separating the classes? Scikit-learn gives you exactly two built-in ways to answer that question for classification trees — criterion="entropy" and criterion="gini" — and the choice is one of the first hyperparameters anyone training a DecisionTreeClassifier runs into.

Both entropy and Gini impurity are impurity measures: functions that score how mixed the classes are inside a node. A node with only one class is perfectly pure. A node split evenly between classes is maximally impure. A decision tree grows by repeatedly choosing the split that reduces impurity the most — the only real difference between entropy and Gini is how they measure that impurity.

By the end of this article you'll know:

• What entropy is, and its formula
• What Gini impurity is, and its formula
• A numerical example computed for both, side by side
• How the two compare on a feature-by-feature basis
• Which one to use, and when it actually matters
• How scikit-learn implements both

If you want the deeper mechanics of how entropy interacts with tree depth and overfitting, that's covered separately in Entropy in Decision Trees: From Information Theory to Overfitting Control — this article stays focused on the entropy-vs-Gini comparison itself.

What Is Entropy in a Decision Tree?

Entropy comes from Shannon's information theory, where it measures how much uncertainty exists in an outcome. In a decision tree node, that "outcome" is the class label of a randomly picked sample from that node.

• A pure node (every sample is the same class) has entropy = 0 — there's no uncertainty left to resolve.
• A mixed node has entropy greater than 0, rising toward its maximum as the classes get closer to a 50/50 split.

Decision trees use entropy because reducing uncertainty at every split is a direct, information-theoretic way of saying "make this node easier to predict."

Entropy Formula

H(S) = -Σ pᵢ · log₂(pᵢ)

Where:

• S is the set of samples in the node
• pᵢ is the proportion of samples belonging to class i
• The sum runs over every class present in the node

For a binary classification problem, entropy ranges from 0 (pure node) to 1 bit (a perfectly balanced 50/50 node).

Entropy Example

Take a node with 10 samples split as:

10 samples
6 = Class A  (p = 0.6)
4 = Class B  (p = 0.4)

H(S) = -(0.6 · log₂0.6 + 0.4 · log₂0.4)
     = -(0.6 × -0.737 + 0.4 × -1.322)
     = -(-0.442 - 0.529)
     = 0.971 bits

That's close to the maximum of 1 bit, which makes sense — a 60/40 split is nearly balanced, so the node is still fairly uncertain about which class a random sample belongs to.

Information Gain and Entropy

Entropy on its own only scores a single node. To choose a split, a tree needs to compare a parent's entropy against the weighted entropy of the children that split would produce. That difference is Information Gain:

Information Gain = Entropy(parent) − Σ ( |childᵥ| / |parent| × Entropy(childᵥ) )

At every candidate split, the tree computes information gain and picks the split that maximizes it — the split that removes the most uncertainty on average across the resulting child nodes.

What Is Gini Impurity?

Gini impurity measures something closely related but framed differently: the probability that a randomly chosen sample from the node would be misclassified if you labeled it according to the class distribution of that node itself.

• A pure node has Gini = 0 — you'd never misclassify a sample, because every sample is the same class.
• A mixed node has Gini greater than 0, again rising as the split gets closer to balanced.

Gini impurity is the default criterion in scikit-learn's DecisionTreeClassifier, and it's the criterion used by the original CART (Classification and Regression Trees) algorithm.

Gini Impurity Formula

Gini(S) = 1 − Σ pᵢ²

Where pᵢ is again the proportion of samples belonging to class i in node S. For binary classification, Gini impurity ranges from 0 (pure) to a maximum of 0.5 (a perfectly balanced 50/50 split).

Gini Impurity Example

Using the exact same node as before, so the two measures are directly comparable:

10 samples
6 = Class A  (p = 0.6)
4 = Class B  (p = 0.4)

Gini(S) = 1 − (0.6² + 0.4²)
        = 1 − (0.36 + 0.16)
        = 1 − 0.52
        = 0.48

Just like entropy landed close to its maximum of 1 bit, Gini lands close to its own maximum of 0.5 — both measures agree this node is nearly as impure as a binary split can get. That agreement is the pattern you'll see throughout this comparison: the two criteria almost always rank splits the same way, even though their raw numbers live on different scales.

Entropy vs Gini Impurity: What's the Difference?

Feature	Entropy	Gini Impurity
Purpose	Measure node impurity	Measure node impurity
Formula	−Σ p log₂(p)	1 − Σ p²
Range (binary classification)	0 → 1 bit	0 → 0.5
Pure node	0	0
Computational cost	Higher (log calculation)	Lower (squares only)
Common use	Information gain	CART
Splitting criterion	Information gain	Gini reduction

Entropy vs Gini: Numerical Example

Take a node with a slightly different split so the contrast is a bit sharper:

Node:
60% Class A
40% Class B

Entropy ≈ 0.971 bits
Gini    ≈ 0.48

It's tempting to compare 0.971 and 0.48 directly and conclude entropy is "reporting more impurity" — but that comparison doesn't mean anything. The absolute values of entropy and Gini aren't on the same scale, so they can't be compared directly. Entropy tops out at 1 bit for a binary problem; Gini tops out at 0.5. What actually matters for tree-building isn't the raw number at a single node — it's how each criterion changes between candidate splits. Both criteria are being asked the same question ("which split reduces impurity most?"), and in practice they agree on the answer far more often than the differing scales might suggest.

Entropy vs Gini: Which One Is Better?

When Should You Use Gini Impurity?

• You want the fastest computation — Gini only needs squaring, no logarithms
• You want a strong, well-tested default — it's scikit-learn's default for good reason
• You're training a CART-style tree, which was built around Gini from the start
• You don't have a specific reason to prefer entropy — in most cases the resulting trees are very similar

When Should You Use Entropy?

• You specifically want information gain as your splitting criterion, e.g. for interpretability tied to information theory
• You're teaching or learning decision trees and want the information-theoretic framing
• You're experimenting with different splitting criteria as part of a broader hyperparameter search

Does Gini Impurity or Entropy Produce Better Decision Trees?

There is no universally superior criterion. In practice, the two produce very similar trees on most datasets, and whichever differences show up are usually swamped by other choices entirely:

• The dataset itself and its class distribution
• Tree depth and other stopping criteria
• Pruning strategy (e.g. ccp_alpha)
• Minimum samples per leaf or per split
• The distribution and cardinality of the features being split on

If you're chasing accuracy, spend your tuning budget on max_depth, min_samples_leaf, and pruning before you spend it on the choice between criterion="gini" and criterion="entropy" — the criterion is rarely the lever that moves the needle.

Entropy vs Gini Impurity in Scikit-Learn

from sklearn.tree import DecisionTreeClassifier

model_gini = DecisionTreeClassifier(
    criterion="gini"       # scikit-learn's default
)

model_entropy = DecisionTreeClassifier(
    criterion="entropy"
)

model_gini.fit(X_train, y_train)
model_entropy.fit(X_train, y_train)

print(f"Gini test accuracy:    {model_gini.score(X_test, y_test):.3f}")
print(f"Entropy test accuracy: {model_entropy.score(X_test, y_test):.3f}")
# On most datasets these two numbers land within a point or two of each other

criterion="gini" is the default in DecisionTreeClassifier, so you only need to set it explicitly if you want entropy. Newer versions of scikit-learn also expose criterion="log_loss", which is mathematically equivalent to entropy for classification and is worth knowing about if you see it in someone else's code.

Gini vs Entropy: Computational Complexity

Gini uses only p² — squaring is cheap, and modern CPUs do it in a single cycle. Entropy uses log₂(p), and logarithms are meaningfully more expensive to compute than a multiplication. Because this calculation happens at every candidate split, across every feature, at every node, during training, the cost does add up — which is part of why Gini is the default in performance-sensitive implementations like scikit-learn's CART.

That said, don't overstate this: on typical dataset sizes, the difference in wall-clock training time between the two criteria is usually modest, not dramatic. It becomes more noticeable on very large datasets or when you're growing many trees, as in a random forest or gradient-boosted ensemble.

Entropy vs Gini Impurity: Advantages and Disadvantages

Advantages of Entropy

• Grounded in information theory — has a clean interpretation as "bits of uncertainty"
• Directly connected to information gain, a well-studied concept
• Useful when you specifically want to reason about uncertainty reduction

Disadvantages of Entropy

• More computationally expensive — requires logarithmic calculations at every split
• Often produces results very similar to Gini, so the extra cost doesn't always buy much

Advantages of Gini

• Computationally simpler — no logarithms involved
• Fast, which matters at scale or across large ensembles
• Effective and widely used as a strong default

Disadvantages of Gini

• Less intuitive from an information-theory perspective
• Its absolute values aren't directly comparable to entropy's — they live on different scales

Entropy vs Gini Impurity: Final Verdict

For most classification problems, Gini impurity is an excellent default — it's fast, well-tested, and produces trees that are competitive with entropy-based trees on the large majority of datasets. Entropy is a strong alternative when you specifically want an information-theoretic splitting criterion, or when you're teaching or exploring the theory behind how trees choose splits.

Need a fast default?      → Gini
Want information gain?    → Entropy
Learning decision trees?  → Understand both

Frequently Asked Questions

Is entropy better than Gini impurity?

Neither is categorically better — they usually produce very similar trees. Gini is faster to compute and is scikit-learn's default; entropy gives you an information-theoretic splitting criterion tied to information gain. Pick based on your priorities, not on an expectation of a large accuracy difference.

Is Gini impurity faster than entropy?

Yes. Gini only requires squaring probabilities, while entropy requires computing a logarithm for each class proportion at every candidate split. The gap is usually modest on typical datasets but grows with dataset size and the number of trees you're training.

What is the difference between entropy and Gini impurity?

Both measure how mixed the classes are in a node, and both hit zero for a pure node. Entropy uses −Σ p log₂(p) and, for binary classification, ranges from 0 to 1 bit. Gini uses 1 − Σ p² and ranges from 0 to 0.5. They're on different scales, so their raw values shouldn't be compared directly — what matters is how each one ranks candidate splits, and they usually agree.

Should I use entropy or Gini in a decision tree?

Start with Gini — it's the default for a reason, and it's faster. Switch to entropy if you have a specific reason to want information gain as your splitting criterion, or if you're comparing criteria as part of a broader hyperparameter search.

What is the formula for Gini impurity?

Gini(S) = 1 − Σ pᵢ², where pᵢ is the proportion of samples belonging to class i in node S. For a pure node, Gini is 0; for a perfectly balanced binary node, it's 0.5.

What is the entropy formula in a decision tree?

H(S) = −Σ pᵢ · log₂(pᵢ), summed over each class present in node S. For a pure node, entropy is 0; for a perfectly balanced binary node, it's 1 bit.

Why is Gini impurity preferred in decision trees?

Mainly for speed — it avoids logarithmic calculations — and because it was the criterion used in the original CART algorithm, which is why it's the default in libraries like scikit-learn. It isn't preferred because it produces meaningfully more accurate trees.

Can entropy and Gini produce different decision trees?

Yes, they can occasionally rank two close candidate splits differently, which can lead to a different tree structure. In practice these differences are usually small and rarely change the model's overall predictive performance in a meaningful way.

What is the difference between information gain and Gini impurity?

Gini impurity scores a single node's mixedness. Information gain is derived from entropy and measures how much a specific split reduces impurity — parent entropy minus the weighted entropy of the resulting children. The Gini equivalent of information gain is sometimes called "Gini reduction" or "Gini gain," computed the same way but using Gini impurity instead of entropy.

Final Thought

Entropy and Gini impurity are two different lenses on the exact same question: how mixed is this node, and which split makes it less mixed? They rank splits similarly far more often than they disagree, which is why the choice between them matters less than most other decisions you'll make when training a tree — depth, pruning, and minimum leaf size will move your accuracy far more than swapping criterion="gini" for criterion="entropy" ever will. Use Gini as your default, reach for entropy when you specifically want an information-theoretic framing, and spend the rest of your tuning effort where it actually pays off.

For more on how entropy specifically interacts with tree depth and the bias-variance tradeoff, see Entropy in Decision Trees: From Information Theory to Overfitting Control.