TL;DR — An exponent builds a number up by repeated multiplication (
10^3 = 1000); a logarithm is its exact inverse — it asks "what power gives me this number?" (log10(1000) = 3). The one property that makes logs indispensable in AI and statistics: a log turns multiplication into addition —log(a·b) = log a + log bandlog(a/b) = log a − log b. So messy products and ratios collapse into tidy sums and differences. That is why logs tame huge products, likelihoods, and ratios. On a log scale, equal ratios are equal distances:0.5(halve) and2(double) sit the same distance from1. This symmetry is exactly why confidence intervals for ratios (RR, OR) are built on the log scale. Push it one step further and you get the logit —logit(p) = ln(p/(1−p))— whose inverse is the sigmoidσ(z) = 1/(1+e^−z), the bridge to logistic regression. And because probabilities multiply, adding their logs (log-loss / cross-entropy) is how we stay numerically stable.
1. Simple explanation
An exponent is a shortcut for repeated multiplication. 10^3 means "multiply three tens together" = 1000. A logarithm runs that machine backwards. It looks at 1000 and asks: "ten raised to what gives this?" The answer is 3, so log10(1000) = 3. Exponent and logarithm are two views of the same fact — one goes forward, one goes back. They undo each other, the same way addition undoes subtraction.
Analogy — a volume knob. Imagine a stereo where each click of the knob doubles the loudness: click 1 is 2×, click 2 is 4×, click 3 is 8×. The exponent answers "I turned it 3 clicks — how much louder?" (2^3 = 8). The logarithm answers the reverse: "it's 8× louder — how many clicks was that?" (log2(8) = 3). Your ear actually hears loudness in clicks, not in raw multiples — which is why decibels, earthquake magnitudes, and pH are all log scales. Equal steps on the knob feel equal, even though the underlying number is exploding.
There is a special base that nature keeps choosing: e ≈ 2.718. Its logarithm is the natural log, written ln. Just as log10 undoes 10^x, ln undoes e^x (written exp(x)). So exp(ln(5)) = 5 and ln(exp(5)) = 5 — round trip, back where you started.
The three questions this article answers: Why does turning products into sums matter so much? Why do ratios live on a log scale? And how does that lead straight to logistic regression?
2. Diagram
EXPONENT and LOGARITHM are INVERSES
┌──────────────┐
3 ───▶ │ 10^x │ ───▶ 1000
└──────────────┘
┌──────────────┐
1000 ───▶ │ log10(x) │ ───▶ 3
└──────────────┘
"10 to the 3rd is 1000" ⇕ "log base 10 of 1000 is 3"
THE KEY PROPERTY — log turns × into + :
log(a · b) = log a + log b
log(a / b) = log a − log b
log(a^n) = n · log a
THE LOG SCALE — equal RATIOS are equal DISTANCES:
linear: 0 0.5 1 2 4
├──────┼─────┼────────┼─────────────────────┤
(0.5 hugs the wall, 2 drifts off alone)
log: 0.5 1 2
┌──────┼──────┼──────┼──────┐
0.25 (÷2) (×1) (×2) 4
└── same distance ──┘
0.5 and 2 are EQUIDISTANT from 1
THE LOGIT / SIGMOID BRIDGE:
probability p ──logit──▶ log-odds z = ln(p/(1−p)) (0..1 ▶ −∞..+∞)
log-odds z ──sigmoid─▶ p = 1/(1+e^−z) (−∞..+∞ ▶ 0..1)
(these two are inverses of each other)
3. How it works
3.1 Exponent and logarithm — the same fact, two directions
Every exponent statement has a mirror-image logarithm statement. They carry identical information.
| Exponent form | Logarithm form | Reads as |
|---|---|---|
10^2 = 100 | log10(100) = 2 | "10 to the 2nd is 100" |
2^5 = 32 | log2(32) = 5 | "2 to the 5th is 32" |
e^0 = 1 | ln(1) = 0 | "anything to the 0 is 1" |
e^1 = e | ln(e) = 1 | "the natural log of e is 1" |
Two facts fall straight out of the table and are worth memorizing: log(1) = 0 (any base — because base^0 = 1), and log(base) = 1 (because base^1 = base). The log of a number bigger than 1 is positive; the log of a number between 0 and 1 is negative; the log of 0 marches off to −∞.
3.2 The key property — log turns multiplication into addition
This is the reason logs are everywhere. Take any product and log it:
log(a · b) = log a + log b (multiply → add)
log(a / b) = log a − log b (divide → subtract)
log(a^n) = n · log a (power → multiply)
Multiplication is "expensive" — it compounds, it explodes, it underflows to zero on a computer. Addition is cheap, stable, and symmetric. The log is a translator that carries you from the hard world of products into the easy world of sums, lets you do the work there, and exp carries you back.
Everyday analogy — a slide rule. Before calculators, engineers multiplied huge numbers by sliding two ruler-like scales past each other. The rulers were marked on a log scale, so sliding (physically adding lengths) performed multiplication.
log(a·b) = log a + log bis literally the law that made the slide rule work: line up lengthlog a, add lengthlog b, read offa·b.
Three places this property is load-bearing in AI:
- Ratios become differences. A ratio
a/bis a single messy number, butlog(a/b) = log a − log bis a clean subtraction. This is why ratios are best analyzed in log space (§3.3). - Likelihoods become log-likelihoods. A model's likelihood is a product of many probabilities,
p1·p2·…·pN. Eachpᵢis below 1, so the product rushes toward zero and underflows. Take the log and it becomes a sum,log p1 + log p2 + … + log pN— every term is a manageable negative number that adds up safely. - Cross-entropy / log-loss. The training loss for classifiers is built from
−log(p)of the true class precisely so that multiplying probabilities becomes adding penalties (§3.5).
3.3 The log scale — equal ratios, equal distances
On a normal (linear) ruler, equal gaps are equal differences: the step from 1 to 2 is the same width as the step from 100 to 101. On a log ruler, equal gaps are equal ratios: the step from 1 to 2 (×2) is the same width as the step from 100 to 200 (×2). Multiplying by a fixed factor always moves you a fixed distance.
The consequence that matters most: 0.5 and 2 sit the same distance from 1. Halving (÷2) and doubling (×2) are mirror moves, so on a log axis they are mirror positions around 1. Proof in one line: log(0.5) = −log 2 and log(2) = +log 2 — same magnitude, opposite sign, equidistant from log(1) = 0.
Easy way to feel it — money doubling.
$10 → $20is the same good news as$100 → $200: both doubled. You react to the ratio (×2), not the raw dollars (a$10gap vs a$100gap). A log scale spaces every "doubling" the same width, so those two jumps are equal-size steps — which is exactly why0.5(÷2) and2(×2) are equal distance from1. (Same idea as a phone camera:1× → 2×is "one zoom," and so is2× → 4×— each tap doubles.)
÷2 ×2
0.5 ◀────────── 1 ──────────▶ 2
log(0.5) = −0.693 log(2) = +0.693 (equal distance either side of 0)
Why does an analyst care? Because ratios are asymmetric on a linear scale and symmetric on a log scale. A relative risk of 2 (twice the risk) and 0.5 (half the risk) are equal-and-opposite effects — but on a linear axis 2 looks "1 unit above" no-effect while 0.5 looks "only 0.5 below." The log axis fixes this: the sampling distribution of a ratio estimate (relative risk RR, odds ratio OR) is skewed on the raw scale but roughly symmetric and normal on the log scale. So you build the confidence interval as exp( ln(ratio) ± 1.96·SE ) — symmetric in log space, then exponentiate back. See 2×2 Tables & Effect Measures (OR, RR, RD, NNT, HR) for that machinery in full.
3.4 Log-odds (the logit) — stretching a probability to the whole line
A probability p is trapped between 0 and 1. That is awkward for a model that wants to add and subtract freely. The fix is a two-step stretch.
- Turn the probability into odds:
odds = p/(1−p). A probability of0.75becomes odds of0.75/0.25 = 3("3 to 1"). Odds run from0to+∞. - Take the log of the odds:
logit(p) = ln(p/(1−p)). This is the log-odds. It runs from−∞(atp=0) through0(atp=0.5) to+∞(atp=1).
| p | odds = p/(1−p) | logit(p) = ln(odds) |
|---|---|---|
| 0.10 | 0.111 | −2.197 (negative: unlikely) |
| 0.50 | 1.000 | 0.000 (the tipping point) |
| 0.75 | 3.000 | +1.099 (positive: likely) |
| 0.90 | 9.000 | +2.197 (mirror of 0.10) |
Notice the symmetry again: p=0.10 and p=0.90 give logits of −2.197 and +2.197. The logit is a log of a ratio, so it inherits the log scale's mirror property.
3.5 The sigmoid — the logit run backwards
The logit maps (0,1) → (−∞, +∞). To turn a model's raw score z back into a probability we need the inverse, which is the sigmoid:
σ(z) = 1 / (1 + e^−z)
Feed the sigmoid a log-odds and you recover the probability. σ(1.099) = 0.75, undoing the logit above. This pair — logit going in, sigmoid coming out — is the entire skeleton of Logistic Regression: the model predicts a log-odds as a weighted sum of features (a linear thing, because logs made it additive), then the sigmoid squashes that back into a valid probability. And the training loss, log-loss / cross-entropy, is −log(p) of the true label — logs again, so that a product of per-example likelihoods becomes a sum the optimizer can walk down.
4. The math
The core identities. For any base b > 0 (and ln is just base e):
b^x = y ⇔ log_b(y) = x (inverse relationship)
exp(ln x) = x and ln(exp x) = x (e and ln undo each other)
log(a · b) = log a + log b (product → sum)
log(a / b) = log a − log b (quotient → difference)
log(a^n) = n · log a (power → product)
Worked example 1 — product becomes a sum. Show that log(8 · 4) = log 8 + log 4, base 2.
Left side: log2(8 · 4) = log2(32) = 5 (since 2^5 = 32)
Right side: log2(8) + log2(4) = 3 + 2 = 5 (since 2^3=8, 2^2=4)
Left = Right = 5. ✓ The multiplication 8·4 turned into the addition 3+2.
Worked example 2 — 0.5 and 2 are equidistant from 1 on a log axis. Use natural log.
distance of 2 from 1 = ln(2) − ln(1) = 0.6931 − 0 = +0.6931
distance of 0.5 from 1 = ln(0.5) − ln(1) = −0.6931 − 0 = −0.6931
|+0.6931| = |−0.6931|. ✓ Same distance, opposite sides of 1.
Because ln(0.5) = ln(1/2) = ln 1 − ln 2 = 0 − 0.6931 = −ln 2.
Halving and doubling are perfect mirror moves in log space.
Worked example 3 — compute logit(0.75), then invert it back with the sigmoid.
Forward (logit):
odds = p/(1−p) = 0.75 / 0.25 = 3
logit = ln(3) = 1.0986 ← the log-odds
Backward (sigmoid), feed z = 1.0986:
σ(z) = 1 / (1 + e^−z) = 1 / (1 + e^−1.0986)
= 1 / (1 + 0.3333) = 1 / 1.3333 = 0.75 ✓ back to the original p
The round trip p → logit → sigmoid → p returns 0.75 exactly. That is the guarantee an inverse gives you: nothing is lost, the two functions cancel.
5. Real code
Pure numpy / math, no special libraries. It demonstrates the product-to-sum law, prints a small log axis to show the 0.5/2 symmetry, and round-trips a probability through logit and sigmoid.
"""Logarithms, the log scale, and the logit<->sigmoid bridge — from first principles.
Everything here is pure numpy/math: no model, no API key."""
import numpy as np
# ---- 1. THE KEY PROPERTY: log turns multiplication into addition ----
a, b = 8.0, 4.0
lhs = np.log(a * b) # log of the product
rhs = np.log(a) + np.log(b) # sum of the logs
print("log(a*b) =", round(lhs, 6), " log a + log b =", round(rhs, 6),
" equal?", np.isclose(lhs, rhs))
# quotient rule too: a ratio becomes a difference
print("log(a/b) =", round(np.log(a / b), 6),
" log a - log b =", round(np.log(a) - np.log(b), 6))
# ---- 2. THE LOG SCALE: 0.5 and 2 are equidistant from 1 ----
def log_axis(values, width=44):
"""Print values on a natural-log axis so equal RATIOS are equal DISTANCES."""
logs = np.log(values)
lo, hi = logs.min(), logs.max()
print("\nlog axis (equal ratios = equal gaps):")
for v, lg in zip(values, logs):
col = int((lg - lo) / (hi - lo) * (width - 1))
line = [" "] * width
line[col] = "|"
marker = " <- no-effect (ratio = 1)" if np.isclose(v, 1.0) else ""
print(f" {v:>4} {''.join(line)}{marker}")
d_half = abs(np.log(0.5) - np.log(1.0))
d_two = abs(np.log(2.0) - np.log(1.0))
print(f" distance(0.5 -> 1) = {d_half:.4f} distance(2 -> 1) = {d_two:.4f}"
f" equal? {np.isclose(d_half, d_two)}")
log_axis(np.array([0.25, 0.5, 1.0, 2.0, 4.0]))
# ---- 3. THE LOGIT <-> SIGMOID BRIDGE (they are inverses) ----
def logit(p):
"""Probability -> log-odds. Stretches (0,1) onto the whole real line."""
return np.log(p / (1.0 - p))
def sigmoid(z):
"""Log-odds -> probability. The inverse of logit; used at the end of logistic regression."""
return 1.0 / (1.0 + np.exp(-z))
p = 0.75
z = logit(p) # forward: probability -> log-odds
p_back = sigmoid(z) # backward: log-odds -> probability
print(f"\np = {p}")
print(f"logit(p) = ln({p}/{1-p}) = {z:.4f} (log-odds)")
print(f"sigmoid(z) = {p_back:.4f} (round-trips back to p)")
print(f"round-trip exact? {np.isclose(p, p_back)}")
# symmetry: p and (1-p) give equal-and-opposite log-odds
print(f"logit(0.90) = {logit(0.90):+.4f} logit(0.10) = {logit(0.10):+.4f}"
f" mirror? {np.isclose(logit(0.90), -logit(0.10))}")
Expected output:
log(a*b) = 3.465736 log a + log b = 3.465736 equal? True
log(a/b) = 0.693147 log a - log b = 0.693147
log axis (equal ratios = equal gaps):
0.25 |
0.5 |
1.0 | <- no-effect (ratio = 1)
2.0 |
4.0 |
distance(0.5 -> 1) = 0.6931 distance(2 -> 1) = 0.6931 equal? True
p = 0.75
logit(p) = ln(0.75/0.25) = 1.0986 (log-odds)
sigmoid(z) = 0.7500 (round-trips back to p)
round-trip exact? True
logit(0.90) = +2.1972 logit(0.10) = -2.1972 mirror? True
Read the axis: 0.5 and 2.0 land the same number of columns either side of the 1.0 no-effect line — the log scale makes halving and doubling symmetric. And the logit/sigmoid pair returns 0.75 untouched, because one is the exact inverse of the other.
6. Real-world example
A spam filter that has to multiply a thousand tiny probabilities.
- Setup. A naive-Bayes-style spam filter scores an email by multiplying the probability of each of its ~1000 words given "spam." Each word probability is small — say around
0.001. The raw score is therefore0.001 · 0.001 · …a thousand times over. - The problem — underflow.
0.001^1000 = 10^−3000, a number so small that floating-point arithmetic rounds it flat to0.0. Two different emails both score0.0, and the filter can no longer tell them apart. The product underflowed. - The log fix. Instead of multiplying the probabilities, add their logs.
log(p1·p2·…) = log p1 + log p2 + …. Now the score is a sum of ~1000 moderate negative numbers (eachlog(0.001) ≈ −6.9), landing near−6900— a perfectly ordinary float. The emails are cleanly separable again. This is exactly the product-to-sum law from §3.2 doing real work. - Turning the score into a probability. The filter's final "how spammy, 0 to 1?" number is produced by pushing the summed log-odds through a sigmoid — the same
σ(z)from §3.5. A total log-odds of+2becomesσ(2) ≈ 0.88, i.e. "88% spam." - Reporting an effect on the log scale. Suppose the word "free" appears 4× more often in spam. That is a ratio of 4. If a rare word appeared 4× less often, that is a ratio of
0.25. On a raw scale4and0.25look wildly different sizes; on the log scale they are mirror images (ln 4 = +1.386,ln 0.25 = −1.386), so the model can treat "4× more" and "4× less" as equal-and-opposite evidence — which is precisely what a logistic-regression weight does. - Payoff. Every hard part — the vanishing product, the squashing into a probability, the symmetric treatment of ratios — was solved by the same two tools: logs turn products into sums, and the sigmoid turns log-odds back into probabilities.
7. Interview questions companies actually ask
Q [Google / ML fundamentals] "Why do we work with LOG-likelihood instead of the raw likelihood?"
A Two reasons, both from log(a·b)=log a+log b. (1) NUMERICAL STABILITY: a likelihood is a product
of many probabilities (each < 1); it underflows to 0 in floating point. The log turns it into a
SUM of moderate negatives that never underflows. (2) EASIER MATH: sums are far easier to
differentiate term-by-term than products, so gradient-based optimizers work on the log-likelihood.
Because log is monotonic, whatever maximizes the log-likelihood also maximizes the likelihood.
Q [Meta / applied stats] "What is a logit, and how is it related to the sigmoid?"
A The logit is the log-odds: logit(p) = ln(p/(1−p)). It stretches a probability from (0,1) onto the
whole real line. The sigmoid σ(z) = 1/(1+e^−z) is its INVERSE — it squashes a real number back into
(0,1). Logistic regression predicts a logit as a linear function of features, then the sigmoid
turns that into a probability. logit(σ(z)) = z and σ(logit(p)) = p.
Q [a fintech risk team] "Why build a confidence interval for an odds ratio on the LOG scale?"
A A ratio can't go below 0, so its sampling distribution is skewed and a symmetric estimate ± 1.96·SE
on the raw ratio is invalid (it could dip below 0). Taking the log makes the distribution roughly
symmetric and normal, because on the log scale equal RATIOS are equal DISTANCES — 0.5 and 2 are
equidistant from 1. So you compute ln(OR) ± 1.96·SE and exponentiate the endpoints. The result is
multiplicatively symmetric around the point estimate.
Q [Amazon / DS] "Explain like I'm five: why is 0.5 the same 'distance' as 2 on a log axis?"
A Because a log axis measures MULTIPLYING, not adding. Going from 1 to 2 means '×2'. Going from 1 to
0.5 means '÷2'. Multiply-by-2 and divide-by-2 are equal-and-opposite moves, so they cover the same
distance in opposite directions. In numbers, ln(2) = +0.693 and ln(0.5) = −0.693 — same size,
opposite sign.
Q [an NLP startup] "Your text classifier multiplies 500 word probabilities and always gets 0. Fix it."
A It's floating-point underflow: 500 numbers below 1 multiply down past the smallest representable
float. Switch to LOG space — add log-probabilities instead of multiplying probabilities
(log(∏ pᵢ) = Σ log pᵢ). The sum stays in a normal range. If you need a final probability, exponentiate
at the very end, or use a log-sum-exp trick to normalize without leaving log space.
Q [Apple / ML] "What does e (and natural log) have that base-10 log doesn't, for ML?"
A e is the base where the exponential is its own derivative (d/dx e^x = e^x), which makes calculus in
gradient descent clean — the derivative of ln(x) is simply 1/x. Sigmoids, softmax, cross-entropy, and
the Gaussian all use e for that reason. You CAN convert between bases (log_b(x) = ln(x)/ln(b)), so the
choice is about convenient derivatives, not information — logs of different bases differ only by a
constant factor.
Q [a healthcare-AI company] "Cross-entropy loss is −log(p) of the true class. Why the log?"
A Because the total likelihood of the data is a PRODUCT of per-example probabilities, and −log turns
that product into a SUM the optimizer can add up and differentiate. It also shapes the penalty well:
−log(p) is ~0 when the model is confident and right (p→1) and shoots to +∞ when it is confident and
wrong (p→0), punishing overconfident mistakes hard. Minimizing summed −log(p) is exactly maximizing
the log-likelihood.
8. When to use / tradeoffs
REACH FOR LOGS WHEN:
✓ multiplying many probabilities/likelihoods — add logs to avoid underflow
✓ the quantity is a RATIO (RR, OR, fold-change, ratios of rates) — analyze in log space
✓ values span many orders of magnitude — a log axis makes the plot readable
✓ you need a probability from a real-valued score — sigmoid (inverse-logit)
✓ you want equal % changes to look equal — log returns show +10% and −10% symmetrically
USE THE SIGMOID / LOGIT WHEN:
✓ mapping a linear score z to a probability p (logistic regression output layer)
✓ you need logit(p) to turn a bounded probability into an unbounded regression target
HONEST LIMITS:
✗ log(0) is −∞ and log of a negative is undefined — add a small epsilon or a continuity
correction before logging counts that can be zero
✗ logit(0) and logit(1) blow up — clip probabilities to (ε, 1−ε) first
✗ a log axis hides absolute size — a huge log-scale gap can be a tiny absolute change; show the
raw numbers too
✗ don't average in log space and report it as an arithmetic mean — exp(mean(log x)) is the
GEOMETRIC mean, a different (and often more appropriate) beast
THE RULE OF THUMB:
products, ratios, and probabilities → move to log space; report back on the natural scale.
Logs are a change of coordinates, not a change of truth. Move into log space to do the arithmetic that products and ratios make painful, then exp back out to speak to humans in ordinary numbers. The one mistake to avoid is forgetting to come back — a log-odds is not a probability until the sigmoid returns it, and a log-scale interval is not a ratio until you exponentiate it.
9. Summary + related articles
- An exponent (
10^3 = 1000) and a logarithm (log10(1000) = 3) are exact inverses;exp(e^x) andlnundo each other. - The load-bearing property:
log(a·b) = log a + log bandlog(a/b) = log a − log b— multiplication becomes addition, ratios become differences. - That property tames huge products (likelihoods → log-likelihoods), keeps computation stable (add log-probs instead of multiplying), and linearizes ratios.
- On the log scale, equal ratios are equal distances:
0.5and2are equidistant from1— which is why ratio confidence intervals (RR, OR) are built asexp(ln(ratio) ± 1.96·SE). - The logit
ln(p/(1−p))stretches a probability onto the whole line; its inverse, the sigmoid1/(1+e^−z), squashes it back — the exact bridge into logistic regression. - Log-loss / cross-entropy uses
−log(p)so a product of likelihoods becomes a sum the optimizer can minimize.
Related: Probability & Statistics Foundations · Logistic Regression · 2×2 Tables & Effect Measures (OR, RR, RD, NNT, HR)
Resources
- Khan Academy, "Intro to logarithms" — https://www.khanacademy.org/math/algebra2/x2ec2f6f830c9fb89:logs
- 3Blue1Brown, "Logarithms, and the logit/sigmoid" visual intuition — https://www.3blue1brown.com
- Goodfellow, Bengio & Courville, Deep Learning, §3 & §6 (log-likelihood, cross-entropy, sigmoid) — https://www.deeplearningbook.org
- Altman & Bland, "Statistics Notes: logarithms" — https://www.bmj.com/content/312/7032/700
- NumPy reference:
numpy.log,numpy.exp— https://numpy.org/doc/stable/reference/routines.math.html
