Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
A probabilistic context-free grammar (PCFG) attaches probabilities to grammar rules; probabilistic CKY uses those rules in a dynamic-programming chart to find the highest-probability parse of a sentence. It does not make an ambiguous sentence unambiguous or guarantee that its winning parse is correct: it selects the best tree under a particular grammar and its probabilities.
From a grammar to a probable parse
A syntactic parser takes a sequence of tokens and builds one or more trees that describe how the words form constituents. A context-free grammar (CFG) licenses possible structures; a probabilistic CFG (PCFG) also assigns preferences to the grammar’s alternative rules. CKY—also called CYK—is a bottom-up chart-parsing algorithm that combines smaller spans into larger ones. With a binary-form grammar and a maximum operation, probabilistic CKY finds a Viterbi parse: the single tree with the greatest probability under the model.
For example, “I saw the man with the telescope” allows at least two readings. The prepositional phrase might attach to the man (the man has the telescope), or to saw (the telescope was used to see him). A CFG may license both trees. A PCFG ranks them; the probabilities do not themselves settle the sentence’s meaning.
Free tools Windows power users keep installed
One-click scans. No signup required.
CFG and PCFG basics
A CFG is commonly written as G = (N, Σ, S, R): N is the set of nonterminal categories (such as NP and VP), Σ is the set of terminals (typically words), S is the start symbol, and R is the set of production rules. A rule’s left side is one nonterminal. For example:
#1 Best Overall
S -> NP VP
NP -> Det N
VP -> V NP
Det -> "the"
N -> "cat"
V -> "sees"A PCFG adds a probability to each production. Under the standard definition, all rules with the same left-hand nonterminal form a probability distribution:
Σ P(A -> β) = 1 for each nonterminal A.
For instance, if VP has rules VP -> V NP [0.7] and VP -> V NP PP [0.3], their probabilities sum to 1. A tree’s probability is the product of the probabilities of the rules used to build it:
P(t) = ∏ P(r), for the rules r used in tree t.
If a tree uses rules with probabilities 0.9, 0.8, 0.7, and 1.0, its probability is 0.9 × 0.8 × 0.7 × 1.0 = 0.504. That number is a probability under this grammar, not automatically a measure of real-world correctness or calibrated confidence. In a basic PCFG, the probability of a rule depends on its left-hand category, not on the whole sentence, the parent context, or all the words around it. This makes the model tractable, but limits what it can express.
For an example of the standard PCFG definition and parse probabilities, see NLTK’s chapter on analyzing sentence structure and its PCFG API documentation.
Where rule probabilities come from
A common starting point is a treebank: a corpus of sentences annotated with parse trees. For each left-hand side, estimate a rule’s probability by relative frequency:
P(A -> β) = count(A -> β) / count(A -> *),
where the denominator counts every occurrence of a rule expanding A. This is maximum-likelihood estimation from the annotated examples; NLTK’s grammar documentation describes the relative-frequency approach.
Unsmoothed estimates assign zero probability to rules never observed. Estimates for rare rules may be unreliable, and the annotation conventions and genres in the treebank affect the resulting grammar. Words absent from lexical rules also cause a practical problem: CKY cannot start a derivation for a token with no lexical category. Systems commonly need an unknown-word class, lexical smoothing, or another fallback. A treebank-derived PCFG is neither universally calibrated nor neutral with respect to its source data.
Recommended Free Tools
Why standard CKY uses binary grammar rules
The familiar CKY recurrence is easiest to apply when rules are in Chomsky Normal Form (CNF), with binary rules A -> B C and lexical rules A -> word. A rule such as A -> B C D can be binarized using an artificial category:
A -> B X
X -> C DThis lets the parser combine two children at a time, but introduces an intermediate node that was not part of the original tree. Keep transformation metadata if you need to remove artificial nodes when reconstructing the result. NLTK’s PCFG API includes binarization support.
Conversion is not just a cosmetic rewrite when probabilities matter. A transformation must assign probabilities consistently so that it preserves the intended derivation scores. Epsilon rules (A -> ε), unary rules (A -> B), and start-symbol restrictions also need attention. The binary recurrence below does not handle them automatically. A parser must eliminate or transform such rules correctly, compute unary closure, or use an algorithm designed for its broader grammar.
What goes in the CKY chart
Let the sentence contain n tokens indexed from 0 through n−1. Use inclusive span boundaries: [i,j] covers tokens i through j. A chart state records the best score for category A spanning that region, written π(i,j,A). For a lexical rule, initialize a one-token span with its rule probability:
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchπ(i,i,A) = P(A -> wᵢ).
If the rule is absent, that entry is unavailable (equivalently, it has score zero, or negative infinity in log space). For every longer span, try all split points and binary rules:
Rank #3
π(i,j,A) = max [ P(A -> B C) × π(i,k,B) × π(k+1,j,C) ],
where the maximum ranges over rules A -> B C and split positions i ≤ k < j for which both child entries exist. Save the winning rule, split point, and child categories as a backpointer. These pointers make it possible to reconstruct the tree rather than return only its score. The recurrence and backpointer method are laid out in Michael Collins’s PCFG lecture notes.
A complete small example
Consider this toy grammar and the three-token input Alice likes Bob:
S -> NP VP [1.0]
VP -> V NP [1.0]
NP -> "Alice" [1.0]
V -> "likes" [1.0]
NP -> "Bob" [1.0]1. Initialize the single-token spans. The lexical rules produce π(0,0,NP)=1 for “Alice,” π(1,1,V)=1 for “likes,” and π(2,2,NP)=1 for “Bob.”
2. Build the two-token span. For [1,2], split after token 1. The rule VP -> V NP combines the entries for “likes” and “Bob”: π(1,2,VP)=1.0 × 1.0 × 1.0 = 1.0. Record that VP points to V over [1,1] and NP over [2,2].
3. Build the full span. For [0,2], split after token 0. S -> NP VP combines “Alice” with the VP over “likes Bob”: π(0,2,S)=1.0 × 1.0 × 1.0 = 1.0. The start symbol covers the complete input, so following the backpointers yields:
(S (NP Alice) (VP (V likes) (NP Bob)))With a grammar that licenses more than one tree, the parser evaluates competing rules and splits for each span and keeps only the highest-scoring candidate for each chart state. If no chart entry for the start symbol spans the entire sentence, the parser has found no parse under that grammar.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Viterbi CKY is not the inside algorithm
The maximum in the recurrence is deliberate: Viterbi CKY returns the single parse with the greatest tree probability, maxₜ P(t). It discards lower-scoring alternatives for the same span and category. That is appropriate when the goal is one best tree, but not when the task needs information about all possible parses.
The inside algorithm instead sums over alternatives, computing Σₜ P(t) for the parses of a span or sentence. This supports sentence-probability calculations and, with the outside algorithm, expected rule counts and constituent marginals. In short: use max for best-tree decoding; use sum when the total probability mass across parses matters. A best parse alone is not sufficient for posterior probabilities or other expected-value objectives.
Implementing CKY safely
Multiplying many small rule probabilities can underflow in floating-point arithmetic. A practical parser stores log probabilities instead. Since log(a × b) = log(a) + log(b), the recurrence becomes:
log π(i,j,A) = max [ log P(A -> B C) + log π(i,k,B) + log π(k+1,j,C) ].
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Represent a zero-probability or absent alternative as negative infinity; do not take the logarithm of zero. Log space changes the arithmetic, not the model or the choice of parse.
Best Value
A minimal Viterbi implementation follows this shape:
for each token position i:
for each lexical rule A -> token[i]:
chart[i, i, A] = log_probability(rule)
backpointer[i, i, A] = lexical rule
for span_length = 2 ... n:
for start = 0 ... n - span_length:
end = start + span_length - 1
for split = start ... end - 1:
for each binary rule A -> B C:
if chart[start, split, B] and chart[split + 1, end, C] exist:
candidate = log_probability(A -> B C)
+ chart[start, split, B]
+ chart[split + 1, end, C]
if candidate beats chart[start, end, A]:
chart[start, end, A] = candidate
backpointer[start, end, A] = (split, B, C, rule)
if chart[0, n - 1, S] exists:
reconstruct the tree by following its backpointers
else:
report no parseIn real code, use a representation that distinguishes a missing entry from a score of zero, keep scores and backpointers in a predictable structure, and index binary rules by their right-hand-side categories to avoid scanning irrelevant rules. Check the start symbol on the full span, and make tokenization and unknown-word handling explicit. If the grammar was binarized, postprocess artificial nodes. These checks catch common causes of surprising results: uncovered words, mismatched capitalization or quotes, unsupported unary rules, faulty span indexing, or incomplete backpointers.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Using NLTK for a small PCFG
NLTK provides PCFG construction and a Viterbi probabilistic parser. This deliberately artificial grammar illustrates the API; it is not a useful general English grammar:
import nltk
grammar = nltk.PCFG.fromstring("""
S -> NP VP [1.0]
VP -> V NP [1.0]
NP -> 'Alice' [1.0]
V -> 'likes' [1.0]
NP -> 'Bob' [1.0]
""")
parser = nltk.ViterbiParser(grammar)
for tree in parser.parse(["Alice", "likes", "Bob"]):
print(tree)The example uses nltk.ViterbiParser specifically; not every parser or chart parser in NLTK should be described as CKY. Consult the NLTK chapter on grammars and parsing, the Viterbi parser documentation, and the supplementary parsing material for the library’s documented interfaces and other parser strategies.
Complexity and practical trade-offs
For a binary grammar, a sentence of length n has O(n²) spans, and each span can have up to O(n) split points. The usual worst-case time summary is O(n³|G|), where the grammar factor depends on how rules and category combinations are represented. With a fixed compact grammar, this is often shortened to O(n³). The chart needs roughly O(n²|N|) space for one score per span and nonterminal, or O(n²) when the nonterminal set is fixed. Rule indexing, grammar sparsity, lexical ambiguity, unary closure, pruning, and sentence length all affect actual runtime and memory use. Stanford’s statistical parsing course places PCFGs and CKY alongside dynamic programming and grammar transformations.
CKY is a good fit when the goal is constituency parsing with a CFG-like grammar, exact Viterbi decoding, and a grammar that can be put into a supported binary form. It is valued as a transparent dynamic-programming method and remains foundational for understanding parsing and inference. It is not a complete language-understanding system, nor is it necessarily the best parser for every application. If you need incremental or prefix parsing, broad support for unary and epsilon rules, posterior marginals, or richer conditioning on lexical and contextual information, choose an algorithm or model built for those requirements. Probabilistic chart or A* approaches and neural constituency parsers are among the alternatives; the choice depends on the task and the desired output.
What the result can—and cannot—tell you
- The model is context-free in its rule choices. Basic rule probabilities do not condition on the surrounding words or full syntactic context, so they may miss lexical preferences, agreement, long-distance dependencies, or discourse effects.
- Coverage matters. A token without a lexical rule can prevent any parse, even when the intended syntax seems straightforward.
- Training data matters. Sparse observations and treebank annotation choices shape the probability estimates.
- Tree structure is not meaning. A high-scoring syntactic analysis is not a full semantic interpretation.
- Scores are model-relative. A parse probability is defined by the grammar’s derivations and parameters; do not read a score like 0.8 as “80% likely to be correct” without a justified probability space and calibration.
When the winning tree looks wrong, first verify the grammar, rule probabilities, tokens, and tree derivation. If those are sound, the problem may be the model’s assumptions or training data rather than the CKY recurrence.
Quick Recap
CKY debugging checklist
- For every left-hand category, do its rule probabilities sum to 1?
- Are all input tokens covered by lexical rules or an explicit unknown-word strategy?
- Does the grammar have only the rule forms your implementation supports?
- Are span boundaries and split points consistent with the chosen indexing convention?
- Are you intentionally taking a maximum (Viterbi) rather than summing alternatives (inside)?
- Are scores stored safely, usually in log space, with absent entries treated as negative infinity?
- Does every winning non-lexical entry retain a backpointer?
- Are artificial nodes from binarization removed or clearly marked in the output?
- Are you interpreting the result as the best parse under the model, not objective linguistic truth?
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
