--- title: Dilated stacks lead: The effective receptive field of a dilated convolution stack is a digit-design representation count: Stern's diatomic sequence for three taps at base 2, with Fibonacci peaks and `2L + 1` single-path lags, and a depth limit that is absolutely continuous exactly when the block polynomial vanishes at the right roots of unity, which the two common residual blocks break at every gain but at most one. figure: research-dilations slug: dilations --- A dilated stack has `L` levels; level `j` is a convolution with `K` taps spaced `b^j` apart, so a path from output to input picks one tap `c_j` in `0..K-1` at each level and reaches back `n = sum_j c_j b^j` steps. That is a base-`b` digit word with digits drawn from a design of `K` symbols ([the core](core.md)), the gains making it a [weighted design](weights.md); the words form its level-`L` [Kronecker power](/wiki/kronecker-product/), and the reach set is their image under `n = sum_j c_j b^j`. The effective receptive field weighs each lag by how many words reach it. Write `Q(z) = sum_k q_k z^k` for the block polynomial, `q_k >= 0` the gain of tap `k`, and `r_Q(n) = [z^n] prod_(j < L) Q(z^(b^j))` for the weighted digit count at level `L`. The generator is the study `lab/py/dilated-receptive-field`, verbs `count`, `limit`, `gradient` and `copy`; the count carry for `Q = 1 + z + z^2` at base `2` is the Stern-Brocot matrix pair of [beneath](beneath.md), `### Stern-Brocot is a design with a rank-2 carry`. ## The field is a digit count The model is linear: `h^(j+1)_t = sum_k A_(j,k) h^(j)_(t - k b^j)`, `C` channels, a readout `y_t = u . h^(L)_t`, and every `A_(j,k)` an independent random `C x C` matrix. Each tap is either zero-mean with iid entries of variance `sigma_k^2`, or the identity plus such a matrix, which is a residual connection folded into the tap; at most one tap per level carries the identity. The gain of tap `k` is `q_k = [k is the residual] + C sigma_k^2`. **`E norm(dy_t/dx_(t-n))^2 = norm(u)^2 r_Q(n)`, the squared gradient summed over input channels. Proved.** The gradient is the sum over digit words reaching `n` of the matrix products along them. Two distinct words differ at some level `j`, where at most one of the two taps has nonzero mean, so the cross term vanishes by independence. For one word, `tr E[A^T M A] = q_k tr M` for every fixed `M`: the mean part gives `tr M` when it is the identity and `0` otherwise, and the zero-mean part gives `C sigma_k^2 tr M`. Starting from `M = u u^T` the trace picks up one factor `q_(c_j)` per level. A pure stack of `K` equal taps has `Q = C sigma^2 (1 + z + ... + z^(K-1))` and field `(C sigma^2)^L r(n)`. The undilated single-channel zero-mean case is Luo et al. ([Luo et al. 2016](../REFS.md), section 2.2, eq. 8): the gradient variance is the weight variance to the power of the depth times the path count. **Verified** by Monte Carlo (`gradient`): `4000` draws of an `8`-channel, `8`-level stack reproduce `r_Q` with median relative error `0.020` and correlation `0.9989` over `511` lags for the pure `K = 3` stack, and `0.022` and `0.9999` over `256` lags for the residual block `Q = 1 + c(1 + z)` at `c = 1/2`. The pure stack's z-scores run heavy-tailed: `1.96%` of its lags sit past `3`. The product formula itself agrees with the digit recursion `r_L(n) = sum_(c = n mod b) q_c r_(L-1)((n - c)/b)` on every lag of five block polynomials (`count`). ## Stern in the plain stack Take `Q = 1 + z + z^2` and `b = 2`, the plain three-tap stack with doubling dilations. Its field spans the `2^(L+1) - 1` lags `0..2^(L+1) - 2` with no hole, and `r` sums to `3^L`. **`r(n) = s(n + 1)` for `0 <= n < 2^L`, `s` Stern's diatomic sequence ([A002487](https://oeis.org/A002487)), and `r(n) = r(2^(L+1) - 2 - n)` above. Proved.** Level-`L` words are the representations of `n` as a sum of powers of `2`, each used at most twice, with powers below `2^L`. Every such representation of `n < 2^L` uses only powers below `2^L`, so `r(n)` is the full hyperbinary count, which is `s(n + 1)`. The mirror holds because `Q` is palindromic, so the product is too, of degree `2^(L+1) - 2`. **Verified** exactly for `L = 1..20` (`count`), and the first `32` terms of A002487 as listed agree with the recursion. **The maximum of `r` is the Fibonacci number `F_(L+1)`. Proved.** On the row `2^(k-1) <= m < 2^k`, induction on `k` with `s(2m) = s(m)` and `s(2m+1) = s(m) + s(m+1)` gives `s(m) <= F_(k+1)` and `s(m) + s(m+1) <= F_(k+2)`, the pair sums obeying the Fibonacci recursion. The bound is attained at `m_k = (2^(k+1) + (-1)^k)/3`: from `m_(k+1) = 2 m_k - (-1)^k` the same two rules carry `s(m_k) = F_(k+1)` and `s(m_k - (-1)^k) = F_k` from `k` to `k + 1`, starting at `s(5) = 3`, `s(6) = 2`. So the maximum over `1 <= m <= 2^L` is `F_(L+1)`. **It sits at exactly the four lags `m - 1`, `3 . 2^(L-1) - m - 1` and their mirrors, `m = (2^(L+1) + (-1)^L)/3`. Verified** for `L = 3..20` (`count`); at `L = 10` these are `682`, `852`, `1194`, `1364`, the outer two near `2^(L+1)/3` and `2^(L+2)/3`, the inner two near `5 . 2^(L+1)/12` and `7 . 2^(L+1)/12`. **Exactly `2L + 1` lags carry a single path: `2^k - 1` and `2^(L+1) - 1 - 2^k` for `k = 0..L`. Proved.** `s(m) = 1` exactly when `m` is a power of `2`, since `s(2m + 1) = s(m) + s(m + 1) >= 2` for `m >= 1`; the powers `1..2^L` give `L + 1` lags in the lower half, the mirror gives `L + 1` more, and the two lists share only the centre `2^L - 1`. At `L = 10` that is `21`. **The count `2(L + 1) + 1` is Refuted**: it counts the centre three times, and the census reads `2L + 1` at every `L = 1..20` (`count`). **Peak over mean is `F_(L+1)(2^(L+1) - 1)/3^L`, and its ratio from level `L` to `L + 1` tends to `2 phi/3 = 1.07869`. Proved**: the mean is the total `3^L` over the lag count, and `F_(L+2)/F_(L+1) -> phi`. The ratio is rational at every level, so it only tends to the limit: the generator prints `1.07143` at `L = 3`, `1.14815` at `L = 4`, and `1.07869` from `L = 17`, with peak over mean `3.0853` at `L = 10` and `6.5835` at `L = 20`. ## When the limit is smooth Rescale lags by `b^L` and normalise by `Q(1)^L`: the field becomes the law of `X = sum_(i >= 1) c_i b^(-i)`, digits `c_i` iid with `P(c = k) = q_k/Q(1)`, as `L` grows. Call that law `mu_Q`. Its Fourier coefficients are `mu_Q^(t) = prod_(i >= 1) Q(e^(2 pi i t/b^i))/Q(1)`, and at integers `mu_Q^(b t) = mu_Q^(t)`. **`mu_Q` is absolutely continuous if and only if, for every integer `k` not divisible by `b`, `Q` vanishes at `e^(2 pi i k/b^i)` for some `i >= 1`. Proved.** If `mu_Q` has a density, Riemann-Lebesgue sends `mu_Q^(k b^n) = mu_Q^(k)` to `0`, so every coefficient at a nonzero integer vanishes; the product converges absolutely, so a coefficient vanishes only where a factor does. Conversely, if every such coefficient is `0` the periodisation of `mu_Q` to the circle is Lebesgue measure, and a null set of the line maps to a null set of the circle, so `mu_Q` has a density. Otherwise `mu_Q` is purely singular: it is the one probability fixed point of `(T nu)(E) = sum_k (q_k/Q(1)) nu(b E - k)`, `T` sends absolutely continuous measures to absolutely continuous ones and singular to singular, so the two parts of `mu_Q` are fixed separately and a nonzero density part would be a second fixed point after normalising. The condition depends on `k` modulo `b^i` and `Q` has finitely many roots, so it is a finite check. For prime `b` and rational gains the primitive roots of one order are conjugate over the rationals, and it reads: some cyclotomic factor `Phi_(b^i)` divides `Q`. The distribution function of `mu_Q` solves a refinement equation with nonnegative coefficients; Protasov ([Protasov 2000](../REFS.md)) gives a criterion for absolutely continuous solutions and classifies the positive case, read in abstract only, and the case `K = b` is the digit-tile setting of Lagarias and Wang ([Lagarias and Wang 1996](../REFS.md)), read in its bibliographic record only. The proof above stands alone. For these measures being a [Rajchman measure](/wiki/rajchman-measure/) and being absolutely continuous coincide, since `mu_Q^(k b^n)` never decays. **For real gains the cyclotomic form is Refuted.** `Q = (z^2 - sqrt2 z + 1)(z^4 + sqrt2 z^2 + 1)(1 + z + z^2)^2` has positive coefficients, the least `2 - sqrt2`, and on the circle vanishes at the turns `1/8`, `7/8`, `3/16`, `5/16`, `11/16`, `13/16` and `1/3`, `2/3`. At `b = 2` an odd `k` is `1` or `7` modulo `8`, covered at `i = 3`, or `3` or `5` modulo `8`, covered at `i = 4`, so `mu_Q` is absolutely continuous. Yet `Q(-1)`, `Q(i)`, `Q(e^(3 pi i/4))` and `Q(e^(pi i/8))` are nonzero and `Phi_(2^i)` has degree above `10` for `i >= 5`, so no `Phi_(2^i)` divides `Q`. The `limit` verb prints the cover levels `3, 4` and the Fourier product below `2.1e-18` at every odd `t < 256`. **For `Q = 1 + z + z^2`, `b = 2` the limit lives on `[0, 2]`, and its fold modulo `1` carries the Stern rows: `s(2^L + j) = r(j - 1) + r(j - 1 + 2^L)` for `0 <= j < 2^L`. Proved**: a hyperbinary representation of `2^L + j - 1` uses `2^L` once or not at all, and the generator checks it for `L = 1..20` (`count`). Baake and Coons ([Baake and Coons 2018](../REFS.md)) build a probability measure on the unit interval from Stern's sequence between consecutive powers of `2` and prove it purely singular, read at source in its abstract; the fold is the measure those rows define, and a measure and its fold are singular together. **Verified** on `35` block polynomials (`limit`): the exact cyclotomic check in rational arithmetic, depth `6`, agrees on every row with the Fourier product, which is below `1e-15` on every absolutely continuous row and at least `1.8e-3` on every singular one. Both sides read the same roots, so this checks the generator's arithmetic, not the dichotomy. The finite trace of the dichotomy is where the mass sits: half the mass of the `K = 3`, `b = 2` field lies on `27.4%` of its lags at `L = 8` and `22.8%` at `L = 20`, still falling, while `K = 4` holds at `33.33%` from `L = 12`. ## Four stacks **Uniform taps give an absolutely continuous limit if and only if `b` divides `K`, and, for `L >= 2`, a flat field `r = 1` exactly when `K = b`. Proved.** At `L = 1` every uniform field is flat. The roots of `1 + ... + z^(K-1)` are the `K`-th roots of unity other than `1`; `e^(2 pi i k/b)` is one of them for every `k` not divisible by `b` when `b` divides `K`, and `e^(2 pi i/b^i)` is one only if `b^i` divides `K`, which fails at `k = 1` when `b` does not divide `K`. At `K = b` the product telescopes to `(1 - z^(b^L))/(1 - z)`; at `2 <= K != b` and `L >= 2` a constant field would be `r(0) = 1` everywhere, yet the total `K^L` differs from the lag count `(K - 1)(b^L - 1)/(b - 1) + 1`, since their difference is strictly convex in `K` with roots `1` and `b`, so the field is not constant. So `K = 4` at `b = 2` and `K = 3` at `b = 3` are smooth, and the plain `K = 3`, `b = 2` stack is not. **A residual block `Q = 1 + c(1 + z)` at `b = 2`, the shape of a WaveNet-style layer, gives a singular limit for every `c > 0`. Proved.** The only root is `-(1 + c)/c`, of modulus above `1`, so `Q` never vanishes on the circle. The pure `K = 2` stack is flat, and the skip connection alone breaks it: at `c = 1/4` half the mass sits on `0.10%` of the lags at `L = 20` (`limit`). **A double-convolution residual block `Q = 1 + c(1 + z + z^2)^2` at `b = 2` is absolutely continuous exactly at `c = 1`. Proved.** On the circle `1 + z + z^2 = z(1 + 2 cos theta)`, so `Q(z) = 0` forces `z^2 = -1` and `c (1 + 2 cos theta)^2 = 1` with `cos theta = 0`, that is `z = +-i` and `c = 1`. At `c = 1`, `Q = (1 + z^2)(2 + 2z + z^2)` carries `Phi_4`, whose two roots cover both odd residues modulo `4`. The digit count covers this block: at each level a word takes the skip or a pair of taps `(a, a')`, one from each convolution; two different choices differ in an independent zero-mean factor, so cross terms vanish, and a pair gives `tr E[A_1^T A_2^T M A_2 A_1] = q_1 q_2 tr M`. With equal gains `q_1` and `q_2` across each convolution's taps, `c = q_1 q_2`, so any gain product other than `1` leaves a singular limit; the block without its skip, `(1 + z + z^2)^2`, is singular too. **The square stack of `3 x 3` convolutions at dilations `2^j` in both axes, its nine taps of equal gain, has field `s(m + 1) s(n + 1)` on its lower quadrant, a Stern plaid with peak `F_(L+1)^2` and its single-path cells where the single-path lags of the two axes cross, `(2L + 1)^2` of them on the whole field. Proved**: the proof of the digit count runs unchanged on two-dimensional lags, and `Q(z, w) = (1 + z + z^2)(1 + w + w^2)` factors, so the count is the product of the two axis counts above. ## The copy task A lag-`n` copy task asks the stack for `y_t = x_(t-n)`. On white input its loss is the squared distance of the end-to-end filter from the spike at `n`, and at initialisation the expected gradient pull on the filter at `n`, summed over parameters, is proportional to `r(n)` by the same cancellation of cross terms, so a lazy picture predicts steps to learn `n` falling like `1/r(n)`. **The `1/r(n)` law is Refuted** at one setting (`copy`): a linear `K = 3`, `b = 2` stack, `L = 8`, `8` channels, full-batch gradient descent at rate `0.005`, five seeds, counting steps until the filter at `n` reaches `0.5`, at eight lags in `128..255` with `r` from `1` to `34`. The median steps fall from `615` at `r = 1` to `94` at `r = 34`, a factor `6.5` against `34`, and the log-log slope is `0.55`, not `1`. **Steps to learn a lag fall monotonically with `r(n)`, roughly as `r(n)^(-1/2)`. Conjecture**: the eight medians fall monotonically at both scales tried, with correlation `0.984` against `log 1/r` at the default scale; seed spread is wide, and at half the initial scale the medians fall from `15760` to `3788`, slope `0.40` and correlation `0.983` over all eight lags, one run at `r = 1` capped at `20000` steps and read as above the cap. The exponent moves with the initial scale, so it is a fit, not a law. ## Where the numbers live - `lab/py/dilated-receptive-field` computes every number on this page: `count` the exact integer field against the digit recursion, Stern and Fibonacci, `limit` the cyclotomic check and the Fourier product, `gradient` the Monte Carlo field, `copy` the training runs. - The Stern carry and Minkowski's `?` are in [beneath](beneath.md); the growth of a peak as a product of carry matrices is the joint spectral radius of [connectivity](connectivity.md). - The concepts are on the wiki: [transfer matrix](/wiki/transfer-matrix/), [Kronecker product](/wiki/kronecker-product/), [Rajchman measure](/wiki/rajchman-measure/).