dilations.md

A dilated stack has L levels; level j is a convolution with K taps spaced b^j apart, so a path from output to input picks one tap c_j in 0..K-1 at each level and reaches back n = sum_j c_j b^j steps. That is a base-b digit word with digits drawn from a design of K symbols (the core), the gains making it a weighted design; the words form its level-L Kronecker power, and the reach set is their image under n = sum_j c_j b^j. The effective receptive field weighs each lag by how many words reach it. Write Q(z) = sum_k q_k z^k for the block polynomial, q_k >= 0 the gain of tap k, and r_Q(n) = [z^n] prod_(j < L) Q(z^(b^j)) for the weighted digit count at level L.

The generator is the study lab/py/dilated-receptive-field, verbs count, limit, gradient and copy; the count carry for Q = 1 + z + z^2 at base 2 is the Stern-Brocot matrix pair of beneath, ### Stern-Brocot is a design with a rank-2 carry.

The field is a digit count

The model is linear: h^(j+1)_t = sum_k A_(j,k) h^(j)_(t - k b^j), C channels, a readout y_t = u . h^(L)_t, and every A_(j,k) an independent random C x C matrix. Each tap is either zero-mean with iid entries of variance sigma_k^2, or the identity plus such a matrix, which is a residual connection folded into the tap; at most one tap per level carries the identity. The gain of tap k is q_k = [k is the residual] + C sigma_k^2.

E norm(dy_t/dx_(t-n))^2 = norm(u)^2 r_Q(n), the squared gradient summed over input channels. Proved. The gradient is the sum over digit words reaching n of the matrix products along them. Two distinct words differ at some level j, where at most one of the two taps has nonzero mean, so the cross term vanishes by independence. For one word, tr E[A^T M A] = q_k tr M for every fixed M: the mean part gives tr M when it is the identity and 0 otherwise, and the zero-mean part gives C sigma_k^2 tr M. Starting from M = u u^T the trace picks up one factor q_(c_j) per level. A pure stack of K equal taps has Q = C sigma^2 (1 + z + ... + z^(K-1)) and field (C sigma^2)^L r(n). The undilated single-channel zero-mean case is Luo et al. (Luo et al. 2016, section 2.2, eq. 8): the gradient variance is the weight variance to the power of the depth times the path count.

Verified by Monte Carlo (gradient): 4000 draws of an 8-channel, 8-level stack reproduce r_Q with median relative error 0.020 and correlation 0.9989 over 511 lags for the pure K = 3 stack, and 0.022 and 0.9999 over 256 lags for the residual block Q = 1 + c(1 + z) at c = 1/2. The pure stack's z-scores run heavy-tailed: 1.96% of its lags sit past 3. The product formula itself agrees with the digit recursion r_L(n) = sum_(c = n mod b) q_c r_(L-1)((n - c)/b) on every lag of five block polynomials (count).

Stern in the plain stack

Take Q = 1 + z + z^2 and b = 2, the plain three-tap stack with doubling dilations. Its field spans the 2^(L+1) - 1 lags 0..2^(L+1) - 2 with no hole, and r sums to 3^L.

r(n) = s(n + 1) for 0 <= n < 2^L, s Stern's diatomic sequence (A002487), and r(n) = r(2^(L+1) - 2 - n) above. Proved. Level-L words are the representations of n as a sum of powers of 2, each used at most twice, with powers below 2^L. Every such representation of n < 2^L uses only powers below 2^L, so r(n) is the full hyperbinary count, which is s(n + 1). The mirror holds because Q is palindromic, so the product is too, of degree 2^(L+1) - 2. Verified exactly for L = 1..20 (count), and the first 32 terms of A002487 as listed agree with the recursion.

The maximum of r is the Fibonacci number F_(L+1). Proved. On the row 2^(k-1) <= m < 2^k, induction on k with s(2m) = s(m) and s(2m+1) = s(m) + s(m+1) gives s(m) <= F_(k+1) and s(m) + s(m+1) <= F_(k+2), the pair sums obeying the Fibonacci recursion. The bound is attained at m_k = (2^(k+1) + (-1)^k)/3: from m_(k+1) = 2 m_k - (-1)^k the same two rules carry s(m_k) = F_(k+1) and s(m_k - (-1)^k) = F_k from k to k + 1, starting at s(5) = 3, s(6) = 2. So the maximum over 1 <= m <= 2^L is F_(L+1). It sits at exactly the four lags m - 1, 3 . 2^(L-1) - m - 1 and their mirrors, m = (2^(L+1) + (-1)^L)/3. Verified for L = 3..20 (count); at L = 10 these are 682, 852, 1194, 1364, the outer two near 2^(L+1)/3 and 2^(L+2)/3, the inner two near 5 . 2^(L+1)/12 and 7 . 2^(L+1)/12.

Exactly 2L + 1 lags carry a single path: 2^k - 1 and 2^(L+1) - 1 - 2^k for k = 0..L. Proved. s(m) = 1 exactly when m is a power of 2, since s(2m + 1) = s(m) + s(m + 1) >= 2 for m >= 1; the powers 1..2^L give L + 1 lags in the lower half, the mirror gives L + 1 more, and the two lists share only the centre 2^L - 1. At L = 10 that is 21. The count 2(L + 1) + 1 is Refuted: it counts the centre three times, and the census reads 2L + 1 at every L = 1..20 (count).

Peak over mean is F_(L+1)(2^(L+1) - 1)/3^L, and its ratio from level L to L + 1 tends to 2 phi/3 = 1.07869. Proved: the mean is the total 3^L over the lag count, and F_(L+2)/F_(L+1) -> phi. The ratio is rational at every level, so it only tends to the limit: the generator prints 1.07143 at L = 3, 1.14815 at L = 4, and 1.07869 from L = 17, with peak over mean 3.0853 at L = 10 and 6.5835 at L = 20.

When the limit is smooth

Rescale lags by b^L and normalise by Q(1)^L: the field becomes the law of X = sum_(i >= 1) c_i b^(-i), digits c_i iid with P(c = k) = q_k/Q(1), as L grows. Call that law mu_Q. Its Fourier coefficients are mu_Q^(t) = prod_(i >= 1) Q(e^(2 pi i t/b^i))/Q(1), and at integers mu_Q^(b t) = mu_Q^(t).

mu_Q is absolutely continuous if and only if, for every integer k not divisible by b, Q vanishes at e^(2 pi i k/b^i) for some i >= 1. Proved. If mu_Q has a density, Riemann-Lebesgue sends mu_Q^(k b^n) = mu_Q^(k) to 0, so every coefficient at a nonzero integer vanishes; the product converges absolutely, so a coefficient vanishes only where a factor does. Conversely, if every such coefficient is 0 the periodisation of mu_Q to the circle is Lebesgue measure, and a null set of the line maps to a null set of the circle, so mu_Q has a density. Otherwise mu_Q is purely singular: it is the one probability fixed point of (T nu)(E) = sum_k (q_k/Q(1)) nu(b E - k), T sends absolutely continuous measures to absolutely continuous ones and singular to singular, so the two parts of mu_Q are fixed separately and a nonzero density part would be a second fixed point after normalising. The condition depends on k modulo b^i and Q has finitely many roots, so it is a finite check. For prime b and rational gains the primitive roots of one order are conjugate over the rationals, and it reads: some cyclotomic factor Phi_(b^i) divides Q. The distribution function of mu_Q solves a refinement equation with nonnegative coefficients; Protasov (Protasov 2000) gives a criterion for absolutely continuous solutions and classifies the positive case, read in abstract only, and the case K = b is the digit-tile setting of Lagarias and Wang (Lagarias and Wang 1996), read in its bibliographic record only. The proof above stands alone. For these measures being a Rajchman measure and being absolutely continuous coincide, since mu_Q^(k b^n) never decays.

For real gains the cyclotomic form is Refuted. Q = (z^2 - sqrt2 z + 1)(z^4 + sqrt2 z^2 + 1)(1 + z + z^2)^2 has positive coefficients, the least 2 - sqrt2, and on the circle vanishes at the turns 1/8, 7/8, 3/16, 5/16, 11/16, 13/16 and 1/3, 2/3. At b = 2 an odd k is 1 or 7 modulo 8, covered at i = 3, or 3 or 5 modulo 8, covered at i = 4, so mu_Q is absolutely continuous. Yet Q(-1), Q(i), Q(e^(3 pi i/4)) and Q(e^(pi i/8)) are nonzero and Phi_(2^i) has degree above 10 for i >= 5, so no Phi_(2^i) divides Q. The limit verb prints the cover levels 3, 4 and the Fourier product below 2.1e-18 at every odd t < 256.

For Q = 1 + z + z^2, b = 2 the limit lives on [0, 2], and its fold modulo 1 carries the Stern rows: s(2^L + j) = r(j - 1) + r(j - 1 + 2^L) for 0 <= j < 2^L. Proved: a hyperbinary representation of 2^L + j - 1 uses 2^L once or not at all, and the generator checks it for L = 1..20 (count). Baake and Coons (Baake and Coons 2018) build a probability measure on the unit interval from Stern's sequence between consecutive powers of 2 and prove it purely singular, read at source in its abstract; the fold is the measure those rows define, and a measure and its fold are singular together.

Verified on 35 block polynomials (limit): the exact cyclotomic check in rational arithmetic, depth 6, agrees on every row with the Fourier product, which is below 1e-15 on every absolutely continuous row and at least 1.8e-3 on every singular one. Both sides read the same roots, so this checks the generator's arithmetic, not the dichotomy. The finite trace of the dichotomy is where the mass sits: half the mass of the K = 3, b = 2 field lies on 27.4% of its lags at L = 8 and 22.8% at L = 20, still falling, while K = 4 holds at 33.33% from L = 12.

Four stacks

Uniform taps give an absolutely continuous limit if and only if b divides K, and, for L >= 2, a flat field r = 1 exactly when K = b. Proved. At L = 1 every uniform field is flat. The roots of 1 + ... + z^(K-1) are the K-th roots of unity other than 1; e^(2 pi i k/b) is one of them for every k not divisible by b when b divides K, and e^(2 pi i/b^i) is one only if b^i divides K, which fails at k = 1 when b does not divide K. At K = b the product telescopes to (1 - z^(b^L))/(1 - z); at 2 <= K != b and L >= 2 a constant field would be r(0) = 1 everywhere, yet the total K^L differs from the lag count (K - 1)(b^L - 1)/(b - 1) + 1, since their difference is strictly convex in K with roots 1 and b, so the field is not constant. So K = 4 at b = 2 and K = 3 at b = 3 are smooth, and the plain K = 3, b = 2 stack is not.

A residual block Q = 1 + c(1 + z) at b = 2, the shape of a WaveNet-style layer, gives a singular limit for every c > 0. Proved. The only root is -(1 + c)/c, of modulus above 1, so Q never vanishes on the circle. The pure K = 2 stack is flat, and the skip connection alone breaks it: at c = 1/4 half the mass sits on 0.10% of the lags at L = 20 (limit).

A double-convolution residual block Q = 1 + c(1 + z + z^2)^2 at b = 2 is absolutely continuous exactly at c = 1. Proved. On the circle 1 + z + z^2 = z(1 + 2 cos theta), so Q(z) = 0 forces z^2 = -1 and c (1 + 2 cos theta)^2 = 1 with cos theta = 0, that is z = +-i and c = 1. At c = 1, Q = (1 + z^2)(2 + 2z + z^2) carries Phi_4, whose two roots cover both odd residues modulo 4. The digit count covers this block: at each level a word takes the skip or a pair of taps (a, a'), one from each convolution; two different choices differ in an independent zero-mean factor, so cross terms vanish, and a pair gives tr E[A_1^T A_2^T M A_2 A_1] = q_1 q_2 tr M. With equal gains q_1 and q_2 across each convolution's taps, c = q_1 q_2, so any gain product other than 1 leaves a singular limit; the block without its skip, (1 + z + z^2)^2, is singular too.

The square stack of 3 x 3 convolutions at dilations 2^j in both axes, its nine taps of equal gain, has field s(m + 1) s(n + 1) on its lower quadrant, a Stern plaid with peak F_(L+1)^2 and its single-path cells where the single-path lags of the two axes cross, (2L + 1)^2 of them on the whole field. Proved: the proof of the digit count runs unchanged on two-dimensional lags, and Q(z, w) = (1 + z + z^2)(1 + w + w^2) factors, so the count is the product of the two axis counts above.

The copy task

A lag-n copy task asks the stack for y_t = x_(t-n). On white input its loss is the squared distance of the end-to-end filter from the spike at n, and at initialisation the expected gradient pull on the filter at n, summed over parameters, is proportional to r(n) by the same cancellation of cross terms, so a lazy picture predicts steps to learn n falling like 1/r(n).

The 1/r(n) law is Refuted at one setting (copy): a linear K = 3, b = 2 stack, L = 8, 8 channels, full-batch gradient descent at rate 0.005, five seeds, counting steps until the filter at n reaches 0.5, at eight lags in 128..255 with r from 1 to 34. The median steps fall from 615 at r = 1 to 94 at r = 34, a factor 6.5 against 34, and the log-log slope is 0.55, not 1. Steps to learn a lag fall monotonically with r(n), roughly as r(n)^(-1/2). Conjecture: the eight medians fall monotonically at both scales tried, with correlation 0.984 against log 1/r at the default scale; seed spread is wide, and at half the initial scale the medians fall from 15760 to 3788, slope 0.40 and correlation 0.983 over all eight lags, one run at r = 1 capped at 20000 steps and read as above the cap. The exponent moves with the initial scale, so it is a fit, not a law.

Where the numbers live

• lab/py/dilated-receptive-field computes every number on this page: count the exact integer field against the digit recursion, Stern and Fibonacci, limit the cyclotomic check and the Fourier product, gradient the Monte Carlo field, copy the training runs.
• The Stern carry and Minkowski's ? are in beneath; the growth of a peak as a product of carry matrices is the joint spectral radius of connectivity.
• The concepts are on the wiki: transfer matrix, Kronecker product, Rajchman measure.