dilated-receptive-fields.md

6.1 kB · markdown

Dilated receptive fields

  • 2026-09-23 [Proved] For a linear dilated stack at levels j < L with dilation b^j, independent random C x C taps, each zero-mean with iid entries of variance sigma_k^2 or, at most one per level, the identity plus such a matrix, E norm(dy_t/dx_(t-n))^2 = norm(u)^2 r_Q(n), where q_k = [k is the residual] + C sigma_k^2 and r_Q(n) = [z^n] prod_(j < L) Q(z^(b^j)) is the weighted base-b digit count; the undilated single-channel zero-mean case is Luo et al., section 2.2. Witness: notes/dilations.md, The field is a digit count.
  • 2026-09-23 [Verified] Monte Carlo over 4000 random 8-channel 8-level stacks reproduces r_Q with median relative error 0.020 and correlation 0.9989 over 511 lags for Q = 1 + z + z^2, and 0.022 and 0.9999 over 256 lags for Q = 1 + c(1 + z) at c = 1/2; the pure stack's z-scores run heavy-tailed, 1.96% of its lags past 3. Witness: lab/py/dilated-receptive-field gradient.
  • 2026-09-23 [Proved] For Q = 1 + z + z^2 and b = 2 the field is Stern's diatomic sequence, r(n) = s(n + 1) for 0 <= n < 2^L (A002487), mirrored as r(n) = r(2^(L+1) - 2 - n) above, recomputed exactly for L = 1..20. Witness: notes/dilations.md, Stern in the plain stack; lab/py/dilated-receptive-field count.
  • 2026-09-23 [Proved] The maximum of the Q = 1 + z + z^2, b = 2 field at level L is the Fibonacci number F_(L+1), attained at n = m - 1 with m = (2^(L+1) + (-1)^L)/3. Witness: notes/dilations.md, Stern in the plain stack.
  • 2026-09-23 [Verified] The maximum F_(L+1) sits at exactly the four lags m - 1, 3 . 2^(L-1) - m - 1 and their mirrors for L = 3..20, at 682, 852, 1194, 1364 for L = 10. Witness: lab/py/dilated-receptive-field count.
  • 2026-09-23 [Proved] Exactly 2L + 1 lags of the Q = 1 + z + z^2, b = 2 field carry a single path, 2^k - 1 and 2^(L+1) - 1 - 2^k for k = 0..L, 21 at L = 10. Witness: notes/dilations.md, Stern in the plain stack; lab/py/dilated-receptive-field count.
  • 2026-09-23 [Refuted] The single-path lag count 2(L + 1) + 1 for the Q = 1 + z + z^2, b = 2 field: it counts the centre 2^L - 1 three times, and the exact census reads 2L + 1 at every L = 1..20. Witness: lab/py/dilated-receptive-field count.
  • 2026-09-23 [Proved] Peak over mean of the Q = 1 + z + z^2, b = 2 field is F_(L+1)(2^(L+1) - 1)/3^L, and its ratio from level L to L + 1 tends to 2 phi/3 = 1.07869, rational at every level, 1.07143 at L = 3 and 1.14815 at L = 4; peak over mean is 3.0853 at L = 10 and 6.5835 at L = 20. Witness: notes/dilations.md, Stern in the plain stack; lab/py/dilated-receptive-field count.
  • 2026-09-23 [Proved] The depth limit mu_Q of a dilated field with nonnegative block polynomial Q at base b is absolutely continuous if and only if for every integer k not divisible by b some Q(e^(2 pi i k/b^i)) vanishes, i >= 1, and is purely singular otherwise; for prime b and rational gains the test is that some cyclotomic Phi_(b^i) divides Q. Witness: notes/dilations.md, When the limit is smooth.
  • 2026-09-23 [Refuted] For prime b and real nonnegative gains, an absolutely continuous depth limit forces some Phi_(b^i) to divide Q: Q = (z^2 - sqrt2 z + 1)(z^4 + sqrt2 z^2 + 1)(1 + z + z^2)^2 at b = 2 has least coefficient 2 - sqrt2, covers every odd k at i = 3 or i = 4, so its limit is absolutely continuous, and is nonzero at -1, i, e^(3 pi i/4) and e^(pi i/8), so no Phi_(2^i) divides it. Witness: notes/dilations.md, When the limit is smooth; lab/py/dilated-receptive-field limit.
  • 2026-09-23 [Proved] For Q = 1 + z + z^2 and b = 2 the fold modulo 1 of the depth limit carries the Stern rows, s(2^L + j) = r(j - 1) + r(j - 1 + 2^L) for 0 <= j < 2^L, checked for L = 1..20. Witness: notes/dilations.md, When the limit is smooth; lab/py/dilated-receptive-field count.
  • 2026-09-23 [Verified] The exact cyclotomic check agrees with the Fourier product prod_i Q(e^(2 pi i t/b^i))/Q(1) on all 35 block polynomials, 21 uniform, 6 single-convolution residual, 7 double-convolution residual and (1 + z + z^2)^2, the product below 1e-15 on every absolutely continuous row and at least 1.8e-3 on every singular one, a check of the generator's arithmetic since both sides read the same roots. Witness: lab/py/dilated-receptive-field limit.
  • 2026-09-23 [Proved] A dilated stack of K >= 2 uniform taps at base b has an absolutely continuous depth limit if and only if b divides K, and for L >= 2 a constant field exactly when K = b; at L = 1 every uniform field is constant. Witness: notes/dilations.md, Four stacks.
  • 2026-09-23 [Proved] The residual block Q = 1 + c(1 + z) at b = 2 has a singular depth limit for every gain c > 0, while the pure K = 2 stack is flat. Witness: notes/dilations.md, Four stacks.
  • 2026-09-23 [Proved] The double-convolution residual block Q = 1 + c(1 + z + z^2)^2 at b = 2 has an absolutely continuous depth limit exactly at c = 1, where Phi_4 divides Q; with equal gains q_1, q_2 across each convolution's taps the digit count covers the block and c = q_1 q_2. Witness: notes/dilations.md, Four stacks.
  • 2026-09-23 [Proved] The square stack of 3 x 3 convolutions at dilations 2^j in both axes, its nine taps of equal gain, has field s(m + 1) s(n + 1) on its lower quadrant, peak F_(L+1)^2, with (2L + 1)^2 single-path cells. Witness: notes/dilations.md, Four stacks.
  • 2026-09-23 [Refuted] Steps to learn a lag-n copy task scale like 1/r(n): a linear K = 3, b = 2, L = 8, 8-channel stack under full-batch gradient descent takes median 615 steps at r = 1 and 94 at r = 34, a factor 6.5 against 34, log-log slope 0.55. Witness: lab/py/dilated-receptive-field copy.
  • 2026-09-23 [Conjecture] Steps to learn a lag-n copy task in a linear K = 3, b = 2 stack fall monotonically with r(n), roughly as r(n)^(-1/2): at the default initial scale eight medians, slope 0.55, correlation 0.984 with log 1/r; at half the scale slope 0.40, correlation 0.983 over the same eight lags, one run capped and read as above the cap; the exponent moves with the scale. Witness: lab/py/dilated-receptive-field copy.