Figure 5. a-c, The linear dependence of the classification capacity of the recurrent readout P on the number of input neurons N, when the number of intermediate perceptrons M is increased proportionally to N, so that c = remains constant (we assumed c = 5). The red and green lines correspond to dense (f = 0.5) and sparse (f = 0.01) representations. The number of feedforward connections per perceptron is CF = 50, and the tolerated error rate is ϵ = 0.05. a, High-noise regime: the noise is large compared with the feedforward input. For the dense case (red line), β = 0.04, and for the sparse case (green line) β = 0.9; these choices correspond to a ratio of the noise to feedforward input equal to 10. b, Intermediate level of noise: the noise is low compared with the feedforward input, but large when compared with the input from the input receiving to the free neurons in the case of sparse input representations (two-subnetwork regime). The red line corresponds to dense input representations (uniform low-noise regime), and the green line corresponds to the two-subnetwork intermediate-noise regime. c, Low level of noise. The red line corresponds to the uniform low-noise regime, and the green line corresponds to the two-subnetwork low-noise regime (same as majority vote). d–f, Change of the slope of the plots from a to c, P/N with the coding level f for different values of c. d, High-noise regime. Different curves correspond to different numbers of perceptrons M per input neuron, expressed as c = . The noise parameter β and the strength of the recurrent synapses α are varied with the coding level f to keep the value of ΔUH = 0.2 and the inequality of Equation 3.42 satisfied by the factor of 10 for every value of f. The last condition implies that the ratio of the noise to the amplitude of the feedforward input is equal to 10 for every point on the curve. e, Intermediate level of noise. The low-f segments of the curves represent the two-subnetwork intermediate-noise regime. Either the noise parameter β or the strength of the recurrent synapses α is varied with f to keep ΔTI = 0.2. The high-f segments correspond to the uniform low-noise regime, and α is varied with f so that ΔUL = 0.2. f, Low noise. Low-f segments of the curves correspond to the two-subnetwork low-noise regime (same as majority vote), the high-f segments are the same as in panel e. The dashed green line shows the performance of the fully connected readout for comparison. The green and red points on the c = 5 curve correspond to the values of f used in a–c. The curves on e and f are discontinuous because there is no consistent way to analyze the recurrent dynamics in the perceptron layer across the entire range of f for these levels of noise. However, we believe that the capacity changes smoothly across the unexplored region, achieving its maximum at approximately f ≈ 0.05 for CF = 50.