The Pixels That Matter

Twelve of the 64 pixels in this dataset have a training variance under 0.5. Three of them never change at all, variance exactly zero across all 1,347 training images. They sit on the left and right edges of the 8x8 box, the columns nobody’s pen reaches. I went looking for them because I wanted to know how lopsided the signal really is. If a fifth of the image is constant, the model is spending weights on nothing.

The map above is mutual information between each pixel and the digit label, laid back onto the grid it came from. The bright band runs down the middle, the edges go dark, the twelve dead pixels are crossed out, and the twenty ringed pixels are the ones that reach 92.9% test accuracy on their own.

The data is sklearn’s bundled UCI Optical Digits set: 1,797 samples, each an 8x8 grayscale image flattened into 64 pixel columns, ten digit classes. Every pixel is an integer from 0 to 16. The columns are named by position, so pixel_4_2 is row 4, column 2, which makes it easy to put a number back onto the grid it came from. I split 1,347 train and 450 test, stratified, seed fixed at 42, and computed everything that touches the label on train only. All accuracies below are test set.

The edges are constant, and you can prove it pixel by pixel

Variance per pixel is the cheapest diagnostic there is, and it tells you where the signal is before any classifier runs. Across the 64 pixels it runs from 0.0 to 42.40, so a cutoff of 0.5 or 1.0 is a pixel that barely moves at all. I reshaped the 64 variances back into an 8x8 grid:

The left panel is variance. It is dark down the left and right columns and bright through a vertical band in the middle. The right panel is mutual information between each pixel and the digit label, and it follows the same pattern: dark at the edges, bright through the middle. A dead pixel carries no information because it never moves.

The dead pixels all sit in the outermost columns, column 0 on the left and column 7 on the right. Widen the threshold to variance under 1.0 and you reach 17 dead pixels, more than a quarter of the image. A pen drawing a digit into a centered 8x8 box rarely reaches the outer columns, so those pixels sit at zero ink and stay there. Mutual information runs from exactly 0.0 for the deadest pixel to 0.4306 for the best.

How few pixels do you actually need

If the information is concentrated, a handful of pixels should recover most of the accuracy. I ranked all 64 by mutual information and trained the same logistic regression on just the top k, standardizing on train statistics each time. Walking k up:

• 1 pixel: 22.9%
• 2 pixels: 40.2%
• 5 pixels: 61.1%
• 10 pixels: 82.4%
• 15 pixels: 89.6%
• 20 pixels: 92.9%
• 30 pixels: 94.9%
• 40 pixels: 95.6%
• all 64 pixels: 97.8%

Twenty pixels, fewer than a third of the image, get you to 92.9%, which is 95% of the full-pixel accuracy. The remaining 44 pixels, more than two-thirds of the data, buy you the last 4.9 points, and going from 30 to 40 pixels buys less than one. The curve is steep early and flat late, exactly what you would predict from a variance map that is bright in the middle and dark at the edges. The single most informative pixel alone, with nothing else, classifies about one image in four correctly against a 10% random baseline.

One caution on the ranking itself. Mutual information scores each pixel on its own, so two neighboring pixels that see the same stroke can both rank high while adding little together. A ranking that accounts for overlap could likely do the same job with fewer pixels; I did not test that.

I will flag the honest part of that top-1 number. 22.9% from a single intensity value sounds almost too good, but this is a ten-class problem where one pixel can rule out half the digits, which makes it a coarse filter, not a classifier.

The most informative pixel sits left of center, and it splits left-heavy digits from right-heavy ones

The winner is pixel_4_2, row 4, column 2, the left-middle of the grid, mutual information 0.4306. It carries the brightest ring on the flagship. To see what it is doing I pulled the mean intensity of that pixel per digit on the train set, and the split is clean at the ends. It is bright for 6 (mean 14.6 out of 16), 4 (14.4), and 0 (11.3), the digits with a heavy left stroke or a closed left loop. It is nearly dark for 2 (1.2), 3 (1.2), and 9 (3.0), digits that leave the left-middle empty because their ink lives on the right or the top. The middle of the range is fuzzier: 8 (6.9), 1 (7.2), 5 (7.7), and 7 (8.6) all sit close to the pixel’s overall mean of 7.6, so this one pixel cannot tell them apart. One pixel splits the ten classes roughly in half, left-heavy from right-heavy.

That single split is the seed of the whole accuracy curve. Stack a dozen more pixels that each carve the space along a different stroke and you have the structure of a digit without ever touching the edges.

Resolution sets the limit

This is not MNIST. These are 8x8 thumbnails, 64 pixels, and the reason the edges are dead is partly that the resolution is so low the ink has nowhere to spread. At 28x28 I would expect both the dead-pixel map and the most informative locations to change, though I have not measured it here. So do not read “a quarter of the pixels are useless” as a property of handwritten digits in general. Read it as: at this resolution, with the digit centered, the signal collapses onto a vertical band and the outer columns are free to throw away.

What I would take from it for any low-res glyph problem: compute per-pixel variance first, before you reach for a model. It costs nothing and it tells you how much of your input is constant. Here it told me a fifth of the image was dead weight, and 20 pixels picked by mutual information recovered 95% of the full-pixel accuracy. The edge columns never had a vote.