I had an intuition about this, a question I kept turning over: that there’s a shared geometry to how brains represent things, and that it probably differs a bit person to person depending on how environment, and maybe genetics, shaped us. This is my first attempt to explore the question with real open data rather than just turning it over in my head. It’s also a small, early-stage project: four MEG participants, one EEG replication, and work done in my own time, without the resources or infrastructure of an established lab..
The specific question I landed on: does the alignment between people’s brains get more complicated over time, as a picture turns into a concept. The idea being that early on, when the brain is just doing edges and colors, a simple match between two people might hold up fine, but by the time it’s turned into “that’s a dog,” you’d need something a lot more flexible to line people up. It’s a reasonable hypothesis, and I expected to find it.
I didn’t. Using the THINGS-MEG dataset (4 people, 1,854 object images) and a much bigger EEG replication (48 people) to make sure I wasn’t fooling myself with a tiny sample, the gain from an unconstrained model over a simple rigid rotation stayed close to zero the whole way through, in both datasets. In the EEG data specifically, I ran a cluster permutation test because I didn’t trust a null result from a single quick look, and it held (p < 0.001). A map trained on brain activity at 20 milliseconds after seeing an image works just as well at 400 milliseconds, deep into “what is this thing” territory. That’s not what I went in expecting, and it took me a while to actually believe the result instead of assuming I’d made a mistake somewhere.
I checked the obvious alternative explanations. Maybe the signal was just too weak to show a gradient either way, so I checked it against the noise ceiling: the EEG alignment reaches about 81.5% of it, which rules out “there’s just not enough signal here.” Maybe it was an artifact of how many dimensions I reduced the data to before aligning, so I redid it across PCA dimensionalities from 5 to 63 components, same flat result every time.
The part that actually surprised me came from a side comparison I almost didn’t run. I compared several image recognition models against the brain data to see which ones predicted neural responses best. A ViT trained on plain ImageNet classification barely beat a version of the same architecture with completely random, untrained weights, r of about 0.008 versus 0.009, basically indistinguishable. CLIP, trained on image-text pairs instead of just labels, did noticeably better at 0.043. Same rough architecture family, and the only real difference is that CLIP had to learn to connect vision to language along the way. ResNet-50, an older and simpler architecture, also clearly beat the supervised ViT despite similar ImageNet training, which wasn’t something I predicted going in either. If you’d asked me beforehand whether architecture or training objective mattered more for matching brain activity, I would not have confidently picked training objective.
There’s also a spatial piece: source-localizing the CLIP alignment points to temporal pole and entorhinal cortex around 495 milliseconds, which fits the general story of the ventral visual stream moving from simple shapes toward semantic meaning, though I’d hold that part more loosely given the usual caveats of template-based source localization.
I don’t think any of this is the final word on cross-subject alignment, four MEG subjects is a small n, and I’d love for someone with a bigger dataset to try to break this result. But I went in with one specific intuition, tested it as carefully as I could, and came out with a flat line instead of what I expected. That in itself felt worth writing down.
If you want to check my work or push on it yourself: Temporal Alignment in MEG. I’d genuinely welcome anyone finding a flaw in this.