Using simultaneous, layer-specific population recordings from primate primary visual area V1, Xu, Morton, et al. show that the spatial configuration of contextual stimuli differentially modulates inter-laminar information flow by changing its efficacy but not its structure. This non-uniform modulation is mediated by a location-dependent output-layer-targeting signal.