Synaptic vesicle glycoprotein 2A (SV2A) is a presynaptic protein targeted by the antiseizure drug levetiracetam. One or more of the three SV2 genes is expressed in all neurons and is essential to normal neurotransmission. Loss of SV2A results in a seizure phenotype in mice and mutations in humans are also linked to congenital seizures. How SV2A action impacts the epileptic phenotype remains unclear, especially among the diverse neuronal populations that regulate network excitability. This study explored how brain structure and function are affected by SV2A conditional knock-out (SV2A-cKO) in specific neural cell subtypes. We show that SV2A-cKO in all neurons of the postnatal brain triggers lethal seizures, suggesting that the seizures observed in earlier knock-out models were not due to aberrant brain development. Similar lethal seizures are detected in mice in which the loss of SV2A is limited to GABAergic neurons, whereas loss in excitatory neurons produces no noticeable phenotype. No apparent gender difference was ever observed. Further investigation revealed that SV2A-cKO in different GABAergic interneuron populations induces seizure, with variable timescales and severity. Most notably SV2A-cKO in parvalbumin interneurons (PV+) leads to lethal seizures in young animals, while SV2A-cKO in somatostatin (SST) inhibitory neurons results in seizures that were scarcely observed only in adult mice. These results support the crucial role SV2A plays in PV and SST interneurons and suggest that the action of levetiracetam may be due largely to effects on a subset of GABAergic interneurons.