The part of the brain that controls nervousness and anxiety is the amygdala. The amygdala is located on both sides of the brain, within the region of the brain called the temporal lobes, which are roughly located on both sides of the brain, behind the ears going inward. So when you sense fear, danger, or stressors, then the amygdala is activated, which then sets off the fight or flight response, which is responsible for the physical symptoms of nervousness and anxiety.
The following video explains how the amygdala controls nervousness and anxiety:
