Language and the Brain
Most of us are familiar with the fact that the left region of brain is activated when a language is used. When a child is not exposed to a language during the critical period for language acquisition, s/he may have experienced some difficult language abilities in later life.
What about the signers? English or any spoken language is a vocal-aural language, whereas ASL or any signed language is a visual-spatial language. Though these modalities are different, languages (signed and spoken) are no different in term of natural human language.
"Language is not mouth stuff -- it's brain stuff." -- Dr. William Stokoe (Gallaudet University).
Let's discover the activities in brain of these languages in different communication modes.
Activities in the left brain
Researchers studied Group #1 of hearing subjects who are prelingually (native) English speakers and who have never learned ASL, and Group #2 of prelingually (native) bilingual hearing subjects who have learned both English and ASL from birth.

Both groups #1 and #2 show activation in the left brain only, when processing English sentences.
On the other hand, when processing ASL sentences, prelingually (native) hearing signers (and also true for prelingually or native Deaf signers) show the same cerebral activity in the left brain as hearing people!
This suggests that the regions of the left brain, which is responsible for language, is not central to sound-speech, as it had been previously believed to be.
"The human brain does not discriminate between the hands and the tongue. People discriminate, but not our biological human brain." -- Dr. Laura-Ann Petitto (Gallaudet Today, Spring 2012, p. 17)
In addition to the left brain, viewing or processing ASL sentences also activated the regions of the right brain.

Interesting, the study shows that the region called "angular gyrus" (AG) in the right brain showed activity found only in prelingually signers, not found in postlingually signers. (Newman)
This result suggests that there is a critical period for language processing in the right hemisphere.
References
Petitto, Dr. Laura Ann. Untitled. Montreal: McGill University media. October 28, 1997.
Newman, A.J., Bavelier, D., Corina, D., Jezzard, P. and Neville, H.J. A critical period for right hemisphere recruitment in American Sign Language processing. Nature Neuroscience, 5:76-80, 2002.
Suggestions for further reading
Emmorey, K., & McCullough, S., The bimodal bilingual brain: Effects of sign language experience, Brain and Language (2008), doi:10.1016/j.bandl.2008.03.005
Karen Emmorey. "Sign Language and the Brain". Language, Cognition, and the Brain: Insights From Sign Language Research. Chapter 9: pp 271-314.