Here's an article about Daniel, an autistic savant who can perform calculations in the blink of an eye. Apparently, Daniel "sees" numbers as shapes, colors and textures. To perform a calculation, he imagines the shapes of the numbers to be crunched and then almost immediately sees the shape of the answer. There is no intermediate thought or calculation in the usual sense of the word. Although the article doesn't discuss an explanation in much detail, it sounds to me that Daniel makes novel use of the brain's visual transformation capabilities to perform mathematics.
A visual cortex contains special circuitry that specializes in parallel processing of two-dimensional visual data. It literally performs millions of calculations every second to convert the raw input from our eyes into the stereoscopic images that we experience. A rough equivalent for the visual cortex can be found in everyday computers, which include a graphics chip containing special circuitry for performing massively parallel mathematical operations on 2D and 3D images in order to rotate, shade, and otherwise manipulate them.
A computing whiz could figure out a way to use a graphics chip to perform high-speed matrix calculations by mapping the matrices to an image, telling the chip to rotate the image, and then mapping the resulting image back to the answer. I bet the same thing is happening in Daniel's brain.
This brings up several interesting questions. For example, might there be a way for everyday folks to perform a similar kind of mapping, harnessing specialized parts of the brain for uses that they were not intended for? In the future, I have no doubt that we will finally figure out how the brain works and then start designing enhanced brains that have this kind of functionality built in. Awesome!