When we load PHP into Apache as a module (using mod_php), each Apache process we run will also contain a PHP interpreter which in turn will load all the compiled in libraries which themselves are not exactly small.
This means that even if the Apache process that just started will only serve images, it will contain a PHP interpreter with all assigned libraries. That in turn means that said Apache process uses a lot of memory and takes some time to start up (because PHP and all the shared libraries it's linked to need to be loaded). Wasted energy if the file that needs to be served in an image or a CSS file.
FastCGI in contrast loads the PHP interpreter into memory, keeps it there and Apache will only use these processes to serve the PHP requests.
That means that all the images and CSS, flashes and whatever other static content we may have can be served by a much smaller Apache process that does not contain a scripting language interpreter and that does not link in a bunch of extra libraries (think libxml, libmysqlclient, and so on).
Even if we only serve pages parsed by PHP - maybe because we process our stylesheets with PHP and because we do something with the served images - we are theoretically still better off with FastCGI as Apache will recycle its processes here and then (though that's configurable) while FastCGI processes stay there.
And if we go on and need to load-balance your application, FastCGI still can provide advantages: In the common load balancing scenario, we have a reverse proxy or a load balancer and a bunch of backend servers actually doing the work. In that case, if we use FastCGI, the backend servers will be running our PHP application and noting else. No web server loading an interpreter loading our script. Just the interpreter and our script. So we safe a whole lot of memory by not loading another web server in the backend (Yes. FastCGI works over the network).