As David Petrou mentions, C++, Java, and Python are the most prevalent.
C++ generally powers compute-intensive back-end services, Java (and a wealth of libraries including servlets, Guice, and Guava) is often used for application logic, and many scripts are written in Python. Parts of YouTube are written in Python as well as some App Engine applications.
Some less dominant languages include:
- Sawzall, for processing log files.
- Go (programming language), for highly concurrent systems. Nowhere near as popular as C++ but that may very well change in a few years.
- Several small DSLs used for monitoring, builds, and deployment.
Perl, PHP, and Ruby are occasionally used for small scripts and web pages but not so much for production sites. Also see Neil Kandalgaonkar's answer to When will Google permit languages other than Python, C++, Java and Go to be used for internal projects?. He brings up some interesting points like readability reviews and how they affect language choice.