When a page is requested from your website, the server reads, parses, and executes potentially hundreds of PHP files to fulfill the request. If multiple requests come in for the same content, this can quickly overload your server.
Caching is the pre-building of parts of your page on the server so they can be delivered faster. However, caching is not a guaranteed performance benefit. Some parts of a website are often dynamic and enabling caching for those parts can cause them to not function as you might expect. For example, caching the search results page will show the search results for the first query when the page is cached, but then the same page for all subsequent queries until the cache is refreshed.
Caching can be enabled at various levels, from individual blocks to full pages. Basic caching can be enabled in the in the Optimization section of the System & Settings part of the Dashboard.