In the last days, I have been working on the final optimizations of a WCM site that will be released soon.
It's never late to remember two of the most important optimizations, in my opinion:
1. Assure that all images, CSS's, JS's and other static content have their cache definitions correct. The client browser should receive the information of the expire date correctly (and set to something like 365 days).
How to check this? Use Fiddler or Firebug (Net tab). If you refresh a page and still get requests sent to the server regarding static content (that receive 304 Not Modified replies) then you have a problem.
The more static content your page loads, the bigger the problem is, since your browser will check changes with the server for every file (therefore, 1 useless request per file).
This is probably the most important checkup you must do, since by default, for example, SharePoint will NOT cache the files in the Styles Library (known bug). Regarding this, I suggest you read a post by Chris O'Brien here.
2. Avoid loading SharePoint files in a fully customized frontend anonymous site
Anonymous users usually don't need the core.js/core.css/etc. These files are heavy and should be avoided