User-agent:* Disallow:/cgi-bin/ Disallow:/flash/ Disallow:/500.php Disallow:/404.php