HEADERS
Try and see what other web servers reply.
loader

This is what your client sends to our servers.

ACCEPT-ENCODING: gzip
CONNECTION: Keep-Alive
HOST: serverstuff.info
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
USER-AGENT: CCBot/2.0 (https://commoncrawl.org/faq/)

This is what your client sends to the server when requesting this site. You can get this by using the following code function in php. It's not available in php-fpm though.

$headers = apache_request_headers(); foreach ($headers as $header => $value) { echo "$header: $value <br />\n"; }

For php-fpm you could use this...


function apache_request_headers() {
  $arh = array();
  $rx_http = "/\AHTTP_/";
  foreach($_SERVER as $key => $val) {
    if( preg_match($rx_http, $key) ) {
      $arh_key = preg_replace($rx_http, "", $key);
      $rx_matches = array();
      $rx_matches = explode("_", $arh_key);
      if( count($rx_matches) > 0 and strlen($arh_key) > 2 ) {
        foreach($rx_matches as $ak_key => $ak_val) $rx_matches[$ak_key] = ucfirst($ak_val);
        $arh_key = implode("-", $rx_matches);
      }
      $arh[$arh_key] = $val;
    }
  }
  return( $arh );
}

You can view this in your chrome's developer tools ( F12 or ctrl+shift+i ) in the network tab.