Author Topic: Website server logs / wordpress etc  (Read 13002 times)

0 Members and 1 Guest are viewing this topic.

Offline
*
Website server logs / wordpress etc
« on: January 13, 2016, 07:56:12 PM »
Hello,

I have a Wordpress site running on CWP. I use the WP stats plugin for statistics. Unfortunately all visits
are coming from the ip of the server..  :( I understand that CWP does not collect access / error logs for
sites running on the server but i am not sure if that is the reason for not seeing the real visitors ip.

Are there any plans / roadmap to implement  access_logs / error logs for the sites running on the CWP like other Control panels
like Plesk / Cpanel / Directadmin have?

I understand that implementing this would possibly put a bit higher load on server but would be a good next step. ..


« Last Edit: January 13, 2016, 08:42:35 PM by devnull »

Offline
*
Re: Website server logs / wordpress etc
« Reply #1 on: January 14, 2016, 04:29:47 AM »
By default CWP's apache web server logs access and errors to /usr/local/apache/logs/
If you're running varnish (or some other caching server) in front of apache then out of the box it will look like all the hits are coming from the server's IP unless you set up:
remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For = client.ip

sub vcl_recv section of default.vcl (for varnish anyways).

You can change what apache logs and where it writes them in  /usr/local/apache/conf/httpd.conf and you can also setup individual domain logging in the vhost configuration.

Offline
*
Re: Website server logs / wordpress etc
« Reply #2 on: January 14, 2016, 07:44:27 PM »

Thanks for your reply.

I use Varnish with my CWP version: 0.9.8.10

The content of my default.vcl is now

backend default { .host = "xxx.xxx.xxx.xxx"; .port = "8181";}
include "/etc/varnish/backends.vcl";
sub vcl_recv { include "/etc/varnish/sites.vcl";}

What should I exactly add in this file to see the real ips?


Thanks in advance!

Offline
*
Re: Website server logs / wordpress etc
« Reply #3 on: January 15, 2016, 01:38:59 AM »
Here's my current default.vcl :
Code: [Select]
backend default { .host = "X.X.X.X"; .port = "8181";}
include "/etc/varnish/backends.vcl";
#set IP for apache logging
sub vcl_recv { include "/etc/varnish/sites.vcl";
remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For = client.ip;
# Setup grace mode.
  # Allow Varnish to serve up stale (kept around) content if the backend is
  #responding slowly or is down.
  # We accept serving 6h old object (plus its ttl)
  if (! req.backend.healthy) {
   set req.grace = 6h;
  } else {
   set req.grace = 15s;
  }
 
  # If our backend is down, unset all cookies and serve pages from cache.
  if (!req.backend.healthy) {
    unset req.http.Cookie;
  }

# Drop any cookies sent to Wordpress.
if(
        req.url ~ "^/administrator" ||
        req.url ~ "^/component/banners" ||
        req.url ~ "^/component/users" ||
        req.url ~ "^/wp-admin" ||
        req.url ~ "^/wp-login.php" ||
        req.url ~ "^/any-other-url-path"
        ) {
return (pass);
    } else if (
                req.url ~ "^/roundcube"
        ) {
return (pipe);
    } else {
unset req.http.cookie;
}
# As mentioned before, remove all cookies for static files, images etc
  # Varnish will always cache the following file types and serve them (during TTL).
  # Note that Drupal .htaccess sets max-age=1209600 (2 weeks) for static files.
  if (req.url ~ "(?i)\.(bmp|png|gif|jpeg|jpg|doc|pdf|txt|ico|swf|css|js|html|htm)(\?[a-z0-9]+)?$") {
    // Remove the query string from static files
    set req.url = regsub(req.url, "\?.*$", "");
 
    unset req.http.Cookie;
 
    # Remove extra headers
    # We remove Vary and user-agent headers that any backend app may set
    # If we don't do this, Varnish will cache a separate copy of the resource
    # for every different user-agent
    unset req.http.User-Agent;
    unset req.http.Vary;
 
    return (lookup);
  }


}
#####
#If something gets super popular, super cache it
sub vcl_hit {
        if (obj.hits == 500) {
                set obj.ttl = 3h;
        } elsif (obj.hits == 10000) {
                set obj.ttl = 2d;
        } elsif (obj.hits == 1000000) {
                set obj.ttl = 4w;
        }
}
#####
#shutdown backend connections so unprivileged users don’t get privileged content
sub vcl_pass { 
    set bereq.http.connection = "close";
    if (req.http.X-Forwarded-For) {
        set bereq.http.X-Forwarded-For = req.http.X-Forwarded-For;
    }
    else {
        set bereq.http.X-Forwarded-For = regsub(client.ip, ":.*", "");
    }
}
#####
#shutdown backend connections so unprivileged users don’t get privileged content
sub vcl_pipe { 
    set bereq.http.connection = "close";
    if (req.http.X-Forwarded-For) {
        set bereq.http.X-Forwarded-For = req.http.X-Forwarded-For;
    }
    else {
        set bereq.http.X-Forwarded-For = regsub(client.ip, ":.*", "");
    }
}

#####
sub vcl_fetch {
# Don't allow static files to set cookies. Cache static content for a long time
  if (req.url ~ "(?i)\.(bmp|png|gif|jpeg|jpg|doc|pdf|txt|ico|swf|css|js|html|htm)(\?[a-z0-9]+)?$") {
    unset beresp.http.set-cookie;
    # default in Drupal, you may comment out to apply for other cms as well
    set beresp.ttl = 2w;
  }
#Cache stuff you shouldn’t for a min, just bout everything else an hour
if (beresp.ttl < 1h) {
            if (beresp.http.Cache-Control ~ "(private|no-cache|no-store)") {
                set beresp.ttl = 60s;
            }
            else {
                set beresp.ttl = 1h;
}
}
 if (beresp.status == 301) {
    set beresp.ttl = 1h;
    return(deliver);
  }
  # Allow items to be stale if backend goes down. This means we keep around all objects for 6 hours beyond their TTL which is 2 minutes
  # So after 6h + 2 minutes each object is definitely removed from cache
  set beresp.grace = 6h;
 
  # If you need to explicitly set default TTL, do it below.
  # Otherwise, Varnish will set the default TTL by looking-up
  # the Cache-Control headers returned by the backend
  # set beresp.ttl = 6h;

  # if you have misbehaving sites (i.e Drupal6 or cookie-setters)
  # and you have forced Varnish to cache them in vcl_recv,
  # here you can instruct Varnish about their ttl, and
  # force Varnish to strip any cookies send from backend
  #if (req.http.host ~ "(?i)^(www.)?yourURL.com") {
  # unset beresp.http.set-cookie;
  # set beresp.http.Cache-Control = "public,max-age=602";
  # set beresp.ttl = 120s;
  #}

}

There's a lot here but the part you're most interest in is at the top. Varnish out of the box doesn't really do much, you really need to tune it to your needs. This works with wordpress, phpmyAdmin, roundcube, and joomla and gives me approx 50% hit rate. Down at the bottom you may notice that I set the cache time for php files to 1hr, that means it may take that long to see changes to to see a post appear on your main page though the post page is perfectly visible. It's a trade off, less time = fewer cache hits.  While on the varnish topic, if you have the RAM I highly recommend switching the path in /etc/sysconfig/varnish where it says DAEMON_OPTS where there's a path for where to keep cache files, switch it to: -s malloc,1G" This tells varnish to save cached content in memory and serve it from there. You can change the size from 1G to 256M, 4G, 1247M, etc whatever you need/can spare. Any changes to either of these files only take effect when you restart the service (service varnish restart).

In /usr/local/conf/httpd.conf
change your logging format to:
LogFormat "%{X-Forwarded-For}i %l %u %t %v \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
and make sure it says "combined" after your logging path:
CustomLog "/usr/local/apache/logs/access_log" combined
restart apache (service http restart) and viola! Logging with IPs through varnish. Still stuck on the error log that all looks like it's coming from my server.

Oh and as always make sure you back up your config files before modifying them!
« Last Edit: January 15, 2016, 01:42:12 AM by Darkroom »

Offline
*
Re: Website server logs / wordpress etc
« Reply #4 on: January 16, 2016, 06:07:23 PM »


Thanks for the thorough explanation. I tried it but unfortunately it did not work out so I created a one time support ticket at CWP.
It is still open but I think they changed something anyway Wordfence is now showing the real ips after changing the setting
How does Wordfence get IPs: to Use the Xforwarded for http header.

My other plugin WP statistics is still showing the server ips ....and i do not see an option in WP statistics to do anything
about getting the ips ... yet.