Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - Darkroom

Pages: 1 ... 4 5 [6]
76
Varnish / Re: default.vcl discussion
« on: January 02, 2016, 03:42:48 PM »
Everything is working well, just when I started I had a huge miss rate and almost no improvement on server speed. This is what I did to improve that. I'm sharing a solution to the "one click thingy" not really helping much. It works and installs varnish but with a minimal config that passes most things and makes your apache logs useless. Then when you start doing things to cache more like stripping cookies and user agents and setting longer cache times it can break things, especially logins. The above config tries to balance caching with usability (esp not interfering with joomla and WordPress logins/administration). I'm  new to varnish and while I think I understand what's going on I wouldn't mind a sanity check on that. In the process, if we can help people who do the "one click thingy" only to be disappointed by providing a fairly straight forward cut and paste solution that has been vetted by the CWP community, all the more better! I spent some time assembling this config and I think it might help others. This config can also probably be improved and suggestions to that effect are welcomed.

In a more perfect world there could be a step in the varnish install of CWP that has check boxes with options and notes like "Strip user Agent (increases hit rate)", "Pass WordPress login/Admin traffic (recommended if running wordpress)", "Cache from Memory or Disk", "Cache Static Content longer" etc. A varnish config generator like the mod_sec only a bit more interactive would be damn skippy!  :D

77
Varnish / Re: default.vcl discussion
« on: January 01, 2016, 11:17:27 PM »
That's how it is setup. The backend should be your apache server. This is how I configured varnish to do things cache static content, pass logins, and stuff like that.

78
Varnish / default.vcl discussion
« on: December 31, 2015, 06:29:10 PM »
Like lot's of folks I installed varnish and saw no speed increase on my site. I switched from disk cache to malloc and still, little improvement with a huge miss rate. I started looking into vanish settings and here's what I cam up with. A little background: I'm running wordpress, joomla, and static web pages. I didn't really write this, I just took the most useful stuff from a number of sources (some of their comments I left intact, some are my own). Right now my hit rate is over 50%, the sites behave like they should (including admin access), apache logging works, everything seems good! Feedback is appreciated. If you're doing something different that's working awesome I'd love to check it out.

Code: [Select]
backend default { .host = "X.X.X.X"; .port = "8181";}
include "/etc/varnish/backends.vcl";
#set IP for apache logging
sub vcl_recv { include "/etc/varnish/sites.vcl";
remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For = client.ip;
# Setup grace mode.
  # Allow Varnish to serve up stale (kept around) content if the backend is
  #responding slowly or is down.
  # We accept serving 6h old object (plus its ttl)
  if (! req.backend.healthy) {
   set req.grace = 6h;
  } else {
   set req.grace = 15s;
  }
 
  # If our backend is down, unset all cookies and serve pages from cache.
  if (!req.backend.healthy) {
    unset req.http.Cookie;
  }

# Drop any cookies sent to Wordpress.
if(
        req.url ~ "^/administrator" ||
        req.url ~ "^/component/banners" ||
        req.url ~ "^/component/users" ||
        req.url ~ "^/wp-admin" ||
        req.url ~ "^/wp-login.php" ||
        req.url ~ "^/any-other-url-path"
    ) {
        return (pass);
    } else {
unset req.http.cookie;
}
# As mentioned before, remove all cookies for static files, images etc
  # Varnish will always cache the following file types and serve them (during TTL).
  # Note that Drupal .htaccess sets max-age=1209600 (2 weeks) for static files.
  if (req.url ~ "(?i)\.(bmp|png|gif|jpeg|jpg|doc|pdf|txt|ico|swf|css|js|html|htm)(\?[a-z0-9]+)?$") {
    // Remove the query string from static files
    set req.url = regsub(req.url, "\?.*$", "");
 
    unset req.http.Cookie;
 
    # Remove extra headers
    # We remove Vary and user-agent headers that any backend app may set
    # If we don't do this, Varnish will cache a separate copy of the resource
    # for every different user-agent
    unset req.http.User-Agent;
    unset req.http.Vary;
 
    return (lookup);
  }


}
#####
#If something gets super popular, super cache it
sub vcl_hit {
        if (obj.hits == 500) {
                set obj.ttl = 3h;
        } elsif (obj.hits == 10000) {
                set obj.ttl = 2d;
        } elsif (obj.hits == 1000000) {
                set obj.ttl = 4w;
        }
}
#####
#shutdown backend connections so unprivileged users don’t get privileged content
sub vcl_pass { 
    set bereq.http.connection = "close";
    if (req.http.X-Forwarded-For) {
        set bereq.http.X-Forwarded-For = req.http.X-Forwarded-For;
    }
    else {
        set bereq.http.X-Forwarded-For = regsub(client.ip, ":.*", "");
    }
}
#####
#shutdown backend connections so unprivileged users don’t get privileged content
sub vcl_pipe { 
    set bereq.http.connection = "close";
    if (req.http.X-Forwarded-For) {
        set bereq.http.X-Forwarded-For = req.http.X-Forwarded-For;
    }
    else {
        set bereq.http.X-Forwarded-For = regsub(client.ip, ":.*", "");
    }
}

#####
sub vcl_fetch {
# Don't allow static files to set cookies. Cache static content for a long time
  if (req.url ~ "(?i)\.(bmp|png|gif|jpeg|jpg|doc|pdf|txt|ico|swf|css|js|html|htm)(\?[a-z0-9]+)?$") {
    unset beresp.http.set-cookie;
    # default in Drupal, you may comment out to apply for other cms as well
    set beresp.ttl = 2w;
  }
#Cache stuff you shouldn’t for a min, just bout everything else a day
if (beresp.ttl < 24h) {
            if (beresp.http.Cache-Control ~ "(private|no-cache|no-store)") {
                set beresp.ttl = 60s;
            }
            else {
                set beresp.ttl = 24h;
}
}
 if (beresp.status == 301) {
    set beresp.ttl = 1h;
    return(deliver);
  }
  # Allow items to be stale if backend goes down. This means we keep around all objects for 6 hours beyond their TTL which is 2 minutes
  # So after 6h + 2 minutes each object is definitely removed from cache
  set beresp.grace = 6h;
 
  # If you need to explicitly set default TTL, do it below.
  # Otherwise, Varnish will set the default TTL by looking-up
  # the Cache-Control headers returned by the backend
  # set beresp.ttl = 6h;

  # if you have misbehaving sites (i.e Drupal6 or cookie-setters)
  # and you have forced Varnish to cache them in vcl_recv,
  # here you can instruct Varnish about their ttl, and
  # force Varnish to strip any cookies send from backend
  #if (req.http.host ~ "(?i)^(www.)?yourURL.com") {
  # unset beresp.http.set-cookie;
  # set beresp.http.Cache-Control = "public,max-age=602";
  # set beresp.ttl = 120s;
  #}

}

79
Mod_Security / Re: Enable Mod_Security for Wordpress
« on: December 22, 2015, 03:25:01 AM »
Security -> mod_security -> Disabled rules link. Replace contents with these (should work with Jetpack, Google Sitemap, Google Adsense, and W3 total cache plugins):

## Rules for the CWP ##
SecRuleRemoveById 910006
SecRuleRemoveById 950000
SecRuleRemoveById 950001
SecRuleRemoveById 950005
SecRuleRemoveById 950006
SecRuleRemoveById 950117
SecRuleRemoveById 950907
SecRuleRemoveById 958039
SecRuleRemoveById 958051
SecRuleRemoveById 958291
SecRuleRemoveById 959006
SecRuleRemoveById 959151
SecRuleRemoveById 960008
SecRuleRemoveById 960010
SecRuleRemoveById 960011
SecRuleRemoveById 960012
SecRuleRemoveById 960035
SecRuleRemoveById 960335
SecRuleRemoveById 960904
SecRuleRemoveById 960915
SecRuleRemoveById 970003
SecRuleRemoveById 970015
SecRuleRemoveById 970903
SecRuleRemoveById 973301
SecRuleRemoveById 973302
SecRuleRemoveById 973306
SecRuleRemoveById 973316
SecRuleRemoveById 973330
SecRuleRemoveById 973331
SecRuleRemoveById 973332
SecRuleRemoveById 973334
SecRuleRemoveById 973335
SecRuleRemoveById 973336
SecRuleRemoveById 973344
SecRuleRemoveById 973347
SecRuleRemoveById 981172
SecRuleRemoveById 981240
SecRuleRemoveById 981241
SecRuleRemoveById 981244
SecRuleRemoveById 981248
SecRuleRemoveById 981249
SecRuleRemoveById 981255
SecRuleRemoveById 981256
SecRuleRemoveById 981260
SecRuleRemoveById 981317
SecRuleRemoveById 981318
SecRuleRemoveById 981319
SecRuleRemoveById phpids-17
SecRuleRemoveById phpids-20
SecRuleRemoveById phpids-21
SecRuleRemoveById phpids-30
SecRuleRemoveById phpids-61
SecRuleRemoveById phpids-17
SecRuleRemoveById phpids-20
SecRuleRemoveById phpids-21
SecRuleRemoveById phpids-30
SecRuleRemoveById phpids-61
## Rules for the CWP ##
SecRuleRemoveById 960017
SecRuleRemoveById 960015
SecRuleRemoveById 960009
########################################
## Removed Rules for Joomla, WordPress and Drupal CMSs ## ########################################
## Joomla ##
SecRuleRemoveById 950120
SecRuleRemoveById 950901
SecRuleRemoveById 960024
SecRuleRemoveById 973300
SecRuleRemoveById 973304
SecRuleRemoveById 973333
SecRuleRemoveById 973338
SecRuleRemoveById 981173
SecRuleRemoveById 981245
SecRuleRemoveById 981257
## Wordpress ##
SecRuleRemoveById 950007
SecRuleRemoveById 950010
SecRuleRemoveById 950911
SecRuleRemoveById 958005
SecRuleRemoveById 958006
SecRuleRemoveById 958030
SecRuleRemoveById 958049
SecRuleRemoveById 958056
SecRuleRemoveById 958057
SecRuleRemoveById 959070
SecRuleRemoveById 959073
SecRuleRemoveById 960020
SecRuleRemoveById 973308
SecRuleRemoveById 973309
SecRuleRemoveById 973314
SecRuleRemoveById 973327
SecRuleRemoveById 959071
SecRuleRemoveById 959072
SecRuleRemoveById 981004
SecRuleRemoveById 981242
SecRuleRemoveById 981243
SecRuleRemoveById 981246
SecRuleRemoveById 981320
## Drupal ##
SecRuleRemoveById 981231
## Removed rules for the webftp_simple ##
SecRuleRemoveById 950109
SecRuleRemoveById 950922
SecRuleRemoveById 981000
## phpMyAdmin ##
SecRuleRemoveById 981205
SecRuleRemoveById 970901

80
I can build it / Cheesy varish stats page for CWP
« on: December 22, 2015, 02:57:34 AM »
Now that I tuned varnish a bit it's recommended to keep an eye on some stats. I made this little page so I could see the output of a few commands from the web panel without having to login.

<?php
if ( !isset( $include_path ) )
{
    echo "invalid access";
    exit( );
}

$vstat = shell_exec("varnishstat -1 | head -n 6");
$vhits = shell_exec("varnishtop -1 -i rxurl | head -n 10");
$vmiss = shell_exec("varnishtop -1 -i txurl | head -n 10");
$vnuke = shell_exec("varnishstat -1 | grep nuke ");
$vmem = shell_exec("ps aux | grep 'varnish' | awk '{print $6/1024 \" MB\";}'");
echo "<h3>Varnish Statistics</h3><br><br>";
echo "<h3>Varnishstat</h3><br><pre>".$vstat."</pre><br><br>";
echo "<h3>Top 10 Hits</h3><br><pre>".$vhits."</pre><br><br>";
echo "<h3>Top 10 Misses</h3><br><pre>".$vmiss."</pre><br><br>";
echo "<h3>Varnish Nukes</h3><br><pre>".$vnuke."</pre><br><br>";
echo "<h3>Varnish Memory</h3><br><pre>".$vmem."</pre><br><br>";

?>

81
Mod_Security / Disable Rules by Vhost?
« on: December 02, 2015, 04:55:22 PM »
Is it possible to ignore rules (SecRuleRemoveById 654321) by vhost instead of globally? I'm running a shared server with clients running wordpress, joomla, and static websites so removing rules for one host is unnecessary for others. What would the syntax for that look like? Any ideas?

82
Installation / Re: Fresh install of CWP on Centos 6.7 cannot access web server on 2030 or 2031
« on: December 02, 2015, 03:57:04 AM »
Reinstalling doesn't matter if you have a NAT'd IP. The install scripts somehow find your actual IP and put that in the /usr/local/cwpsrv/conf.d/cwp-ssl.conf file. I never told the install what my public IP was and it was still somehow set in that file. Sneaky.  ::)

83
Installation / Re: Fresh install of CWP on Centos 6.7 cannot access web server on 2030 or 2031
« on: November 29, 2015, 07:13:41 AM »
Quote from: GuardMoony on November 27, 2015, 11:11:55 AM
I've also done a fresh install with Centos 6.7  ( actually my 3rd attempt ). I followed the instruction like provided on the website.

These are the problems i found:

  • port 2031 reply's as http not https
  • PHP doesn't seem to install correctly. Looks like the build fails and then i just gets a other version with yum when installing extra tools?
    Warning: file_get_contents(http://xxx.xxx.xxx.xxx/phpinfo.php): failed to open stream: HTTP request failed! HTTP/1.0 404 Not Found in /usr/local/cwpsrv/htdocs/resources/admin/modules/php_info.php(1) : eval()'d code(1) : eval()'d code on line 3
  • Installing Softaculous works. But when trying to open it, you just get back to the login page
  • Adding a user, The dns given to the users doesn't show up. If trying to add it again it says already in use

The http on port 2031happened to me as well. It was from using a 192.168.x.x address and network address translation. CWP doesn't support RFC1918 address and this was one place where it bit me. Check the file /usr/local/cwpsrv/conf.d/cwp-ssl.conf and make sure the cwp ssl server is listening on the right IP (change it if not). Also I changed my IP and shared IP and this file was not updated to reflect those changes. Once I was on a public IP with the correct configuration it worked fine. Others report it will work with NAT if you put the RFC1918 address it has in the config file.

The Softaculous is a known bug, should be fixed in the next version of Softaculous. In the mean time there is a workaround:
http://forum.centos-webpanel.com/softaculous/unable-to-access-softaculous-after-installation-via-cwp/

84
Installation / Re: Fresh install of CWP on Centos 6.7 cannot access web server on 2030 or 2031
« on: November 19, 2015, 05:13:44 PM »
did you try the command:

service cwpsrv start

?
try connecting with http on 2030 first.

85
DNS Manager / Re: FreeDNS not accepting changes
« on: November 18, 2015, 08:31:25 PM »
Took  me a second to figure out, but this is a problem on the freedns.centos-webpanel.com server/cluster.

86
DNS Manager / Re: FreeDNS not accepting changes
« on: November 17, 2015, 09:00:29 PM »
I have a similar problem. I get the same error "/usr/sbin/named-checkconf exit status 1" but it seems to save setting but not write them to the HD. Maybe it's a bug in CWP 0.9.8.10?

Pages: 1 ... 4 5 [6]