In: PHP
30 May 2013If you have a website that gets millions of page requests per day each starting a session and you wish to get the last ounce of performance from your server and you’re still using file-based sessions then by all means give this a try. The more volume of dynamic content you’re serving, the more you’ll benefit in terms of load average and site-speed.
One of my website gets about 8 million page views per day almost all needing a session to be started. Using memcached-based sessions helped reduce the load at least 6-folds and sped-up the site considerably.
Since I came to know about it rather late in my career I say it’s one of least known but much simple and effective way to solve performance problems when you have lots of traffic. Remember high-traffic is a must to see any considerable effect.
Use the following code in the bootstrap script of your website:
ini_set('session.save_handler', 'memcache'); ini_set('session.save_path', "MEMCACHE_SOCKET_PATH");
You can also set the above variables in the php.ini
file for activating it globally. You must have memcache and memcache PHP extension installed for this to work.
You’ll have to estimate the amount of memory sessions will require and set memcache with a higher value than that. You can check the /tmp
directory for the session files and see how much memory is being taken by sess_
files. If you give less memory session data might clear automatically (FIFO by memcached) and users may get logged out etc. depending on what you’re using sessions for.
You can also use redis in place of memcached if you absolutely don’t want the session data to get cleared in any way (other than by the GC).
In: Experiments
8 Aug 2011Some analgyph images I shot using a regular camera. Two shots had to be taken from two different positions and post-processed to create these. They look quite satisfactory to me!
In: MySQL
20 Apr 2011For a long time I’d been seeing high server load on the server MyWapBlog.com is hosted. I thought the increasing traffic was the cause. MySQL, I knew was the causing the load, but I thought it was all natural for a dynamic site with NO caching.
One day, just when I was seriously thinking about implementing some kind of cache, I found about mtop
– small tool like top
to display running MySQL queries in real-time, running it I found that every second there were many queries (same query but made by different requests) that got stuck in the “PREPARING” state and sometimes took as long as 2 secs. to complete. The evil query was:
SELECT c.post_id FROM category_post_relationship c WHERE (c.cat_id IN (SELECT c2.id AS c2__id FROM categories c2 WHERE (c2.user_id = ?)))
Thinking what it does? Let me make it easier – it fetches “post ids” of all the posts of a user that are categorized.
1. Running the query
184 rows, Query took 0.0812 sec
2. Profiling:
starting 0.000132 checking permissions 0.000011 checking permissions 0.000011 Opening tables 0.000037 System lock 0.000015 Table lock 0.000022 init 0.000065 optimizing 0.000025 statistics 0.000027 preparing 0.000027 executing 0.000011 Sending data 0.000047 optimizing 0.000023 statistics 0.000086 preparing 0.075275 end 0.000015 query end 0.000010 freeing items 0.000033 logging slow query 0.000009 cleaning up 0.000011
3. EXPLAIN:
id select_type table type possible_keys key key_len ref row Extra 1 PRIMARY c index NULL PRIMARY 8 NULL 53898 Using where; Using index 2 DEPENDENT SUBQUERY c2 unique_subquery PRIMARY,user_id PRIMARY 4 func 1 Using where
If you’d ask some novice to do what the query does many of them would use two separate queries and believe me (I’ve done the testing as well) that’ll be much faster!
Googling “subquery optimization” got me this:
MySQL evaluates queries “from outside to inside.” That is, it first obtains the value of the outer expression outer_expr, and then runs the subquery and captures the rows that it produces.
From http://dev.mysql.com/doc/refman/5.1/en/in-subquery-optimization.html
The page also tells us some tips on how to optimize subqueries, going by the suggestions we get the following query:
SELECT c.post_id FROM category_post_relationship c WHERE EXISTS (SELECT 1 FROM categories c2 WHERE c2.user_id = 7639 AND c.cat_id = c2.id)
Still, running it takes similar query times and as such doesn’t help.
Again from the same page:
After the conversion, MySQL can use the pushed-down equality to limit the number of rows that it must examine when evaluating the subquery.
Though I didn’t read the whole page thoroughly (I’m lazy and since I got the job done by some other technique) still I’m sure that this “optimized” query only “optimizes” the subquery while our biggest problem is that the outer table (with about 50000 rows) is getting evaluated.
I felt, this was one query you’d rather not “optimize” and be happy with two separate queries.
But, this is not to say it can’t be optimized, it can be very easily – by NOT using subqueries at all. I used JOINS:
SELECT cc.post_id FROM categories c RIGHT JOIN category_post_relationship cc ON c.id = cc.cat_id<br> WHERE c.user_id = ?
Now the query takes 0.0008 sec compared to 0.0812, that’s a hundred fold improvement!
Some tips:
WHERE
condition on the outer query.
In: PHP
9 Apr 2011Created this data validation class a couple of days back for validating some forms. Thought it might be useful for others. This one’s very basic and light-weight but still fully working with many pre-defined rules.
Before listing the class, let me first show how it’s used:
require 'Validator.class.php'; $validator = new Validator(); // Add rules $validator->addRule('name', array('minlength' => 5, 'maxlength' => 20)); $validator->addRule('age', array('min' => 13, 'max' => 35)); $validator->addRule('url', array('url')); $validator->addRule('about', array('require')); // Data to be validated, would normally come from a form $data = array( 'name' => 'Arvind Gupta', 'age' => '80', 'url' => 'http:www.arvindgupta', ); // Set data to be validated $validator->setData($data); // Check if ($validator->isValid()) { echo '<h1>Data is valid!</h1>'; } else { echo '<h1>Data is not valid!</h1>'; echo '<ol>'; // Get and print errors in a nice manner foreach ($validator->getErrors() as $field => $messages) { if (count($messages) == 1) { echo "<li><strong>$field</strong>: $messages[0]</li>"; } else { // If a field has more than one error echo "<li><strong>$field</strong>:</li>"; echo '<ol>'; foreach ($messages as $message) { echo "<li>$message</li>"; } echo '</ol>'; } } echo '</ol>'; }
Very straightforward!
Validating a form is equally straightforward. The best way would be to have the form elements named like arrays, for example:
<form ...> <input type="text" name="form1[text1]" value="" /> <input type="text" name="form1[text2]" value="" /> </form>
And, use something the following line to provide the form data to the validator at one go:
$validator->setData($_REQUEST['form1']);
That’s it! Here is the class code:
/** * Validator * * Data validation class * * @author Arvind Gupta <contact [ AT ] arvindgupta [ DOT ] co [ DOT ] in> * @copyright Arvind Gupta (c) 2011 * @link http://www.arvindgupta.co.in * @license You're free to do whatever with this as long as this notice * remains intact. */ class Validator { protected $_rules = array(); protected $_data = array(); protected $_messages = array(); protected $_errors = array(); public function __construct() { $this->setDefaultMessages(); } /** * Add a rule * * @param string $field Field name (index of data to be validated) * @param array $rules Array of rule(s) */ public function addRule($field, array $rules) { $this->_rules[$field] = $rules; } /** * Set data to be validated * * @param array $data Data to be validated */ public function setData(array $data) { $this->_data = $data; } /** * Set error message for rule * * @param string $rule Rule name * @param string $message New message */ public function setMessage($rule, $message) { $this->_messages[$rule] = $message; } /** * Validates current data with current rules * * @return boolean */ public function isValid() { $valid = true; foreach ($this->_rules as $field => $rules) { $value = isset($this->_data[$field]) ? $this->_data[$field] : ''; foreach ($rules as $rule => $parameter) { // If rule does not require parameter if (is_int($rule)) { $rule = $parameter; $parameter = null; } if (!$this->check($value, $rule, $parameter)) { $valid = false; if (stripos($this->_messages[$rule], '%s') !== false) { $this->_errors[$field][] = sprintf($this->_messages[$rule], $parameter); } else { $this->_errors[$field][] = $this->_messages[$rule]; } } } } return $valid; } /** * Get error messages if validation fails * * @return array Error messages */ public function getErrors() { return $this->_errors; } protected function check($value, $rule, $parameter) { switch ($rule) { case 'require' : return!(trim($value) == ''); case 'maxlength' : return (strlen($value) <= $parameter); case 'minlength' : return (strlen($value) >= $parameter); case 'numeric' : return is_numeric($value); case 'int' : return is_int($value); case 'min' : return $value > $parameter ? true : false; case 'max' : return $value < $parameter ? true : false; case 'url': // Regex taken from symfony return preg_match('~^ (https?):// # protocol ( ([a-z0-9-]+\.)+[a-z]{2,6} # a domain name | # or \d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3} # a IP address ) (:[0-9]+)? # a port (optional) (/?|/\S+) # a /, nothing or a / with something $~ix', $value); case 'email': return preg_match('/^([^@\s]+)@((?:[-a-z0-9]+\.)+[a-z]{2,})$/i', $value); case 'regex': return preg_match($parameter, $value); case 'pass': return true; default : return false; } } protected function setDefaultMessages() { $this->_messages = array( 'require' => 'Field is required.', 'maxlength' => 'Too long (%s characters max).', 'minlength' => 'Too short (%s characters min).', 'numeric' => 'Value must be numeric.', 'int' => 'Value must be an integer.', 'max' => 'Value must be at most %s', 'min' => 'Value must be at least %s', 'url' => 'Value must be a valid URL.', 'email' => 'Value must be a valid email.', 'regex' => 'Invalid value.', ); } }