EXEC dbo.sproc_exportObjectToCSV ‘view_Customers’, ‘C:\Data\MSSQL\customers.csv’ GO

Here’s the SQL script:

/****** Object:  StoredProcedure [dbo].[sproc_exportObjectToCSV]    Script Date: 03/02/2012 21:00:39 ******/
SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

sp_configure 'Ad Hoc Distributed Queries', 1;
RECONFIGURE;
GO

CREATE VIEW view_Columns AS SELECT '' AS COLUMN_NAME
GO

CREATE PROCEDURE [dbo].[sproc_exportObjectToCSV]
	@table_name varchar(255),
	@file_name varchar(1024)
AS 
BEGIN
	DECLARE @column_name varchar(255), @sql_column_names varchar(4000), @sql_columns varchar(4000), @sql varchar(4000), @db_name varchar(255)

	SET @db_name = DB_NAME()
	SET @sql_column_names = 'SELECT '
	SET @sql_columns = 'SELECT '
	EXEC('ALTER VIEW view_Columns AS SELECT COLUMN_NAME FROM OPENROWSET (''SQLOLEDB'',''Server=(local);TRUSTED_CONNECTION=YES;'',''set fmtonly off exec ' + @db_name + '.dbo.sp_columns ''''' + @table_name + ''''''') AS tbl')
	DECLARE c1 CURSOR FOR SELECT COLUMN_NAME FROM view_Columns
	OPEN c1

	FETCH NEXT FROM c1 INTO @column_name
	WHILE @@FETCH_STATUS = 0 BEGIN

		SET @sql_column_names = @sql_column_names + '''\"' + @column_name + '\"'' AS ' + @column_name + ', '
		SET @sql_columns = @sql_columns + '''\"'' + CAST(REPLACE(' + @column_name + ',''\"'',''\"\"'') AS varchar(4000)) + ''\"'', '

		FETCH NEXT FROM c1 INTO @column_name
	END

	CLOSE c1
	DEALLOCATE c1

	SET @sql_column_names = LEFT(@sql_column_names,LEN(@sql_column_names)-1)
	SET @sql_columns = LEFT(@sql_columns,LEN(@sql_columns)-1) + ' FROM ' + @db_name + '.dbo.' + @table_name
	SET @sql = 'bcp "' + @sql_column_names + ' UNION ALL ' + @sql_columns + '" queryout "' + @file_name + '" -w -t , -T -S ' + @@servername
	EXEC master..xp_cmdshell @sql
END

GO

This stored procedure will automatically write out column headers, wrap your columns in double-quotes, and escape double-quotes in your columns. In other words, this outputs a *valid* CSV, which BCP does not.

The magic of this is the dynamically generated view, which we use to grab the column names for the view or table, and then iterate through them (using a cursor — not many good uses for cursors, but this is one of them) to build our column headers.

Recruit members and offer incentives for genuine participation in return for their advocacy.

Most healthy online communities have passionate stewards leading the way. Lurkers are the majority, and most of them will never participate beyond creating an account and occasionally logging in. Increase the chances of retaining a visitor and encourage them to participate in a meaningful way by drawing their attention to something new, interesting, and immediately relevant. The trend online is increasingly attention-deficit, and the newest social aggregators like Pinterest are a smoking gun.

You can’t create this content yourself. You can’t create 1/10 of it yourself. Have your core members create it for you. Or curate it for you. Run contests. Exchange pictures, video footage, and other content for merchandise. Set up guest blogging arrangements to draw both search engines and new visitors to your community. Reward your best participants in a meaningful, valuable way. What is it really worth?

Excerpt pulled from a series of thoughts written in 2009 for one of my clients

Focus.

You can’t cater to everyone. Choose one audience and win them over first.

The success of three of the largest social networks can be traced to a focused attempt to corral an audience and turn them into a powerful core group of early adopters. In Facebook’s case, this core group was university students, followed by savvy technical professionals at software companies and consulting firms. Facebook first focused their marketing at university students who would appreciate the “exclusivity” of a social network designed specifically for their needs, and then moved on to attract technical professionals at companies like Accenture and Microsoft who were comfortable online and, by the very nature of their jobs, dealt with multitudes of equally savvy contacts.

LinkedIn, a professional social network with an initial focus on independent professional marketing and advertising consultants, formed a critical mass of members which radiated outwards to form the 35,000,000 strong community it is today. As with Facebook, the strategic insight was to hitch the vehicle to a group of members (independent consultants) who interact with a lot of people on a regular basis, like bees pollinating flowers. Today, LinkedIn welcomes members of all professions.

MySpace made a well-considered strategic decision to embrace independent musicians and bands, and has since dominated the category. By providing the tools to host and play music, provide event listings, and communicate with fans, MySpace is now the de- facto place for an independent band to use to communicate with their fans.

The importance of picking a target audience and delivering a specific message and feature-set that resonates with that audience cannot be overstated. Personalized, one- on-one communication forms the basis for effective marketing. Listen and respond to your audience’s needs, develop functionality specifically for them, shepherd them, and watch these people turn into raving fans.

Excerpt pulled from a series of thoughts written in 2009 for one of my clients

-- assuming Magento 1.6.1 data model
SELECT DISTINCT customer_email,
	COUNT(*) AS purchases,
	(SELECT LEFT(created_at,11) FROM `sales_flat_order` WHERE customer_email = a.customer_email ORDER BY created_at DESC LIMIT 0,1) AS last_purchase,
	(SELECT COUNT(*) FROM sales_flat_order_item WHERE product_type = 'simple' AND sku LIKE '%IB%' AND order_id IN (SELECT entity_id FROM `sales_flat_order` WHERE customer_email = a.customer_email)) AS icebreaker,
	(SELECT COUNT(*) FROM sales_flat_order_item WHERE product_type = 'simple' AND sku LIKE '%TNF%' AND order_id IN (SELECT entity_id FROM `sales_flat_order` WHERE customer_email = a.customer_email)) AS the_north_face,
	(SELECT COUNT(*) FROM sales_flat_order_item WHERE product_type = 'simple' AND sku LIKE '%MHW%' AND order_id IN (SELECT entity_id FROM `sales_flat_order` WHERE customer_email = a.customer_email)) AS mountain_hardwear,
	(SELECT COUNT(*) FROM sales_flat_order_item WHERE product_type = 'simple' AND sku LIKE '%MAR%' AND order_id IN (SELECT entity_id FROM `sales_flat_order` WHERE customer_email = a.customer_email)) AS marmot,
	(SELECT COUNT(*) FROM sales_flat_order_item WHERE product_type = 'simple' AND sku LIKE '%LOLE%' AND order_id IN (SELECT entity_id FROM `sales_flat_order` WHERE customer_email = a.customer_email)) AS lole,
	(SELECT COUNT(*) FROM sales_flat_order_item WHERE product_type = 'simple' AND sku LIKE '%SW%' AND order_id IN (SELECT entity_id FROM `sales_flat_order` WHERE customer_email = a.customer_email)) AS smartwool,
	(SELECT COUNT(*) FROM sales_flat_order_item WHERE product_type = 'simple' AND sku LIKE '%OSP%' AND order_id IN (SELECT entity_id FROM `sales_flat_order` WHERE customer_email = a.customer_email)) AS osprey,
	(SELECT COUNT(*) FROM sales_flat_order_item WHERE product_type = 'simple' AND sku LIKE '%SOR%' AND order_id IN (SELECT entity_id FROM `sales_flat_order` WHERE customer_email = a.customer_email)) AS sorel,
	(SELECT COUNT(*) FROM sales_flat_order_item WHERE product_type = 'simple' AND sku LIKE '%ARC%' AND order_id IN (SELECT entity_id FROM `sales_flat_order` WHERE customer_email = a.customer_email)) AS arcteryx,
	(SELECT COUNT(*) FROM sales_flat_order_item WHERE product_type = 'simple' AND sku LIKE '%COL%' AND order_id IN (SELECT entity_id FROM `sales_flat_order` WHERE customer_email = a.customer_email)) AS columbia
FROM `sales_flat_order` AS a
GROUP BY customer_email
ORDER BY COUNT(*) DESC

The output of that query tells me how many times a particular email address has purchased from us (this aggregates together guest and logged-in-customer purchases, which is what I want), the date of their last purchase, and the number of items they have purchased from each of the major brands that we carry. The output looks something like this:

Now, it’s fairly obvious that the customer in row #2 is a great customer. They have made 7 purchases, each time buying a product from Arc`teryx, one of our premium brands. This person clearly loves Arc`teryx. Could they make my job as a marketer any more clear?

In most cases, you don’t see such a clear, strong, signal. I believe when you do, it should be acted on. This kind of business intelligence is immediately valuable. In LiveOutThere.com’s case, the Spring 2012 line is right around the corner, and I know that I will be able to sell that customer some great kit in a personal, intelligent way.

This brings me around to the title of my post, which questions the similarities between customer service and inside sales roles, and how the lines should blur in a boutique e-commerce operation like LiveOutThere.com. There are some fundamental aspects to customer service – fulfillment, communication, availability, etc. – but when you have those under control, I believe you need to build on those fundamentals and have your “customer service” team sell back to your customers in a precise, targeted way. And that’s a sales force, not a customer service department.

I believe we sell some truly great gear — the best available. Our team knows their stuff and can give you well-founded recommendations based on their personal experience with the product. When we know the most about your buying habits and preferences, we can serve you better than anyone else, and that’s just the reality. It’s better for our customers and it’s better for us.

On a practical level, we use this intimate knowledge of our customers to quickly create segments in our email marketing system. Take a peek:

How do you feel about that? Would you rather receive emails that are increasingly relevant to your needs, interests, and position in the buying cycle, or does it just piss you off?

Marketing has become increasingly micro-targeted over the last 10 years and you will see me talking about this more and more in the context of Magento e-commerce. It’s a fascinating, contemporary subject and brings up many interesting topics (privacy concerns certainly one of them) but I believe that in another 10 years our customers will absolutely not stand for marketing with a broad brush-stroke.

I’ve been interested in website conversion optimization tools since about 2006 – why has this area stayed so stagnant in the last 5 years? Are the tools too hard to use? Or is it that web content creators just can’t fit these tools into their process? Having worked in an agency, I can certainly vouch for the latter…

Of course there are a few high-end systems like Omniture (now Adobe) Test & Target. I haven’t personally used it — it’s a bit too rich for my blood, but I know there are bigger companies who use it to do very neat things. What about the smaller guys? Where are the Lean Startup style testing tools that allow you to rapidly roll out iterative changes to slowly but steadily improve your product? What are the 100,000+ Magento installation owners using?

Bottom line is, none of these tools work particularly well for testing features anyways. GWO can’t do it all – what if you wanted to try out a different product comparison tool on your Magento e-commerce store? Good luck. I haven’t ever tried to implement something like this with Optimizely, and maybe you can, but in the rest of this post, I’ll share the home-grown solution I use to roll-out changes to LiveOutThere.com and measure the results.

Start testing features in 5 minutes with this simple jQuery-based tool

Go and grab this gist from Github and include it on your website. It includes jQuery at the top so it works out of the box, but you’ll probably want to remove that. You’ll also need a testing “endpoint”, which you’ll set up on line 46, but don’t worry about that for now.

The gist contains a Javascript object called DG_tester that allows you to set up tests very quickly, just like this:

<script type="text/javascript">
jQuery(document).ready(function() {
	DG_tester.test(
		'red_or_green',
		function() {
			jQuery('#color').css('background-color','green');
		},
		function() {
			jQuery('#color').css('background-color','red');
		}
	);	
});
</script>

<div id="color">The background of this div will change color.</div>

DG_tester.test is a function that accepts a minimum of three arguments. The first is a string: the name of your test, and the rest are functions. The functions are passed as arguments so DG_tester can call them later – these functions are the different options you want to try on the page. For instance, swapping copy, swapping colours, or adding or removing entire chunks of functionality from your website.

When your visitor arrives, DG_tester will randomly pick one of your functions – let’s call them “tries”. Each function is assigned a letter of the alphabet by our script, so the first function following the test name is try A, the second try B, and so on. This choice will persist with the visitor for 7 days; when they do something you want, you score the try.

When you load the complete example you will now see a div that will be either red or green.

If you keep refreshing, you’ll see the same colour. Click the ‘Delete cookie’ link, try a few more refreshes, and you’ll get the other colour.

Every time a try is shown to a visitor, this little script calls your endpoint to tell it that a try has happened. When your visitor does the thing you want (like adds a product to cart, purchases something, clicks the ‘Score’ link in the example, or simply continues to the next page) you would call DG_tester.score, like this:

jQuery('#add-to-cart').click(function() {
	DG_tester.score('red_or_green');
});

If you are monitoring your network traffic in the browser console, or if you look at the generated source of the example page, you’ll see that two images have now been created. Of course, both of these resolve to 404 pages, since you don’t have an endpoint yet.

The purpose of the endpoint is to record scores and tries. This is really pretty simple – don’t let anyone else tell you otherwise. Cavemen did it by recording scratch marks onto clay tablets. We use a really fast Node.js backend that uses MongoDB to record data, but you don’t have to do anything like that. A simple PHP script that grabs the testId out of the query string and writes it to a MySQL table will work just as well. Or you could append to CSV files and interpret the results later with Excel.

At any rate, when you’ve got a backend rigged up and recording data, let your tests run for a while, divide scores by tries for each try, and pick the try that worked best! My rule of thumb is that you should have about 400 tries before you consider anything statistically significant.

Of course, none of this is probably particularly statistically significant anyways – but that’s not the point! You need to be testing. Test, test, test, you’ll get the hang of it. Then tell me what you’re testing, because nothing I’m testing this week seems to make a damn bit of difference. And that’s a blog post for another day: what kind of tests are likely to generate the most informative results and guide your path? Why, when I test, do most of my tests seem inconclusive at best? That’s perhaps the real reason why a lot of these testing tools have never taken off…

For instance, if you send feeds to shopping price comparison networks, use this little trick to set up a hidden 5% price difference and you can easily position your products ahead of the competition while staying on-price. At first consideration, it would seem that you should be able to set this up using a catalog price rule rather than a shopping cart price rule, but unfortunately (as far as I’ve been able to tell), you can’t, at least for not-logged-in visitors.

To get Magento to apply coupon codes automatically, you will need to create a controller that looks for a coupon code in your query string, then forwards the visitor on to the requested product or category page. The URL to make this happen will end up looking like this:

http://www.liveoutthere.com/applycoupon/index/?url=buy/the_north_face.html&coupon_code=TNF10

Shouldn’t this be done with an event observer?

No. Personally, I see two good reasons for using a controller and redirecting, rather than just appending a query string variable to the destination page URL and catching it with an event observer: future-proofing, and cache-breaking. On the topic of future-proofing, a controller will allow you to hack in additional functionality more easily when you want to set up complicated discount strategies, it will allow you to do things like check for stock prior to delivering someone to an out-of-stock product page; and in my opinion it is easier to understand than diving right into event observers (although you’ll need one of those too – give me one more paragraph).

As far as cache-breaking, if you’re using a full-page cache like TinyBrick’s LightSpeed, doing things this way will make life easier for you when you have to figure out how to serve special content for discounted products and categories – because the LightSpeed logic happens before everything else in the Magento system, it is *way* easier to deal with these special cases prior to page load than trying to use event observers and complicated cache hole punches. Trust me on this.

Now, before getting to the meat of this how-to, there are many great Magento event observer tutorials, and I suggest you spend some time familiarizing yourself with event observers before proceeding if you haven’t already. They are certainly one of the most powerful aspects of Magento.

Creating a Magento extension to apply coupon codes automatically

The first thing you need is an extension in your /app/code/local directory – I’ve named mine ApplyCoupon. The purpose of this post is not to explain Magento extension development, so I’m just going to give you a few GitHub gists and a screenshot and leave it at that.

First, you need to save the coupon code in session. Because of the way Magento works, we can’t actually apply a discount to anything unless we have a quote, and we can’t have a quote until we have a product in the cart… so this is a bit of a catch-22, and we will need an event observer to resolve it.

Before you worry about that, the class in your IndexController.php needs to contain this method:

public function indexAction() {    
    $coupon_code = $this->getRequest()->getParam('coupon_code');
    
    if ($coupon_code != '') {
        Mage::getSingleton("checkout/session")->setData("coupon_code",$coupon_code);
        Mage::getSingleton('checkout/cart')->getQuote()->setCouponCode($coupon_code)->save();
	Mage::getSingleton('core/session')->addSuccess($this->__('Coupon was automatically applied'));
    }
    else {
        Mage::getSingleton("checkout/session")->setData("coupon_code","");
	$cart = Mage::getSingleton('checkout/cart');
	foreach( Mage::getSingleton('checkout/session')->getQuote()->getItemsCollection() as $item ) {
	    $cart->removeItem( $item->getId() );
	}
	$cart->save();
    }

    if ($this->getRequest()->getParam('url')) {
	//using raw header instead of _redirect because _redirect appends a /
	header('HTTP/1.1 301 Moved Permanently');
	$gclid = this->getRequest()->getParam('gclid');
	$url = $this->getRequest()->getParam('url');
        header('Location: /' . $url . '?gclid=' . $gclid);
	die();
    } else {
        $this->_redirect("/");
    }
}

Note the gclid query string parameter we’re appending to the URL on line 21 – this is really important. If you don’t append it, you will lose your auto-tagged campaign URLs in Google Analytics. Learn from me and don’t make that mistake!

Now, you need an event observer attached to checkout_cart_product_add_after that fires when your shopper adds something to their cart and applies the discount. The class in your Observer.php file will contain one method, applyCouponEvent, to grab the coupon code out of session and apply it to the current quote:

public function applyCouponEvent($observer){
    $coupon_code = trim(Mage::getSingleton("checkout/session")->getData("coupon_code"));
    if ($coupon_code != ''){
        Mage::getSingleton('checkout/cart')->getQuote()->setCouponCode($coupon_code)->save();
    }
}

And here are the bits you’ll need in your config.xml:

<frontend>
	<events>
		<checkout_cart_product_add_after>  
		<observers>  
			<ApplyCoupon_apply>
				<type>singleton</type>  
				<class>ApplyCoupon/Observer</class>    
				<method>applyCouponEvent</method>            
			</ApplyCoupon_apply>              
		</observers>  
		</checkout_cart_product_add_after>
	</events>
	<routers>
		<applycoupon>
			<use>standard</use>
			<args>
				<module>You_ApplyCoupon</module>
				<frontName>applycoupon</frontName>
			</args>
		</applycoupon>
	</routers>
</frontend>

At this point this will all actually work – you will be able to load your controller action, add something to your cart, and see the discount applied. Unfortunately, like most things Magento, this won’t cut it. Your visitors still have no indication that there has been a discount applied until they add a product to cart.

Just a few more steps…

You can add this code to your template/catalog/product/view.phtml and list.phtml template files to get the discounted price as if the discount code had already been applied:

$productPrice = $this->getPriceHtml( $_product, true );
$coupon_code = trim(Mage::getSingleton("checkout/session")->getData("coupon_code"));

if ($coupon_code != '') {
	$db = Mage::getSingleton('core/resource')->getConnection('core_read');
	$sql = "SELECT discount_amount, conditions_serialized
		FROM salesrule_coupon AS a INNER JOIN salesrule AS b ON a.rule_id = b.rule_id
		WHERE simple_action = 'by_percent' AND a.code = ?";
	$sql = $db->quoteInto($sql, $coupon_code);
	$rows = $db->fetchAll($sql);
	if (isset($rows[0])) {
		$discount = ($rows[0]['discount_amount'] / 100);

		/*** This assumes that if there is a shopping cart price rule condition,
		  *  it is a condition with only one parameter - that the product belongs
		  *  to a particular attribute set.
		***/
		$conditions = unserialize($rows[0]['conditions_serialized']);
		$conditions = $conditions['conditions'][0]['conditions'][0];
		if (is_array($conditions) && $conditions['value'] == $_product->getAttributeSetId()) {
			// someone want to write me a regex for this nasty piece of work?
			$discountedPrice = str_replace(' ','',strip_tags($productPrice));
			$discountedPrice = str_replace("\n",'',str_replace('$','', $discountedPrice));
			$discountedPrice = round($discountedPrice - ($discountedPrice * $discount),2);
			$productPrice = "<div class='price-box'><span class='regular-price'>
					 <span class='price'>$" . $discountedPrice . "</span>
					 </span></div>";
			$sale_price = true;
		}
	}
}

Use the $sale_price flag to turn on sale badges, red-tag creative, etc., and echo $productPrice where you would normally. There are probably a few purists out there who would object to this approach, and I’d love to hear from you if you can think of a better way to do this. This was the best I could come up with.

The condition check on line 15 will allow you to display the discounted price, only if the product in question belongs to the attribute set specified by the condition in the shopping cart price rule — even though the coupon code will not apply if you have these conditions set up, you still need the template to be aware of the conditions so that you display the correct pricing. That’s a mouthful!

In LiveOutThere.com’s case, we add all of our The North Face products to their own attribute set, so we can show 10% discounts on just The North Face products when you come through our paid search campaign.

If you are using a full-page cache, you will also need to create a hole punch for the value of $productPrice, but that’s a topic for another post.

That’s it from me. Questions?

If there is enough interest, I would be happy to write another post about how to approach even more complex shopping cart price rules that involve giveaways and gift cards – a great example of something we’ve done at LiveOutThere.com is run a promotion where we gave our shoppers a free $75 gift card (we use Unirgy’s uGiftCert) if they spent $250 or more. This was a big hit with our customers and is a great promotional strategy that you should be using too.

In order to give our customers on the west coast a chance to buy product before everyone on the east coast snapped it up, we decided to put up a holding page and close the doors until 11am MST. I placed our Google Analytics tag on the holding page and watched eagerly as hundreds of concurrent visitors piled in.

11am – the sale is on!

When 11am came, I removed the RewriteRule that forced all of our inbound visitors to the holding page, and boom – the website instantly choked. The disappointment my partners and I felt as we imagined the frustration these hundreds of eager shoppers must have experienced hung in the air. This was not supposed to happen.

We have two servers – one front-end web / cache server, and one dedicated database server. We do not run a dedicated reverse proxy like Squid or Varnish, and in this case, I don’t think it would have made a difference. We already use Amazon CloudFront to host our media assets and CSS. Our CSS and JavaScript are minified, and we try to follow all of the YUI guidelines. It was safe to say that this was not a front-end problem. In anticipation of the traffic increase, I had recently switched to Redis for our Magento cache backend. We use Redis in combination with TinyBrick’s LightSpeed extension to store the full HTML of our pages, which will theoretically reduce the database traffic to near zero if the cache is hot.

11:15am – initial diagnosis and damage control

Communicate early and as often as possible.  Our Facebook page allowed us to continue a conversation with our waiting customers, and ultimately serve them one-on-one.

Our dedicated MySQL server has 24GB of RAM and uses the key configuration values shown in the pull-quote above. This was based on the best practices found at magentocommerce.com. This is certainly not a cheap piece of hardware, and even under heavy load it should have performed adequately – something wasn’t adding up. I tried restarting MySQL a few times. No dice. I would get 2 seconds of operation and would immediately be back in the 100% utilization range. I knew I had two different challenges – first, how to bring the site back up in some sort of minimal viable configuration, and second, what the hell was causing the utilization in the first place? Our Redis cache was hot, and the database server should have been under almost no load.

11:30am – minimum viable fix

I realized that the only option I had would be to use mod_rewrite to do traffic control — by letting some customers in, but not all of them, I could keep the load on the database server under control and allow at least some of our customers to make purchases. This posed a few problems – you can’t let someone in and then kick them back out again, so I knew I eventually had to solve the real issue, as I would eventually begin to swamp the database server anyways. My first instinct was to do this by IP subnets and I broke the fully-allowable range of IP addresses into a few separate chunks. This was an obvious technical solution, but within a few minutes I realized it was going to just make our customers even more mad. I needed a human solution.

Learn from Disney. Make the waiting experience a shared, collective experience. Even if it’s frustrating, it’s memorable and it’s an opportunity to let your real strengths shine through – this is ultimately a story about great customer service, not database server performance.

Solution #2 was to engage each customer one-on-one. I instructed Alex, our customer service lead, to collect IP addresses from our most vocal Facebook customers so I could let them in personally. This was a seat-of-the-pants decision that turned out to be a wise one. Where before I had hundreds of customers, none of whom seemed to be able to get into the store (and I guess that’s statistics for you), now I had hand-picked customers who seemed to be particularly patient, particularly frustrated, or particularly interested in communicating with us while we determined the problem.

Allowing our most vocal Facebook fans in first quickly turned the tide of discussion on our Facebook page from frustration to hopeful anticipation.

As we slowly began to process sales, a vibe developed on our Facebook page that was all about the exclusivity of access, the anticipation of being allowed into a great sale, and the shared experience of waiting. Customers posted IP address after IP address, and as Alex read them to me I retyped out RewriteCond directives to give each person access. When they received access, they would almost always make a positive comment about it on the wall.

1:00pm – the true nature of the problem

After an hour and a half of coordinating Facebook sales, I had a chance to think about what was going on with the database server – and had no choice – as our utilization was creeping higher because by that point I had about 150 people shopping. The obvious next step was to use MySQL’s SHOW PROCESSLIST command to see exactly what kind of queries were being requested in the first place. At first glance, this showed me a whole bunch of queries that were in SLEEP state. I assumed these queries were waiting for row locks, but some of the times were so high (in the 30+ second range), that I figured just killing them outright would be better than serving a page in 30+ seconds anyways.

You can use pt-kill, a utility from Percona’s advanced MySQL toolkit, to selectively kill queries that match your criteria.

Using pt-kill, I set up a process that would scan for SLEEPing queries every 30 seconds and kill them – I figured this would mean better performance for the queries that were getting served, and at the very least would serve error pages to our customers quicker than letting them grow older watching a spinning cursor. This would also reduce our connections count significantly every 30 seconds, so we weren’t throwing max_connections exceeded errors. This provided another stop-gap measure that bought me another hour while I could diagnose the real problem.

2:00pm – partial resolution and the opening of the floodgates

After watching SHOW PROCESSLIST carefully I began to pick out particular queries that didn’t make a lot of sense to me; namely involving customer sessions. This was my first “aha!” moment all day.

I had no choice – it was either switch the location sessions were being saved to, or eventually choke the MySQL server again. I made the change and refreshed Magento’s configuration cache.

Within a few minutes the utilization on the database server dropped to a more reasonable level. I began removing RewriteConds cautiously, monitoring MySQL as I saved each copy of .htaccess back up to our web server. Within 10 minutes, I had removed all of the rewrite conditions and every one of our customers was able to shop. Unfortunately as a result of changing the session_save location, we did have a number of angry customers who had items in their cart or had almost completed checkout, and we made every effort to make it right for each one of these customers on a case-by-case basis. In many cases, we captured the order details using our Olark LiveChat instance and simply placed the order on the customer’s behalf.

3:00pm to 1:00am – seeing it through

Now that customers had full access to the store, we began receiving orders one after the other. For the remainder of the day, we averaged about 40 orders an hour. At any given time, we had between 100 and 200 active shoppers, and dozens of people checking out simultaneously. We use Dg_Matrix to watch shopper behaviour, and the screen was flying. Alex and I stayed on live chat until approximately 1:00am, and processed many one-on-one sales. At one point, Alex had 36 chats going at once. Somehow, she kept it together.

At LiveOutThere.com, we attempt to offer a full-service approach comparable to an in-store experience over LiveChat. That means helping you determine exactly what winter jackets you should compare, how they should fit, how much you should spend, and what other alternatives might be available to you.

We finished the sale day between midnight and 1am, ringing up $100,000 in sales and hopefully delighting more customers than we disappointed. The customer service follow-up has been steady, and we have been quick to offer extensions of our sale prices for anyone who was unable to purchase in the initial few hours of the sale. In conclusion, this experience has certainly shown the way forward for future optimizations, illustrated a clear need to lint our configuration files, and taught us valuable lessons about damage control and the power of open, transparent communication with customers.

To write data to Redis you will need to add a new cache server model to your LightSpeed extension. Create the file /app/code/community/TinyBrick/Lightspeed/Model/Server/Redis.php with the contents of the following gist:

<?php
class TinyBrick_LightSpeed_Model_Server_Redis {
	protected $_server;
	protected $_enabled 		= false;

	public function getServer() {
		if (!$this->_server) {
			require_once($_SERVER['DOCUMENT_ROOT'] . '/lib/Credis/Client.php');
			require_once($_SERVER['DOCUMENT_ROOT'] . '/lib/Zend/Cache/Backend.php');
			require_once($_SERVER['DOCUMENT_ROOT'] . '/lib/Zend/Cache/Backend/Interface.php');
			require_once($_SERVER['DOCUMENT_ROOT'] . '/lib/Zend/Cache/Backend/ExtendedInterface.php');
			require_once($_SERVER['DOCUMENT_ROOT'] . '/app/code/core/Zend/Cache/Backend/Redis.php');
			$options = array();
			$options['server'] = (string)Mage::getConfig()->getNode('lightspeed/cache/backend_options/server');
			$options['port'] = (int)Mage::getConfig()->getNode('lightspeed/cache/backend_options/port');
			$options['database'] = (int)Mage::getConfig()->getNode('lightspeed/cache/backend_options/database');
			$options['timeout'] = (int)Mage::getConfig()->getNode('lightspeed/cache/backend_options/timeout');
			$options['force_standalone'] = (int)Mage::getConfig()->getNode('lightspeed/cache/backend_options/force_standalone');
			$options['automatic_cleaning_factor'] = (int)Mage::getConfig()->getNode('lightspeed/cache/backend_options/automatic_cleaning_factor');
			$this->_enabled = true;
			$this->_server = new Zend_Cache_Backend_Redis($options);
		}
		return $this->_server;
	}

	public function save($key, $data, $expires=0, array $tags=array()) {
		$server = $this->getServer();
		if ($server && $this->_enabled) {
			$server->save(serialize($data), $key, $tags, $expires);
		}
	}

	public function cleanByTag($tag) {
		if ($this->_server) {
			$this->_server->_removeByMatchingTags($tag);
		}
	}

	public function clean($tags=array()) {
		if ($this->_server) {
			if (count($tags) && !in_array('LIGHTSPEED', $tags)) {
				$this->_server->_removeByMatchingTags($tags);
			}
			else {
				$this->_server->clean(Zend_Cache::CLEANING_MODE_ALL, $tags);
			}
		}
	}
}

Then, you need to modify the TinyBrick_LightSpeed_Model_Server::getServer function in /app/code/community/TinyBrick/Lightspeed/Model/Server.php so that it knows about the new Redis cache server model you just created:

	public function getServer()
	{
		if(!$this->_server){
			$type = strtolower(Mage::getConfig()->getNode('lightspeed/cache/type'));
			if ($type == "memcached"){
				$this->_server = Mage::getModel("lightspeed/server_memcache");
			}elseif ($type == "redis"){
				$this->_server = Mage::getModel("lightspeed/server_redis");
			}else{
				$this->_server = Mage::getModel("lightspeed/server_files");
			}
		}
		return $this->_server;
	}

Now, you can modify the LightSpeed configuration section of your local.xml file to use Redis instead of Memcache. Mine looks like this (the following XML should be a child of your <config> tag of local.xml:

<lightspeed>

    <global>
    REDACTED
    </global>

    <session>
    REDACTED
    </session>

    <cache>
      <type>redis</type>
      <backend>Zend_Cache_Backend_Redis</backend>
      <backend_options>
        <server>/usr/home/outthere/redis.sock</server>
        <port>6379</port>
        <database>3</database>
        <force_standalone>0</force_standalone>  <!-- 0 for phpredis, 1 for standalone PHP (slower) -->
        <automatic_cleaning_factor>20000</automatic_cleaning_factor> <!-- optional, 20000 is the default, 0 disables auto clean -->
      </backend_options>
    </cache>

</lightspeed>

Good work. Now LightSpeed will save information into Redis instead of Memcache! You can verify this is occurring by opening the redis-cli command line utility and running the MONITOR command. You will see a lot of activity every time you request a page – pore through it and verify that LightSpeed is actually saving what you think it should be saving to cache – primarily, the HTML of your page!

Now, you just need to modify lightspeed.php to serve the information you’ve stored in Redis. This is a little easier. First, you need to add a case for Redis in the PageCache::loadConfiguration function of lightspeed.php:

case 'redis':
try {
	require_once('lib/Credis/Client.php');
	require_once('lib/Zend/Cache/Backend.php');
	require_once('lib/Zend/Cache/Backend/Interface.php');
	require_once('lib/Zend/Cache/Backend/ExtendedInterface.php');
	require_once('app/code/core/Zend/Cache/Backend/Redis.php');
	self::$cacheEngine = 'redis';
	$options = array();
	foreach($child2->backend_options as $el){
		$options['server'] = (string)$el->server;
		$options['port'] = (int)$el->port;
		$options['database'] = (int)$el->database;
		$options['timeout'] = (int)$el->timeout;
		$options['force_standalone'] = (int)$el->force_standalone;
		$options['automatic_cleaning_factor'] = (int)$el->automatic_cleaning_factor;
	}
	self::$cacheData['server'] = new Zend_Cache_Backend_Redis($options);
}
catch (Exception $e) {
	die('Redis can not be used as the backend for Lightspeed. Check local.xml for proper configuration.');
}
break;

 Then, you’ll need to add a case for Redis in the PageCache::get function:

case 'redis':
return unserialize(self::$cacheData['server']->load($key));
break;

There you go. That should be enough to get you up and running. LightSpeed is a great extension for Magento, but unfortunately since TinyBrick has been acquired their support has become non-existent. Hopefully this functionality can be incorporated into a future version of LightSpeed. Hello TinyBrick? Anyone there?

Before I moved us over to Redis, we were running a single memcached instance on the same server as our web server, and we had mounted Magento’s SQLite “slow cache” in a tmpfs file system in RAM. We used the memcache 2.2.6 extension to talk to memcached (seems this is a point of confusion for some people, as there are two PHP extensions to talk to memcached). My main complaint was that I could not selectively refresh anything in the cache without bringing the whole store to a grinding halt. Memcache does not support cache tags natively, so without getting into the dirty details, what the Magento team has done is use their second “slow cache” to store the key names in Memcache for specific tags. This is clever, but not very efficient, and now that there are better caches that actually support cache tags natively, like Redis, it becomes obvious how much this was a square peg in a round hole solution.

Now, we are still running Redis on the same server (we don’t have a dedicated cache server), but I am seeing much better performance and most importantly, it is easier to manage. Redis has a command line interface that allows me to selectively remove an item from the cache, and if you’re using a full page cache extension like Lightspeed, this is a god-send.

For instance, if our designer wants to add a new banner to the home page, I can just do this to remove the home page cache key:

DEL "zc:d:www.liveoutthere.com/"

This was possible with Memcache if you were willing to write your own custom cache interface, but I’ve never heard of anyone doing this with Magento out in the wild. That’s what the Refresh Cache buttons are for (right?) – yeah sure, go ahead and press it on your busy production website. I would see our MySQL usage peg the processor near 100% for 15 or 20 minutes while the cache was rebuilding, and the site would be so slow it was unusable. Theoretically, the Magento CMS is supposed to remove the cache key when you save your CMS page, but this doesn’t work when you have an additional cache system like TinyBrick’s LightSpeed in the mix.

Using LightSpeed? You’ll need to do a few more things to get Redis working.

So, how do you do this yourself? I followed the instructions at Flexiblog and they worked for me. Try it out and let me if it’s made managing your cache as easy for you as it has for me.