Web Scraping for Retail: When to shop at Abercrombie & Fitch

According to Luxury Daily, online retail has grown 113 percent over the past ten years while traditional retail has only grown 17 percent.  As the market size for online retail increases, the ability to access retail data becomes more important.  That’s where Mozenda comes in.  We decided to tackle a few complex retail sites to see how our tool performed and to develop some agent-building best practices.  This blog post represents a sneak peak at some of our findings and some insights we gained into the pricing behavior of Abercrombie & Fitch.

Our Working Data Set

Each week beginning last June we used Mozenda to capture product content from Abercrombie’s website.  Over time we kept track of retired products and new products using a unique product ID found on the site. After one year we concluded our research and began to play with the data.  We organized the data into tables of values that we could query and analyze. This allowed us to gain insight into things like percentage of products on markdown, average markdown as a percentage, and count of new products.

The Best (and Worst) Time to Shop

One of our goals was to determine the best and worst time to shop at Abercrombie. The chart below demonstrates this. The (red) line shows the percentage of products that are discounted and the (green) line shows the average discount per product. The (blue) line is a multiplication of those two percentages and was our way to determine the best and worst times to shop. The closer that both lines are to 100%, the better  the time to shop.  Low points for all lines would indicate fewer items available at less of a discount, or, the worst time to shop.

From the chart we are able to see the best and worst times to shop, the expected and the unexpected. A quick glace at a peak in the middle of this graph shows us that “Black Friday” lives up to its name. One of the very best times to shop was the end of November. Almost immediately after Black Friday we find one of the worst times to shop which is the week or two leading up to Christmas.  Make sure to do your Christmas shopping early!  A few other good times to shop were the middle of July and September while a few not so great times to shop were mid October, late January and early April.

How long does it take before an item goes on sale?

Like most retailers, Abercrombie has several types of sales. The first type of sale has no specific category and is simply a type of markdown. This is when you see two prices, one of which is crossed out to indicate a drop in price. Interestingly enough this markdown is often on brand new products. From day one there are many products that are on “sale.” Abercrombie also has two other sale categories, one marked “sale” and one marked “clearance.” In order to determine how long it took for a product to have some type of price reduction we ignored all three of these categories and captured the price at which you could purchase a product when it was first introduced, then tracked that product until the purchase price dropped.

The graph above represents a basket of 7,000 products and the time it took for an initial price reduction. What we found was that  65% of products had a reduction within the first 30 days, 85% within 60 days and 90% within 90 days. As good as this sounds, its actually deceiving because some products actually went up in price before coming back down below the original offering price.  In fact, around half of the products had a price increase at some point after their initial introduction.   A good example of fluctuating pricing can be found with Abercrombie’s Super Skinny Jeans.  The graph below shows the purchase price week by week.  Notice that the jeans were most expensive in the month of September and least expensive in the month of July just a few months prior.

Introducing the “Kill Rate”

One of the metrics we like to track for retail is called the “Kill Rate.” The kill rate is the rate at which a retailer’s products disappear from the shelves.  The kill rate can be one of the most interesting pieces of data because it can theoretically reveal the success or failure of a given product line. When a product dies it could indicate a defect, a loss of consumer interest, supply problems or simply a change of season. When a large number of products die over a short period of time it usually indicates that there is a serious problem.

Above is the graph for our 8 Week Kill Rate. Each point on the graph represents the percentage of products that disappeared over the previous eight weeks. One interesting find was that there was a  significant kill rate in the August/September 2014 time frame. Simultaneously with that above average Kill Rate, Abercrombie’s stock price started its long  slide downward as shown by the (Orange) graph.  Is there correlation? Can a high Kill Rate partially predict performance?  We’re not saying either way, but the data sure is interesting.

From Internet to Intelligence

Mozenda helps companies world wide transform the internet into actionable intelligence. As the amount of accessible data increases, the ability to make smarter decisions increases. Collecting retail data over a period of time to reveal patterns is only one application of using Mozenda.  Please take a moment to think how new data, better data, and more data can benefit your organization.