Tuesday, December 30, 2014

Scraping Air Fares


 Airlines have arcane pricing structures, which sometimes allow for loopholes in pricing.
This young man exploited a huge hole in the pricing system and is now being sued.

When I was a young hotshot attorney, I used to fly to Silicon Valley every month, from Washington, DC.  If I went on a Monday and came back on a Thursday, the air fare was very high.   In order to get the lower fares, you have to stay over a weekend - or at least that was the rule back then.

Why this rule?   I think it was to differentiate between the business traveler (traveling during the week) and the vacation traveler (traveling over the weekend).   The vacation traveler is price-conscious.  The business traveler, less so.

Owning my own business, I was price-conscious.  Since I made the trip once a month,  I booked a one-way ticket to SFO and then booked a round-trip flight from SFO to Dulles, "staying over" in Washington, DC for four weekends.  The airfare dropped from $1000 to $250 this way - round-trip.

So long as I kept traveling and booking these tickets, it worked well.  When my main client closed their Silicon Valley office, I stopped traveling - and threw away one-half of that last round-trip ticket.  While that was "wasteful" I did end up saving thousands of dollars in airfare.

And back then, frequent flyer miles were awarded based on miles traveled - not money spent.  So I quickly racked up frequent flyer miles to the point where I could get an upgrade to business class or even first class, if the planes were empty.   And I quickly figured out which flights were most likely to be empty, and booked those.

But of course, the airlines cried foul over this, and threatened to revoke tickets of passengers who attempted this scheme.   But it was an empty threat.   How could they tell I wasn't staying over in Washington for four weekends?   They wrote the ticket, took my money, and that was that.

Recently, a young Engineer was sued by both United Airlines and Orbitz for operating a website called skiplagged.com   What was he doing that was illegal?

Well, it turns out, nothing.  But the airlines are pissed, as he is providing a service to allow users to "scrape" lower fares from the airlines.

What is fare scraping and do you need to use this website to scrap fares?  The answer to the second question is "No."   Anyone can scrape fares and it takes only seconds to do.

For example, I went on www.usairways.com and looked for fares to Washington DC from Jacksonville, Florida on January 11th.  The first flight to pop up was this one, at $204 non-refundable one-way.




  • 4546 
  • 7:20 AM JAX
  • 9:17 AM DCA
Operated by Republic Airlines dba US Airways Express
(0) 1h 57m $204 $411 $353 $560

OK, that seems reasonable.  But can we get it cheaper?   Well, try to book to another city that might use DCA as a layover.  For example, New York's JFK airport:


  • 4546 
  • 7:20 AM JAX
  • 9:17 AM DCA
Operated by Republic Airlines dba US Airways Express

  • 3370 
  • 10:40 AM DCA
  • 11:59 AM JFK
Operated by Envoy Air dba American Eagle
(1) 4h 39m $137 $385 $456 $1,129


Whoa, did that just happen?   It is cheaper to fly to New York than it is to fly to Washington, DC, even though both itineraries use the same base flight.  And not just cheaper - a whole lot cheaper.   Almost 50% off - $137 versus $204, one-way!   Note however, how the prices change as you move up the scale.  First class passengers pay twice as much to go to JFK, but intermediate fares may vary.

To "scrape" the best fare, you book one-way, to a further destination, and simply get off the plane at the transfer point and don't bother getting on the second leg.

This does require that you book one-way and not check any luggage.  If you checked your bags, they would go through to JFK, which would obviously create problems for you - and the airline (since you would not be on the flight, your bags would be yanked off - or should be, anyway).

It should be noted also, that usually, you have to get on the plane at the first leg.

But what about round-trip airfares?  Aren't they cheaper?   Well, I tried booking round-trip from JAX to DCA on USAir, and the airfare was...... $204 each way.

By the way, do you see why I say that airline miles are shit?   We got a "companion certificate" from U.S. Airways, offering to fly a companion anywhere for $99 if you buy another ticket "at the regular price".   When regular prices are $137, a $99 fare is not really a huge incentive - especially considering what you have to go through to get it.

So why do airlines have such wacky fares?    Well, USAir may be competing with other carriers for flights to JFK, and thus has to offer a lower fare to that city to compete.   But for flights to DCA, there may be less competition, so people are willing to pay more, so they raise the price.

Again, prices are not determined by adding up your costs and then tacking on a "reasonable profit" - communists and emotional thinkers actually believe that.  Rather, you charge what the market will bear and in some cases, this may be below your costs.    An airliner leaving the gate with an empty seat is an airliner not optimizing its revenue.  So if you can sell that last empty seat, even if the revenue is half the cost-per-seat to fly a passenger, you still come out ahead.   Obviously, you can't fill the whole plane with passengers like that - you'd go bankrupt.  But a hundred bucks more is a hundred bucks more, so you take the money if you can get it.

And what people are "willing to pay" is quite easily quantifiable.   And yes, companies realize that some folks will use techniques like "fare scraping" or "ultimate couponing" to score deals.   This is part of the marginal pricing strategy.   Those desperate enough to seek out extreme deals, often can get better deals.  To those whom price matters less, well, they pay more.   Why offer low prices to people who don't care about price?   This does not mean, of course, that you can always score extreme low prices - or that the effort involved is always worth it.   I am not a fan of couponing as a way of getting rich.  Couponing is not a way of getting rich - it is a sign that you are poor.

The CNN article cite above states that in order to "scape" airfare, you should book to a less popular city.  I think they have this backwards, and clearly New York City (JFK) is not a "less popular" city.  And in fact, if you book to a less popular city, prices go up even further.


Now this same flight to DCA, continued to Syracuse, gets really expensive:


  • 4546 
  • 7:20 AM JAX
  • 9:17 AM DCA
Operated by Republic Airlines dba US Airways Express

  • 4668 
  • 9:59 AM DCA
  • 11:14 AM SYR
Operated by Republic Airlines dba US Airways Express
(1) 3h 54m $377 $901 $632 $1,076

Wow!  I can fly to JFK to $137, but if I wanted to fly to Syracuse (which is not that much further) it costs nearly three times as much.   Why is this?   Well, likely there is less competition on that flight (no one wants to go to Syracuse, believe me!) and they can get away with charging more.

And these fares are not set manually by human beings, but programmed by computer algorithms, who check competing fare prices, plane schedules, historical data, and the like.   They try to optimize how many seats are empty (preferably none) versus the optimized fare price they can receive.

So why is this young man in hot water?   Well, his website, skiplagged.com basically automates what I found out in about two minutes of clicking online, only with more options, routes, and airlines.  And I must confess that I cheated - I searched initially for a JAX-JFK flight and then found that for USAir, all flights go through DCA, and then checked on that.  If I was booking to DCA, I probably would not have thought to check flights to JFK as part of a fare scraping scheme.

Should you fare scrape?   Well, I would have to say, use at your own risk.   The airlines can deny service to people for a number of reasons, and if they think you are scraping fares this way (which they would figure out, when you continually "skip" those last legs of flights) they might ban you from the airline or deny you boarding.

But they threatened to do that back in the 1990's, when I was using a similar technique to fly across country on reverse round-trip tickets.   They never caught up with me.

I suppose we should argue that airlines should have more transparent and logical pricing.   But that ain't gonna happen anytime soon.   The name of the game is to fill each seat with the passenger who will pay the most for each seat on the plane, until the plane leaves the gate with every seat full (or just one seat empty, which means you have just satiated demand).

So airline pricing games will be around for a long time.   And so long as they have these pricing games, there will be loopholes you could possibly exploit.