There’s always something to howl about

Thinking skeptically to rain on’s parade . . .

I’m not a Jesuit, but I play one on BloodhoundBlog. The real truth is, I’m a roll-your-own Jesuit, more auto-didact than anything. I didn’t have Brian Brady’s inestimable advantage of having had the gift of reason literally pounded into me. Instead, I had to stuff it between my own ears by hand. But one way or another, lay student or Brother, if you walk in the path of Ignatius Loyola, you learn to think skeptically. Any affirmative claim is far more likely to be false than true.

This morning, posted a claim that MLS results “prove” that Redfin agents are better negotiators than other agents in the Seattle area. If CEO Glenn Kelman had made a claim like this in Brother Paul’s class, he’d be up late tonight writing a paper, striving either to prove or disprove it.

The problem is not that the claim is necessarily false. The problem is that that there are so many ways that it might be false that, to call it true without eliminating each one of these canards and false paths is an inherently tendentious statement — suasion, not persuasion.

Before I begin work on my much shorter paper on why the claim is dubious, I want to raise three meta-issues. First, I do not have access to the underlying data. If I did, I might write a much longer and much more conclusive paper. Second, I would have much greater faith in the mainstream media if more reporters were tuned to a Jesuitical tenor of skepticism. And third, the tabbed browser window is an excellent tool for organizing the resources to be used in an exercise like this.

First, Redfin claims that its results rebut the claim that a salaried (and possibly inexperienced) agent will not negotiate as aggressively as a traditional real estate agent working on a straight commission compensation plan:

After a year in the market, we decided to put our theory to the test, by querying the Northwest Multiple Listing Service for data on every home or condominium sold via a brokerage from February 6, 2006 (the date of Redfin Direct’s launch) through February 5, 2007. Since we didn’t offer a service for sellers or support areas outside King County until much later in the year, we limited the data to King County and we only evaluated our capacity as buyers’ agents.

But we still had the problem that Allan highlighted, namely that there is no “set base” price for a home.

So we compared what buyers’ agents negotiate for — the final price — to what the sellers’ agents ask for — the asking or listing price; some sellers’ agents may ask for too much, others for too little, but all King County brokerages are negotiating against the same set of asking prices (note that evaluating a seller’s agent is problematic, since the seller’s agent only competes against the prices she sets herself.)

The results were striking; Redfin customers paid on average under asking price, whereas customers of all other brokerages paid on average over asking price. The difference in negotiations was .9% of the home price, equivalent in King County to over $4,000, on top of a commission refund of nearly $10,000.

In greater detail:

  • Redfin King County customers paid on average 99.329% of the listing price while buyers with other brokerages paid 100.233% of listing price for a difference of .904%, for an average savings of $4,474;
  • The total Redfin Advantage, combining the negotiating advantage and an average Washington commission refund of 1.952%, was 2.856%. The total savings for an average Washington customer was $14,134;
  • 170 customers in King County bought a home through Redfin, 200 across Washington state;

The Seattle Times bought it, as did Inman News to a lesser degree. The Future of Real Estate Marketing weighed in, as did TransparentRE. There was coverage at Inman Blog and the Redfin cheerleader delegation was represented by Matt Goyer.

When Redfin’s post appeared this morning, I passed it along to Marlow Harris of 360 Digest for comment — she being a mathematically-adept agent with access to the NWMLS database. She sent back a PDF file from NWMLS showing numbers radically different from those claimed by Redfin. These were the figures cited in the Inman coverage. (More from Marlow.)

The first discouraging word was issued by Freakonomics Blog:

Of course, it is hard to know exactly what to make of these results. Redfin would like you to believe it is that they have crack negotiators doing the bargaining. Another possibility is that the home buyer that seeks out a discount broker is cheaper and more savvy than the typical home buyer to begin with. It is possible (although there is no direct evidence to support such a claim) that these homebuyers could have made even better deals if they had worked with a full-service agent.

It is not easy doing data-driven research that is completely convincing. This research is not completely convincing. I do think it is sensible and interesting, at least, so hats off to Redfin.

Although the data here do not show it, one of the legitimate fears a client should have when using a discount broker is that lower fees translate into even weaker incentives to do well for the client. This is why I think that the real-estate market is more likely to move towards a flat-fee system (where a fee is paid to list houses and then agent compensation is based on an hourly wage as opposed to a share of the sale price), rather than to a discount system which is still tied to sales price.

Glenn Kelman offers his demurrers to these concerns in the comments to this post.

With all that as introduction, let’s go through some reasons why’s claims to superior negotiating skills might be called into question by thoughtful Jesuits — and those who aspire to be thoughtful Jesuits:

  • Redfin agents may not be negotiating at all. The business model is based on the buyer driving the process. The lowball negotiators may be the buyers, not their Redfin agents.
  • Lowball offers tend to fail far more often than market-price offers. Redfin is measuring only closed transactions, not all those that might have failed due to failed lowball offers — and which might not have failed in the hands of better agents.
  • A salaried Redfin agent will have less incentive to keep difficult transactions alive, again skewing the results of closed transactions.
  • Redfin’s actual claim is that it “beat the field” in gambling terms. That is, its performance was better than average, which average would have been weighed down by the worst performers. “We’re better than the worst!” is not a persuasive boast.
  • By comparing itself to the field, Redfin avoids two more apposite comparisons: Against other discounting buyer’s agents (if there are any in Seattle) and against very experienced or very successful agents (which may not be possible, since experienced and successful agents tend to be listers, not buyer’s agents).
  • As Freakonomics notes, that Redfin might have done well with its clients does not imply that other agents might not have done even better.
  • The subset of clients attracted to Redfin may be so delighted by the idea of lowballing that no other consideration — e.g., getting the perfect house rather than the perfect price — compares in importance.
  • To my knowledge, there is no provision made for more complex terms and conditions. A complicated negotiation might result in a higher reported sales price but a better overall deal for the buyers — for example, repairs or even remodeling, seller concessions, no out-of-pocket costs, etc. These are the types of arrangements more likely to be made by more-experienced agents.
  • The data deployed by runs from 02/06/05 to 02/05/07. In that span of time, was ramping up production, so it is reasonable to surmise that its claimed production of 170 transactions was back-loaded into the latter half of the year. Unlike other markets, to the extent that Seattle has experienced a housing slowdown, it started much later, perhaps as late as July or August 2006. In other words, Redfin is very probably comparing its results in six buyer’s market months against what was for the market as a whole still largely a seller’s market.
  • In King County in 2006, 27,834 homes sold. Of those, something less than 170 were sold to buyers. Whatever claims Redfin might want to make from its results, at 0.6% of the Seattle market, Redfin’s results are statistical noise.

A controlled apples to apples comparison might yield more information. What Redfin has so far is a press release — one which does not bear up to thoughtful scrutiny…

Technorati Tags: , , ,


24 Comments so far

  1. Brian Brady February 26th, 2007 8:50 pm

    Some initial thoughts:

    1- Glenn did a nice job in his comments playing to the premise outlined in Freakonomics (Real estate agents and the KKK) when he said the buyer’s agent has an incentive to overpay.

    2- MLS data are easy to manipulate for one’s own gain. Did anyone throw out the dual agency transactions where the agent/broker conceded part of the brokerage fee as a buyer credit to net out a higher price?

    3- The real argument should be about missed opportunities. The Redfin buyer may be tripping over dollars to grab the nickels. Woe betide the niggardly negotiator in Manhattan last summer. Focus on the 2% differential would have cost them dearly this winter.

    Jesuits have no corner on the logic market, Greg. They’ve just promoted their expertise loudly and frequently…kind of like Glenn’s trying to do.

  2. Glenn Kelman February 26th, 2007 9:06 pm

    Greg, thank you for this very thoughtful analysis. It’s very well-written and carefully reasoned.

    We agree that a Redfin agent is more likely to counsel clients to walk from a bad deal and also more likely to go to the trouble of putting in a hopeful offer with a low chance of success, but this is also what any real estate agent would be more likely to do if you were his friend rather than his client. It is a major reason our clients trust our agents, and why our agents get better prices on the houses their clients ultimately buy.

    We agree with you and Freakonomics that one problem with the study is self-selection, with Redfin tending to appeal to price-sensitive customers. This is an issue we acknowledged in our report. It is a virtuous cycle, in which we attract the most aggressive customers, who get the best deal, and help us in turn attract the most aggressive customers. This is one reason that next year’s data will hopefully be better (but margins may be worse).

    And yes, we worried about our business ramping in the back half of the year. We originally were going to exclude January 2007 data so we could publish the report on February 6, 2007 (January 2007 data would not have been fully entered into the MLS by February 6), our anniversary, but the Seattle Times delayed publication of its story, giving us more time to collect the last month of data. This actually increased our advantage.

    Saying we’re better than the average agent is not the same as saying we’re better than the worst agent. The worst and the best are included in the average and Redfin is better than the average. Average and worst are two completely different concepts.

    The idea that our agents don’t negotiate the price or the more complex terms is not based in reality. This is our main function.

    Dismissing the data as statistical noise is not, according to our own calculations, mathematically valid. Redfin’s erstwhile statistician is going to post an analysis tomorrow on how statisticians assess whether results are statistically significant.

    So we agree with you on some points, disagree with you on others.

  3. […] Greg Swann,Thinking skeptically to rain on’s parade . . . Go team! […]

  4. Brian Brady February 26th, 2007 9:16 pm

    >We agree that a Redfin agent is more likely to counsel clients to walk from a bad deal and also more likely to go to the trouble of putting in a hopeful offer with a low chance of success, but this is also what any real estate agent would be more likely to do if you were his friend rather than his client.

    Glenn, what, if anything, would your agents counsel in a rising market (like Manhattan)? Can you see how the “virtuous cycle” could be perhaps…counterproductive to the buyer’s best interest?

  5. John K February 26th, 2007 9:32 pm

    Greg, did you get my comments about Bloodhound’s much ballywhood “flat fee” scheme and how it compares to Redfin’s discounted fee model?

    I can’t seem to find it, anywhere. Maybe it got lost in your spam folder?

  6. Cari McGee February 26th, 2007 9:35 pm

    Greg pointed out that…[In King County in 2006, 27,834 homes sold. Of those, something less than 170 were sold to buyers. Whatever claims Redfin might want to make from its results, at 0.6% of the Seattle market, Redfin’s results are statistical noise…]
    Too true! The Tri-Cities, WA’s average DOM is 120 days. My first three listings sold in an average of 35 days. Did I proclaim myself a superior listing agent? No! Because how I did in that infinitesimal number of the transactions in my market is completely irrelevant. For Redfin to extrapolate that their system is superior due to a tiny number of theoretical positive results is truly asinine.

  7. Greg Swann February 26th, 2007 9:43 pm

    > So we agree with you on some points, disagree with you on others.

    I’ll make you a deal. Send me PDF scans of the 170 files. I’ll make a server avaiable for FTP, and y’all can redact for personal details. I can reconstruct a transaction from the file, so I can vet the quality of the work in full, not just as regards price. For example, I can see how complicated the deals are, and how much Redfin’s buyer’s agents are bringing to the transaction. I’ll report my findings in detail, and you can get your incredible PR machine to promote them far and wide. What could be more fair?

    I truly do not believe you can deliver the service you promise and promote, and I do not believe that ordinary buyers can adequately evaluate the quality of work you are doing, but I am completely prepared to be proved wrong — and I am never shy about admitting it when I am.

    Are you game? Either way, I thank you for your thoughtful remarks.

  8. Greg Swann February 26th, 2007 9:47 pm

    > Greg, did you get my comments about Bloodhound’s much ballywhood “flat fee” scheme and how it compares to Redfin’s discounted fee model?

    Apparently not.

    > I can’t seem to find it, anywhere. Maybe it got lost in your spam folder?

    I don’t have an email on it, in any case, which I would have if it posted or got caught (like this comment from you) in the moderation queue.

  9. Marlow February 26th, 2007 9:52 pm

    The Seattle Times printed Redfin’s “press release” almost word-for-word, without any fact-checking. I don’t think it’s a good idea to just print a press release as “news”. Notice that, instead of using easy-to-check facts, Redfin’s “white paper” describes an excruciatingly difficult method of data extraction that would take a statistician many hours to extrude and analyze. And only licensed brokers have access to that data base, so the media can’t check Redfin’s claims, even if so-inclined. Instead of just using simple year-end data Jan-Dec, they purposely used a Feb-to-Feb time period, most likely to their benefit.

    If it is impossible to duplicate or check these numbers, I think there is good reason to discount them.

  10. Elizabeth Weintraub February 26th, 2007 10:02 pm

    Only excessively lazy reporters would ever in a million years dream of reprinting a press release verbatim. No, wait, exceedingly stoopid ones would do so also. But I don’t think any of those got high marks in journalism school.

    Actually, one can make numbers say anything one chooses. I can play with units, sales prices, closed by buyers, closed by sellers, closed by all parties, closed within 30 days, closed within 90 days, or closed within 6 months, and then begin sorting by brokers and agents until those numbers present the picture I prefer. So can any marketer.

    One hopes that when it comes to the press, a reputable reporter would refute and analyze those numbers, but the truth is most don’t know have a clue as to how to decipher them. Sad, I know.

    P. S. Besides being a Realtor, I come from a line of hard-hitting, investigative reporters.

  11. Roberta Murphy February 26th, 2007 10:35 pm

    Bloodhound Blog: 5
    Seattle Times: 0

    One more reason newspapers are doomed.

    cc: San Diego Union Tribune

  12. […] Greg gives a thorough round-up (as usual) with more than a dash of opinion thrown in […]

  13. Seattle Real Estate News February 27th, 2007 2:15 pm

    Redfin’s Latest…

    Redfin issued a press release yesterday and I’m sure anyone reads the Seattle Times saw the big front page article in the Real Estate section. Honestly, when I read the article the first thing I thought of was: “I wonder…

  14. FSBOLawCenter Blog February 27th, 2007 2:57 pm

    Is Redfin really all THAT?

    In the last couple of days, there’s been a lot of discussion in the blogosphere in regards to a recent announcement by Redfin, a discount brokerage based in Seattle (but operating in LA and SF, I believe, with plans to further expand nationally). The…

  15. […] My first impression was,! That’s definitely interesting. However, never having been a person to just take someone’s word blindly, I did my own number crunching. I originally posted these findings over on Rain City Guide earlier today. More discussions on this are on 360Digest, Bloodhoundblog, Freakonomics, and another one on Rain City Guide. […]

  16. […] Greg Swann (Accused Real Estate Flunky) Thinking skeptically to rain on’s parade . . . […]

  17. Eric March 25th, 2007 8:59 pm

    You sure spend a lot of time rebutting “statistical noise” here.

  18. […] Negotiation and the writing of an acceptable, binding contract. Redfin’s claim to the contrary, good agents know what will work, what won’t work, and how to make what won’t work, work. Contingencies are balanced against price, motivations are weighed, the contract is written, then the agent will present the offer to the seller in person. If necessary, there can be lengthy negotiations through counter-offers. The result is not only better contract terms for the buyer — or the fact that there’s a contract at all if there are competing buyers — but, done right, there’s a sense of cooperation and a smoother transaction to close. […]

  19. Eric August 25th, 2007 9:03 am

    Greg, at the end, your first 2 bullet points are spot on with my thinking.

    I still can’t help but think that any “success” is merely coming from providing and supporting a FSBO mentality of “I can do it myself” with just enough representation to feel that they’re not doing it themselves

  20. […] Publishing MLS data that shows that Redfin got a better deal for buyers than agents at other brokerages sparked a riot yesterday: here, here, here, here, here, here, here, here, here, here and here. We also showed up in Freakonomics (holy cow!). There has been a healthy discussion about how to interpret the data: whether Redfin agents negotiate better, for example, or its customers tend to seek better deals. We think both factors contribute to our success, and we love the debate. […]

  21. […] the kind of rah-rah-for-us stuff I leave for the vendor cheerleaders and the mainstream media. But: I gave Redfin a lot of grief last year when they made a similar announcement, so today I’ll give them a bit of their own back: Online real estate broker Redfin […]

  22. […] with data from our site. Last year we tried to give all the credit to our agents, but now, we think Greg Swann was right: a lot of credit goes to our customers […]

  23. Frank Borges LLosa- April 27th, 2008 8:47 am

    For what it is worth, I had a listing recently and Redfin submitted an offer. I have to say it was the most aesthetically pleasing offer of the 5 that I received.

    Redfin lost the bid by $15,000.

    What their stats don’t show is deals that they lose. But again, I attribute that to the client type and not to Redfin. Even their website offering system asks the client why they are putting in a lower offer. Normally the agent gathers the data first, and the price is discussed. Maybe this will be another premium service?

    But again, I have to commend their agent was very sharp, responsive digitally inclined (text, emailed contracts etc), unlike many agents in our area.

    After he lost, I did say “Did you lose because corporate has a quota of sorts that they don’t want to mess up?”

    Frank Borges LL0SA-
    Owner The Wiki MLS