When Zillow was brand new, Catherine Reagor, the dippy local real estate reporter for the Arizona Republic wrote:
Want a reality check? Go to zillow.com, a new Web site with a program that calculates a home’s value for free. It values several Valley homes for tens of thousands less than the price listed on them.
At the time, I said this in response:
If you want to know what your house is worth, do not go to Zillow.com, which delivers completely useless estimates of value for free. Even Net Value Central, a tool used by professionals, lags the market by a month or more. The only way to price a house is to work as rigorously as possible from current and recently-sold listings for extremely similar properties. If you price your house to sell from sources like zillow.com, you will give thousands of dollars away. If you rely on zillow.com to tell you how much to offer on a home, you will see it sold to someone else.
(You can prove all this to your own satisfaction, if you like. Most of Ms. Reagor’s mistakes seem to come from falling in love with ideas she doesn’t check out. Here she tells us that she ran Zillow.com on live listings and found it came in much lower than the listed prices. How did it do against sold listings? She didn’t check, but you can. Run Zillow.com on the sold homes documented in your local section of the Republic. You’ll see that, time after time, Zillow.com is substantially under real-life market results. It’s a useless toy, which Ms. Reagor might have discovered on her own had she bothered to test it properly.)
Just lately, I needled the Zillowites a little more:
All right, here’s the deal with Zillow.com:
I decide I’m going to buy you a pair of designer jeans, nothing but the best for you. I know that fit is important, so I go to three of your best friends to get their sizes. Not yours, theirs. I strike a happy medium amidst the diversity, reckoning that — what the heck! — you can’t be that different from your friends.
If the jeans I buy for you actually happen to fit, this will be a happy accident. More likely, the jeans will be a close but not perfect fit.
You understand why, of course. Epistemological error was built into my sizing algorithm. I chose a method that might have been convenient, but which cannot possibly produce objectively accurate results with any degree of confidence. Arguably, the more of your friends I measure, the smaller my margin of error. But I am still pursuing an inherently erroneous sizing methodology.
For Realtors, a perusal of the tax records, the equivalent of a Zillow Zestimate, is the first step in comping, the step known to be least accurate. The next step is comping the house one-for-one with recent past sales and currently-marketed (competitive) listings. The last step is working all those numbers against the subject property in its current state of upkeep and upgrades.
In the same way, if I don’t take a tape rule to your inseam, the chances of my getting jeans that fit your unique physique are very poor.
A very nice flak-catcher — everyone at Zillow.com seems to be very nice, and they are all over the Cluetrain idea — had this to say in reply:
I obviously can’t share any secret math but can let you know that you’ve only got one part of the process captured in this story – for example, you forgot to mention that you also took last pair of jeans I bought from my closet and recorded its size – and you did the same with all my friends who bought jeans around the same time . . . etc.
That’s cute, but it’s not responsive. The main charge stands: Zillow.com cannot do what it promises and purports to do, because it is simply not possible to price a home you have not seen. That the algorithm they’re using might be better than I envision it to be means nothing. That it might be successively revised through time to become even better means nothing. The entire methodology is based on a fundamental — and painfully obvious — epistemological error, and if particular results happen to be more or less consonant with objective truth, nevertheless, Zillow.com cannot ever achieve — nor even approach — objective truth.
Want proof? Let’s look at some houses.
These houses are all from an historic neighborhood in Central Phoenix called the F.Q. Story Historic District. This is the first true suburb of Phoenix, the first organized, subdivided stretching out from the original square mile of Downtown Phoenix. Story was a commuter suburb: Residents rode their horses to town, or took wagons, automobiles or streetcars. The homes were built mainly in the 1920s and 1930s, and they were all one-off, custom construction. Further north, you can see the beginnings of the techniques that became production, tract-home building, but in Story the homes were all built like custom-tailored gowns, each builder expressing his own ideas of what a home should look like. In consequence, these homes are highly prized and sell at a huge price premium over later-built tract-homes with comparable space and amenities.
To be fair to sellers and other Realtors, I’m citing either homes we listed or which are not currently for sale.
We sold this house in February of this year for $480,000. Zillow has it at $493,712, a reflection of the process my correspondent was talking about. If fact, this Zestimate is probably too high, given that nothing is selling in the neighborhood right now. Don’t worry, though. This is as close as we’re going to get to reality.
We have this house under contract right now. The list price is $495,000. Our price is based on a complete appraisal. Zestimate: $297,319 — only off by $200,000. Realtors on the ground can see the value that computers can’t. You can barely see the house for all the trees, but that is a huge bonus in scorching Phoenix. The house also has a huge sun room, which we excluded from the appraisal but which sells the house to everyone who sees it. It’s the ultimate family space, a vast retreat for playing foosball or watching TV. (Addendum: This house sold for $475,000 on 10/12/06.)
We have this home listed for $450,000, again backed up with an appraisal. This is two doors west of 1102 West Culver Street, but evidently the Zillow algorithm doesn’t lift all boats with a rising tide. This home is also larger, eight years older and has a second bathroom, all of which are worth money. On the down side, it lacks a pool and doesn’t have elaborate desert landscaping. We are very confident the price is reasonable. (Addendum: We reduced the price to $424,900 after I wrote this.) The Zestimate: $311,648.
This home is listed at $519,000. It’s a breath-taking restoration of a 1935 Moroccan Revival home with a sybaritic master suite and a spacious guest house. Zestimate: $377,483.
Now, to this point, you could say that this is all so much sour grapes. Zillow.com doesn’t like our listings. Boo hoo! I invite you to pay close attention to the next house, which is not currently for sale:
925 West Willetta Street
The brown stuff you see where the roof should be is planking. The gray stuff showing through the planking is sky. The roof has been off of this house for a long time. Until this spring, there was a tarp over the planking in back. That failed in a storm, and I can’t imagine how much water was dumped into the house. It rained hard last night, too, so, whatever the home was worth yesterday, it’s worth less today. But don’t think for a minute that it’s worthless: It’s Zestimated value is $337,725, more than 922 and 1110 West Culver Street.
But here is my favorite:
1142 West Culver Street
The bamboo fencing is so demure, providing that added little bit of privacy. Can you see through it? Let’s take a closer look:
1142 West Culver Street — close-up
When we pull away the veil, we reveal… Nothing. The house burned down about 18 months ago. Our friend Jerry Kadansky was the demolition contractor, but the neighbors pitched in by hauling away much of the undamaged material, including every last piece of what was once a very impressive slate roof. But where you see a vacant lot, Zillow.com sees the house it has absolutely no way of knowing is no longer there. Zestimated value: $325,787, again beating out 922 and 1110 West Culver Street.
This is shooting fish in a barrel, but that’s not the point. The point is that, whatever it is that Zillow.com might be doing, what it is not doing — what it cannot be doing — is evaluating houses. This simply cannot be done by the methodology Zillow.com has employed.
The real estate industry is beset by a sea of troubles, not the least of which is the reputation some Realtors have earned by telling lies to their clients. Zillow.com does not tell lies. It simply does not tell the truth. By the nature of its methods, flowing without opposition from its underlying epistemology, Zillow.com cannot possibly, cannot ever, cannot under any circumstances tell the truth.