There’s no substitute for a substitute

Eric Fischer, after heroically reconstructing San Francisco housing data for much of the 20th Century, published an analysis of the determinants of median rents. The hat tip goes to Tyler Cowen, who concludes “basically SF is ****ed.” Less pithy commentators did what less pithy commentators usually do, and used the analysis to claim that it basically supports their preconceptions and what they have been saying all along. It was only March when Hamilton Nolan helpfully concluded, “Build some housing, assholes. A lot!”.

Well, far be it from me to buck the trend. The analysis supports my preconceptions and what I’ve been saying all along.


Basically, Fischer estimates a model that puts plausible magnitudes on the price effect of new housing supply. How much new housing would we actually have to build in San Francisco to address the housing affordability problem? The model is certainly contestable, but at least it gives us plausible magnitudes to talk about. To stabilize real rents at their current, absurdly unaffordable level, Fischer finds that the number of housing units would have to increase in real terms by 1.5% per year, holding other factors constant. [*] So, is it plausible that San Francisco could build its way out of its housing crisis? As Fischer notes, that would imply a unit growth rate more than 3 times the average rate since 1975. Hamilton Nolan of build-some-housing-assholes fame concludes that

Those fortunate enough to have nice places to live in San Francisco (and the rest of the Bay Area) have had decades to get this right. And they haven’t. Drastic measures are in order.

Decades! Hamilton, you are more right than you know. The last time San Francisco achieved a unit growth rate of 1.5% was in… 1941. So many decades of NIMBYs! What was really different about 1941 compared to now? No exclusionary zoning regs? No pesky environmentalists? Maybe! But perhaps a more parsimonious explanation is the one that Fischer himself gives.

From 1935 to 1943, the Central Sunset and Parkmerced filled in. From 1944 to 1954, the Outer Sunset and Ocean View were built. And that was essentially the end of the easily developed greenfield housing.

In the history of San Francisco, 1.5% unit growth has never been achieved via “infill” development of an already occupied peninsula. There was a brief boom that managed a few years of 1% to 1.3% growth in the mid-1960s, but during that period developers “still could fill in the hillsides of Twin Peaks and above O’Shaughnessy Boulevard.” After that, the vacant land was gone. From 1967 to the present, the city managed a growth rate of more than 0.5% in barely one third of years (17 years out of the 48 from 1967 through 2014, when the data ends.) During the post-greenfield, post-1967 period, which years were best? Was it during the good old days before “homevoters” shut down the “growth machine” with their exclusionary zoning laws? No. The best years of the post-1967 period in terms of unit growth were 2008-2009, and then again in 2014 (the most recent year in Fischer’s data). No year exceeded 1% unit growth. But 2009 and 2014 did achieve a remarkable 0.9%.

Conventional wisdom has coalesced on the notion that it is NIMBY-ism and exclusionary zoning that are responsible for the crazy, crazy housing prices in San Francisco and other high rent cities, and so the solution to the problem must be a bloody, painful battle to overcome greedy incumbents’ attachments to their homes and neighborhoods. But before we destabilize neighborhoods and displace humans in the name of housing supply, we might want to ask, will all that pain really address the problem? Sure, at the margin, more construction will yield lower prices. And I understand that, following construction boom years, rental prices have stabilized in cities like DC and Chicago.

But within developed cities, construction booms are short and finite. Chemotherapy may be worth the nausea and hairloss if it adds years to ones life, but would it be worth it for an extra week? Infill densification is socially painful and physically expensive in terms of demolition and retrofitting infrastructure. And, yes, buying off the evil NIMBY’s and the permitting authorities who serve them adds to those costs. But how many examples are there of cities that have grown their housing stock in place at anything like the rate that would be required to meet the burgeoning demand in San Francisco or New York? Before we wage war on ourselves, maybe we should inquire whether victory is plausibly achievable. And, if it isn’t, maybe we should come up with a different plan?

The situation is even worse than it appears. The current craze, the only hope on the political horizon, is “affordable density“, which would eliminate regulatory impediments to construction for developers who reserve a percentage of units for means-tested tenants who would pay below-market rent. This trend reflects some mix of a well-intentioned attempt to address displacement and pragmatic acknowledgment that a city that has priced out its teachers and service workers might have a hard time functioning. However, an awkward fact of market pricing is that effective supply is not the number of units built, but the number of units actually made available to the market. Compared to a laissez-faire counterfactual in which redevelopment simply pushes poorer residents out of the city (often the case historically), every human not displaced diminishes the degree to which densification reduces market-rate rents. A market is not a dinner party. The higher the affordable housing requirements, the greater the rate of unit growth required to stabilize market-rate prices. Have we mentioned already that infill unit growth is really hard?

Cities evolve. They grow, they change, they do become more dense. And that is great. San Francisco NIMBYs are always on about Hong-Kong-ization or Manhattanization of the city. I like Manhattan and Hong Kong, and would be excited to see San Francisco’s aesthetically blah, largely single-family-detached west side turn fun like that.

But those neighborhoods are already inhabited. People live in the single family homes. They plant gardens in the generous backyards. In time, I hope those neighborhoods will change, and become more dense. I’m for a whole lot of redistribution, but there are reasons why civilized countries redistribute via financial tax-and-transfer rather than “land reform“, i.e. direct reallocation of real resources. To people who characterize homeowners’ informal sovereignty over their neighborhoods as a subsidy to the “upper middle class” at the expense of the “economically vulnerable”, I’d ask a few simple questions.

  • In a country where the homeownership rate is more than 63%, is it right to characterize homeowners broadly, even in San Francisco, as “upper-middle class”? Many homeowners have lived in their homes for years, and many new homeowners are mortgaged to the hilt.

  • Given that even an ahistorical, sustained trebling of unit growth would probably only stabilize, not reduce, the real price of housing in San Francisco, is it fair to characterize the people who would be helped by increased space for new residents as the “economically vulnerable”?

  • And given that, for perfectly understandable reasons, homeowners and residents resist fast-paced densification of their neighborhoods, which homeowners and residents would most likely be forced to tolerate changes they dislike or that threaten the value of properties? San Francisco has its share of stunningly beautiful neighborhoods affordable only to plutocrats. Will we put high-rises in those neighborhoods? Or, in the anodyne language of economists for every bad thing, will it be the economically vulnerable who must “adjust”?

Is Tyler Cowen right, then? “Basically SF is ****ed?” No. San Francisco could be just fine. The thing about San Francisco is that while greenfields have been exhausted in the city, the San Francisco Bay Area is largely undeveloped. We are always arguing over San Francisco, or Palo Alto (ick). Outside of the 47 square miles of San Francisco proper are almost 3200 square miles in San Mateo, Santa Clara, Alameda, and Contra Costa counties. (I’ll leave out the hoity-toity North Bay counties — Marin, Sonoma, Napa — but if it’s Latin-America-style land reform you want, the vines there are ripe for revolution!) Nobody wants new suburban sprawl, thank goodness. But dense development is not sprawl, even when it is greenfield development. When I argue that Singapore is an example we look should look to, people think I’m trying to make some left-wing point about public housing. I’m not. I don’t actually care very much about that. What excites me about Singapore is this:


It’s silly to characterize what Singapore does as “basically greenfield suburban growth“. Singapore’s new towns are denser than any US city, and nicer than most of them. They are designed for their density, not retrofit. What distinguishes Singapore is a “can do”, dirigiste approach to developing new living space, and a remarkable competence at making density green and livable. Singapore is an exuberant site of architectural experimentation, in both private and public building projects. Singapore’s “new towns” can house 100,000 people in less than 5 square miles. In the San Francisco Bay Area, there is plenty of space for Singapore-style new towns. Even Back East, room could be found for these compact conurbations.

Every piece of Earth has its stakeholders. As with densification of existing cities, plans to build new cities and their supporting infrastructure will provoke bitter controversy. But stakeholders for exurban land are fewer and more dispersed, and so less intensely affected, than city dwellers. The fights will be more winnable and the victories more meaningful in terms of the numbers of new dwellings that can be constructed. In the old city, we are condemned to bitter struggle over what ultimately may be too little to matter.

There are glimmers of hope for new towns, even in the US. See The Economist on “ersatz urbanism” in Florida. But it is in the San Francisco Bay Area, with its dreadful, painful housing situation, and its science fiction tycoons (several of whom individually could provide the necessary finance) where a full-scale, ecotechnological US microcity should really be attempted.


Personal Epilogue: I live in San Francisco. I am a renter. I live in a neighborhood with no pretty Victorians, on a block with little anyone would want to preserve for character. My personal preference would be for a lot more density, in my own neighborhood and many others. My apartment building, like the vast majority of SF multi-family dwellings, is rent-stabilized. I support San-Francisco-style rent stabilization, under which initial rents are “market rate” but rent increases are regulated. I think it is a good policy regime. I don’t think the $300-ish per month I may be saving (after a three year tenure, figuring two 10% rent increases) much impacts my views, but “none of us can be perfectly objective arbiters of our own conflicts of interest“, so you be the judge. I detest San Francisco’s high housing (and other) prices, and feel that the cost makes the city culturally gross, a place full of insecure people (very much including me) tacitly or not so tacitly competing in crude financial terms for the right to live here. Conversations always turn to housing, in overt commiserations or covert attempts to feel out where the other person lives, whether they have space, how they can afford it. As I said, it is gross.

Nevertheless, I don’t support the broad-brush anti-NIMBY, anti-zoning, affordability-through-density movement, because I think it is counterproductive. I want more housing, more density, and more development, but I don’t imagine those can possibly address affordability even over the medium term. I think that attempts to supercharge in-place densification i) will not succeed at any remotely sufficient scale; and ii) will cause real harm, and not just or mostly to unsympathetic upper-middle-class baby boomers protecting price-appreciation windfalls. I am very fond, intellectually and personally, of people like Matt Yglesias and Ryan Avent who have made strong cases for upzoning and densification as the way to go. They deserve congratulations for how persuasive their arguments have been. At least within my socioeconomic milieu, their ideas have become enshrined as conventional wisdom. But in my opinion — which of course might be mistaken! — these ideas have supplanted consideration of more effective and less difficult solutions to the very real problems they mean to address.

[*] I’ve tried to reproduce Fischer’s work, see the embedded image above (also Mathematica source code). My version yields slightly different coefficients and requires only 1.3% growth to stabilize prices. Hooray!

22 Responses to “There’s no substitute for a substitute”

  1. DavidC writes:

    I haven’t looked in enough detail to be sure, but it sounds like the Governor’s proposal would help a lot with doing what you suggest in the less developed Bay Area cities, right?

  2. DavidM writes:

    Not relevant to your main points, but some of the density figures in the density vs. livability scatter plot look quite bizarre to me. New York and LA the same density? NY 1/5 of London and Paris 1/7th of London? Moscow 2x London, 10x New York? Lagos 20x Nairobi?

  3. Tanner writes:

    Same feedback as David M. — checked the comments to see if anyone else had noticed this.

  4. A writes:

    If you only look at historical unit growth rates, then you ignore population and demographic trends. With such a focused view, the inference about construction “booms” is extremely weak. Or it relies upon very weak supply responses, regardless of policy environments, to changes in demand.

  5. Eamon writes:

    Thanks so much for this much-needed analysis. I too find the NIMBY thing to be a total distraction. Appreciate your point that the neighborhoods that will bear the brunt of the changes caused by densification within SF, whether it is the key to making the city affordable or not, won’t be Pac Heights and Forest Hill but rather HP, Vis Valley, Oceanview etc. I took a stab at writing about this and would appreciate any comments on what I got wrong or right especially with my attempts at grappling with the economics (as a non-economist).

  6. JohnJ writes:

    I agree that SF will not build its way out of its affordability crisis, but that’s not the right definition of victory. Think of Raj Chetty’s recent research. SF is an uncommonly high-opportunity area, and I would argue that every additional person is a victory. The opportunity aspects of this issue should be weighted appropriately against the neighborhood character arguments you make.

    The nature of that opportunity, deriving as it does from agglomeration effects, makes your “just build in Marin” argument less appealing (though I agree there should also be more housing in Marin).

    I’d also take issue with your contention that a building rampage would be harder on the economically vulnerable than not building. We can argue about the means, but in general it’s easier to provide for everyone (including the economically vulnerable) in a place that’s growing. The current state is a war of all against all for an extremely limited resource. It’s not as if the political will to protect the economically vulnerable (and the laws and regulations that emanate from that) will disappear entirely in an era of higher growth.

  7. DavidM2 writes:

    Yes, LA is almost as dense as NY, at least in the urban, inhabited areas. Many people don’t realize how dense LA is, it was urbanized pre-WWII and based on streetcars for getting around. I don’t think that is including suburbs, though. Census Bureau called LA the most dense, but it’s a matter of definition after all.

  8. I actually like your chemotherapy metaphor, because the point of chemo is to kill cancer. Hurting NIMBYs isn’t an unfortunate side effect, it’s the whole point. If the bay area had the kind of responsible, effective Singapore-like governments that was capable of the kind of development you’re pining for then we wouldn’t have this problem and we wouldn’t be having this conversation. We don’t, and it’s way too late for that now. Now it’s war, and anything that hurts NIMBYs is good for the rest of us. The bay isn’t a community, it’s two groups of people who hate each other living in the same space.

  9. ugh. sorry that was way too nasty.

  10. Devin writes:

    When you say “It’s silly to characterize what Singapore does as ‘basically greenfield suburban growth'” it makes me think you missed the point: where *precisely* should this new city go? Menlo Park? Atherton? Palo Alto? What do you have to say to the homeowners in that city?

    And if you’re thinking about true greenfield, you’re either going out past Tracy or you’re talking about fighting the environmentalist groups of whatever suburb (and probably the whole bay) you pick to put it next to. I’m not sure how you conclude that upzoning is the unrealistic choice here!

    In the meantime, Denver in 2014 permitted at a rate of 2% growth annually; Seattle at 2.4%. 2015 was even higher! Obviously neither hits those marks every year, but if you hit 2+ in the good years then you don’t need to hit 1.5 in the bad ones. And frankly, on a block of 20 single family homes, you’ve got to turn one of them into a four-unit building once per *decade* to hit 1.4% growth. This isn’t seizing the land of the nobility or whatever kinda odd metaphor for upzoning that somehow involves property theft you would like to use. It’s the gradual movement of a suburb into a city, and a small city into a larger one.

  11. Mercury writes:

    “I detest San Francisco’s high housing (and other) prices, and feel that the cost makes the city culturally gross, a place full of insecure people (very much including me) tacitly or not so tacitly competing in crude financial terms for the right to live here.”


    Yeah but at least they’re concentrated in an area where the rest of us don’t live.

    Clearly the real heroes here are the multi-decade owner/occupiers of San Fran housing stock. Why do we never hear triumphant tales about these people? I’m sure that over the period discussed, many family fortunes have been made, poor immigrants have become wealthy and savvy investors who saw $$$ potential when everybody else saw only dirty hippies on Haight St. That’s my glass-half-full take and the flip side to the tale of woe sung by the abovementioned unlovables.

    Anyway, nothing a decent earthquake can’t fix.

    My advice: stay out of state, keep your powder dry and wait for The Big One to hit.

  12. Lord writes:

    The story here .. Buy! Buy now before you will never afford it! It probably isn’t that great a surprise the rate of return is roughly an after tax stock return.

    I like the idea of new development with a minimum rather than maximum density. It could really change things.

  13. Bryan Willman writes:

    Just why is it that poorish people who cannot move to the city are more important than middle class and poorish people who have already been paying taxes there?

    Why does noone ever think “the solution to this is to force the largest employers to take the lead value jobs and distribute them much more diffusely throughout the US”

    That alone ought to end density hell in SF, Seattle, etc.

    (By the way, for the harsh suggestions above – if you zap all the single family house dwellers in SF, and they mostly respond by moving really far away, do you have a city left? It seems to not occur to some that the poor and middle are coming to BE NEAR THE WELL OFF and if you chase the well off away you have screwed youself.)

  14. Lord writes:

    (Leverage makes this much more attractive than stocks but leverage is only as good as your income to pay it off, typically by retirement or 30 years anyway. An earthquake is just a buy/build opportunity since nearly all the value is in the land/location.)

  15. traininthedistance writes:

    Couple not-so-quick things:

    1) LA shows up as denser than NYC because its arid climate cannot support the truly large-lot fringe that Eastern metros have; in other words its exurbs are relatively dense whereas places like Sussex County counteract the benefits of Manhattan. If you use tract-weighted density, which better measures the experience of the typical resident, NYC reclaims its rightful place as by far the densest American city. LA is still good as far as US cities go, certainly its suburbs have better “bones” for transit and densification than cities back East and that’s worth celebrating, but let’s not read too much into what is ultimately a misleading measurement choice.

    The sort of abundant, concentrated housing stock you have in NYC’s core delivers massive benefits that can’t be captured even by “better” suburban growth, and are not entirely negated by the existence of a sprawly fringe.

    2) This article seems to turn on the fulcrum that “since it’s politically impossible to build enough housing to get rents down to normal livable levels, it’s not worth advocating for building more housing at all.” But that’s a false choice, and it’s particularly galling to see it come from an economist! More housing will make a difference on the margins; even if we can’t get things to where we should, we still have a moral obligation to stop digging the hole even deeper.

    3a) I’d like to echo Devin’s comments in two ways– both in that it’s not actually true that the fight for more housing is more winnable on the fringe, and that there are real historical and current counterexamples of us building enough housing to keep rents down. To the latter point I’d add pre-1961 New York, a place where detached houses were replaced by brownstones were replaced by apartments as a regular fact of life, and rents remained more or less reasonable.

    3b) And of course the Bay Area is actually quite thoroughly suburbanized already. The greenfields are hours away, or they’re mountains where you can’t/shouldn’t build. If you want to build housing stock in an area outside of San Francisco proper, but you don’t want to foster the environmental and social cataclysm of supercommuting sprawl (which, the distances are far enough that even Singapore-style towers will hardly help), then you have to take aim at the already-existing suburbs. I don’t care how greenwashed your suburban tracts are if you have to drive forty miles through low-density suburbs to get anywhere else. Now, let me be clear: I would *love* to take aim at them. Turning Atherton into a mini-Manhattan of six story tenements seems like a fantastic social leveling project to me. I think that city-dwellers who want development to go elsewhere would be much more productive if they took explicit aim at densifying the existing suburbs, if they had a specific Yes counteroffer rather than just a general yawp of No. And it would be a win in terms of fair housing law, too!

    4) Right, fair housing law. Which gets into the question of “neighborhood character,” character for whom? The motivations are not always the same among all people and geographies, but we all know that in the suburbs “neighborhood character” and restrictive zoning are all-but-explicitly tools used to keep poor people and minorities and (I cringe even typing this word) “transients” out. Maybe the motivations are purer where you live, but it’s still the case that young people and immigrants and the like are systematically excluded from the decision-making process. For example, take a “community meeting” that anti-growth retirees can go to and dominate the conversation: what about the shift worker who can’t take off to make her voice heard? What about the kid in school who’d like to stay in town but can’t find a place to rent; or the other kid who’s prevented from coming at all? Who gets to count as “community?” They should by all rights count as stakeholders too.

    Look, obviously people derive psychological and material benefit from excluding others. I make no claim that the pro-density position is universally popular, or that it is an easy political lift, or whatever. That doesn’t mean we ought to valorize it opposition to it as any sort of positive good just because that opposition exists.

    (Personal note: I live in an old, small, and relatively cheap co-op deep in Brooklyn. We would never live anywhere where car ownership is necessary. We were lucky to find what we did, and it’s my sense of justice that makes me want as many people my age and younger as possible to have the same option.)

  16. Cameron writes:

    First, thanks for the MMA notebook with data etc. I know it’s not your research, but I have a few questions.

    1. Why ignore the population? Clearly new housing units can’t be expected to do much if they simply increase in proportion to population. Why not adjust for that (i.e. new housing per new person)?
    2. Why not include household income, or the share of income spend on rent? After all, one suspects that the primary diver of rents is the ability of one household to outbid the other for a location.
    3. Isn’t the distribution of household locations important? Building more homes on the fringe will reduce median and average rents simply because there are more homes in inferior locations. You said as much about this relationship historically.
    4. Does allowing more dense buildings actually provide additional incentive to bring forward new housing construction? Or is it a freebie to landowner who will sit on their development option, just as many thousands of landowners currently sit on their option to develop.

    To make my point clear, the % increase in homes was greater than the % increase in population from 2000 to 2010. Data is here

    And if you take into account the transport costs of living in different locations, the rents aren’t as bad as they seem in many areas. See this link for a housing and transport cost index

    Home building at 3x the normal rate would soak up resources from elsewhere, meaning it is not clearly obvious this is a good thing. For example, I am familiar with the Australian data where 6% of GDP is on housing building. Could we really spend 18% of GDP on it instead, over a period of decades?

    Anyway here are some of my previous writings on this topic
    On standard urban economics models
    My lens to analyse the housing market
    Doing the math on housing supply

  17. S writes:

    Moscow is supposed to be 10x denser than New York? I admittedly haven’t been to Moscow but that chart looks like it’s suffering from some serious data quality issues.

    That aside, what excites me about that chart is that Paris is generally estimated to be 2x as dense as SF. So, you could build all of the housing Fischer talks about right in SF and still be meaningfully less dense than Paris. Hardly a terrible outcome.

  18. reason writes:

    Just a question about the CLC matrix. The placement of the axes seems pretty arbitrary. The argument for putting Hong Kong outside the high-density/highly liveable quadrant and Singapore in seems pretty weak. And the density figures strike me also in some ways odd. Are Tokyo or Vancouver – really low density (or do parts of them push up the average a lot). It occurs to me that there ways of measuring of HK that would make it lower density too.

  19. hhoran writes:

    People forget how huge the population density variations are within cities and metro areas.
    I recall Shoup’s discussion of density in “the High Cost of Free Parking” where he tried to counter LA vs NY preconceptions by explaining that central LA had the density of Los Angeles, but was surrounded by miles and miles of land that also had the density of Los Angeles. Midtown Manhattan, on the other hand had the density of Hong Kong, but was surrounded by miles and miles of land with the density of Dayton, Ohio.
    Manhattan’s pop/sqmi is 66k, but populated zipcodes within Manhattan vary from 126 to 23
    (Brooklyn averages 34, Queens 20)
    San Fran averages 16, but zipcodes range from 53 to 6.
    But the real problem is that any discussion of how to get population density to go higher than what San Fran is today have to start with transit infrastructure. You can’t fill the Sunset with tall apartment buildings unless transit goes in first. Singapore (and Taipei and other more livable big Asian cities) put the rail systems in first. New York is dense because rail lines were built in the Bronx and Queens when the surrounding land use was still mostly agricultural.
    Think political resistance from San Francisco homeowners is a big problem? The ones who have been given staggering wealth by the rest of society (thanks to zoning and many interrelated political decisions) and feel they can veto any policy change unless paid off for the full expected value of future appreciation? That’s the easy part. Wait till you try and get the funding (and planning permission) for all that transit infrastructure.

  20. Pepe writes:

    New housing development in places like SF tends to skew heavily upmarket largely because land prices are so high that new developments don’t pencil unless they are ultra-high-end. So even if the SF housing stock were to grow at a multiple of 1.5% CAGR, affordability may not necessarily improve as a result.

  21. wayne writes:

    I heard about san frans navigation center for the homeless. My question as a struggling student with a time or two outside is why isn’t something like that on the free rental market? Even homeless can afford $200-$300 per month. Rather than spending money to accommodate a small passing fraction of homeless they city could MAKE money off the issue. If ran with no strings attached like curfews, enforced chapel, or restricted access throughout the day, and if it had equivalent day room to dorm space then it would feasibly begin to attract struggling students and people between jobs or new to town. If these people are paying for this rather than paying an arm and a leg just to maintain their own privacy then that means there is something on the market which is capable of dropping rental prices. And no I don’t see shared room situations as equivalent to this because 2 people in a room is an awkward silence nobody wants to go for. But from experience 200-300 people in a large enclosure is much less personal (there are no awkward silences) and nobody owns it so feels less intrusive.

    It makes little economic and practical sense why things like this do not exist on the market. Space to sleep, sit, clean oneself, cook, etc. is essential, but privacy is not. I wonder how small and expensive units have to get before people get over it and realize it is not a big deal.

  22. wayne writes:

    and I don’t think supply and demand is a solution to housing costs. This is not the clothing market and developers do not want to surpass the point where supply meets demand so as to be required to compete with one another. They are instead very careful in analyzing the market and predicting whether they can fill the units or not.