Tuesday, August 10, 2021

health care

These two graphs really say it all:


A few years ago I got a freak knee infection that landed me in the hospital. The experience was so smooth, non-stressful, and NOT followed up by months of fighting insurance companies over insane charges I had no way of knowing or avoiding ahead of time, that I decided in that moment that we would stay in Australia long enough to get citizenship and thereby always have access to Australian medicare. Matt has twice needed an ambulance*, and we had a baby, and we've made countless same-day appointments for minor ailments, and each of those experiences has made my mind boggle at how well it works and how we ended up in this magical place almost by accident.

The pandemic has re-affirmed that many times over. Life has been basically normal since last July, the economy is healthy, and while certainly things are far from perfect, 600,000 lives in the USA alone would have been saved if they had achieved our death-per-capita rates, so I'm not complaining. More generally, of course there are problems, big problems in fact, with the system, but the gulf between the system here and the system I used to put up with in the US is so vast that there is no question whatsoever which is better. I wish so much that my friends and family could live with the same peace of mind. And that I could convince more Americans to come visit for a few months because that's how little time it would take for even the most stubborn among them to realize what they're missing out on.

Monday, September 28, 2020

JoEP's excellent editorial on standards and practices

I was so thrilled to read this editorial in the Journal of Economic Psychology. I think they've hit the right balance on every point except possibly 14, despite having to balance disparate traditions from psychology and economics. Here's the text distilled into a list:

  1. Quality replication studies are encouraged and JoEP has a section dedicated to them (which is slowly increasing in usage.)
  2. They have added a "brief reports" section to the journal "to speed up the pace of research by conveying short, concise points without the need to artificially inflate the discussion to meet some arbitrary length standard."
  3. Null results are welcomed, especially as replication/brief report papers, but need to be accompanied by properly conducted power analysis.
  4. Pre-registration will not be considered a pro for publication. The details and justification for this point are fantastic so you should read section 1.2 in its entirety.
  5. Multiple studies in a single paper are also not considered a pro for publication; scientific logic dictates how many studies are appropriate.
  6. Making data and relevant code available will be enforced going forward, except in exceptional circumstances.
  7. At the same time, "data free-riding" is discouraged.
  8. "Authors need to disclose any use of deception and the necessity for its usage will be evaluated."
  9. "In studies of economic decision making incentivization is the standard practice, and this is the standard we adhere to."
  10. While in an ideal world we would abolish significance categories altogether, for now p<.05 is a one star effect, p<.01 is two stars, and p<.001 is three stars. Given that p values are themselves random variables, three categories is about the right level of precision for reporting them.
  11. Effect sizes should be presented with the highest possible clarity, for example by standardizing regression coefficients.
  12. "Multiple comparison corrections or strategies of avoiding multiple comparisons should be used", noting that the necessity of multiple comparisons depends on the specificity of the hypothesis being tested.
  13. "We ask authors to state in writing that they have reported all implemented experimental conditions ... and disclosed all measured variables; as well as how their sample size was originally determined. We also plan to add additional guidelines concerning indiscriminate removal of participants from the sample."
  14. Submissions at JoEP are not anymore anonymous.

I hope that none of these are particularly controversial, even though standard practice most certainly does not reflect them and they are inconsistently enforced even when required. 

I've highlighted 4 and 13 because 4 is certainly the most controversial, and 13 is relevant to that controversy. In fact it's the key to the solution. Those who object to mandatory pre-registration complain that it places an undue burden on the researcher and restricts the natural exploratory research process. Those who favor pre-registration counter that the pre-registration doesn't bind you to do exactly what the documents lays out; it simply requires you to explain the deviation or update the pre-registration. And to that I say: if you honestly don't want to hamper the exploratory research process, then it should be sufficient to adhere to 13. If all designs tried and refined are described, if all measured variables are reported, etc, then a careful reviewer can easily detect fishy statistical practices including all of the practices mentioned in the editorial as motivating the pre-registration movement. And it's much easier to detect these practices if this information is concisely included in the paper itself rather than having to dig through a separate pre-registration document(s); I guarantee this digging simply won't happen. Proposals to require these kinds of statements have been around since long before the pre-registration craze and I wish there were more than a handful of referees and editors out there sporadically enforcing the practice.

I'll skip commenting on the rest and refer you to the editorial instead, since it does such a good job. Except, on point 14. It's not that I even disagree with the decision, given the reality of the situation. I'm just frustrated that this seems to be an important issue that journals have collectively thrown up their hands over due to the reality of the situation. There must be other improvements that can be made. E.g. authors should be able to optionally remove all identifying information from their submissions, temporarily remove drafts from the internet where possible, and reviewers can be asked to promise not to look up the authors or papers online until after their reviews are submitted; of course this is unenforceable but would go along way towards establishing a norm. This is only 30 seconds of brainstorming; surely the profession as a whole can come up with better solutions than all-out surrender.

And to those who insist that author identity and affiliation don't actually affect the results of peer review, I invite you to conduct the following study that would be able to actually provide RCT evidence: Authors from top institutions and/or with particular notoriety in their field partner with authors from lower-ranked institutions and/or lesser name recognition who work in the same field. Journals are asked for permission prior to the study to, at some point in the future, submit genuine research papers with only the author and institution falsified, on the condition that the author and institution will be corrected, regardless of outcome, when the paper is either rejected or accepted. Any papers written by involved authors who want to submit to one of the involved journals will be assessed by the authors themselves and perhaps by a third party(ies) as plausibly submitted by another author from the opposite category. If so, it is submitted randomly under one or the other name. In the case of multiple-author papers, the lead author would be the single name on the submission until the review process is concluded, at which point it will be corrected to include all others. Of course that's only the basic gist, but you get the idea.

This would be much more logistically feasible to arrange by a senior faculty with sway at a wide range of journals, so how about one of you put your money where your mouth is? :) Even if the study is failure due to reluctance from journals or from researchers, that's an interesting finding of mismatch between stated and revealed beliefs...

Thursday, July 30, 2020

economics is a disgrace

I can't (yet) bring myself to do the multi-tweet reply thing, so I will link to this instead.

If you are an academic economist and somehow haven't yet read Claudia Sahm's post, you are obligated to do so now.

I admit I have been dismissive of complaints about the culture in the past, and I want to explain why and how I have come to recognize the problems, in case it helps anyone else bridge the understanding gap as well.

Most complaints about the culture of economics are about treatment of women and minorities specifically. These accusations invite defensiveness, because they are easy (in isolation!) to re-interpret as bad behavior that has nothing to do with gender or race. I have reacted that way myself because I personally have not experienced anything I would call sexist, and have seen plenty of effort to help women and minorities. In the absence of real statistics about how advisors treat minorities versus non-minorities, about the disparity in feedback those groups receive (properly controlling for other factors of course!) it's easy to write off bad behavior as unrelated to identity. Perhaps I should be more trusting of personal accounts that are interpreted as racially/gender motivated, but I'm naturally skeptical. And I will probably get in trouble for saying this (maybe this will be the 1000th blog post I don't publish out of fear of getting in trouble for attempting to discuss something sensitive...) but I honestly don't think that skepticism is a bad thing. Better evidence would go a long way not just to convince me of the problem, but to understand the problem better and make effective changes. Skepticism that demands such evidence is, I believe, a good thing.

However, over the last year or two I've realized that I'm very oblivious to the effect of my gender on my interactions. I'm obviously a woman but that fact is entirely irrelevant to my identity; I don't notice when I'm the only woman in the room, I don't hear comments about women as applying to me, and it always catches me by surprise when my gender is explicitly referred to as a factor (usually 'we need you for this committee because there has to be at least one woman.') It never occurs to me that particular interactions might be influenced by my gender, so I don't experience them as sexist. But the deluge of accounts that have, by the participants, been interpreted as sexist, prompted me to reconsider my interpretation. That has changed my prior belief about the likelihood that accounts of bad behavior are, in fact, due to gender/race/etc.

More importantly, however, I also got a little bit outside of the economics bubble and saw just how terrible our culture is by comparison. Regardless of whether any of our cultural problems are solely or particularly problematic for women and minorities, our culture is nasty, overly competitive*, overly cliquish**, ludicrously hypnotized by the power of the Top 5 journals to cast the ultimate judgement of the quality of research (rather than the quality of research that happens to fall into the narrow fads that are currently deemed to represent 'general interest'), dismissive, elitist, rude, and negative.

My work lies at the border of social psychology and behavioral economics, and UQ is fortunate to have an amazing social psychology group right next door to the School of Economics. I therefore started attending, at least when the topic veered close enough to my interests, the social psychology group's weekly seminars. I can't even express how different they feel to an economics seminar. Instead of dourly entering the judgement room where we will listen to a presenter and attempt to score points by identifying faults in their research, their seminars are friendly, positive, constructive, understanding, encouraging, and just a huge breath of fresh air by comparison.

Economists go to work with an attitude problem. The toxic culture is what results. If it's blatantly obvious in seminars, it has to be much more detrimental in every other context***. It certainly disproportionately affects women and minorities, but regardless, it hurts everyone.

Lastly, I want to point out that while Claudia's post is excellent and brave, the full extent of the problem will not be recognized without more comprehensive data, and that will not be forthcoming as long as people in a position to report incidents fear retaliation. We need a way to anonymously share our accounts (experienced or witnessed). That isn't EJMR...


* Sure, competition that leads to better research is great. Competition for the sake of winning is not. A very well-respected economist at a "top 5 school" once told me that if anyone ever criticizes your work, the only way to respond is to immediately double down and go on the offensive. The sadder thing is that, within the culture, that's probably true.

** It's amazing how much easier it is to get an email reply from economists you don't know when your email signature is from a "top 5 school", as mine was in grad school, than when it's from a lesser-known institution. Sure, I understand signaling, but if your judgment of potential value of of interaction is based so much on the signal sent by institutional affiliation, you are being immorally lazy. Or if your judgment of potential value of interaction is based on "how likely is it that this person will be in a position of power over me in the future", well... if that isn't a sign of a culture problem, I don't know what is.

*** This is why our publication process is so godawful compared to other fields. Refereeing/editing should not be a game in which to score points or an avenue to promote personal interests!

Monday, March 9, 2020

so wrong, in every way

Bloomberg spent $500 million on his campaign. Multiple people on TV actually said or agreed with the statement that he could have instead given $1 million to every American. This is so unfathomably wrong, it's hard to describe. It's true that you don't need a calculator to figure out that it's wrong, but that's being generous - you barely need any common sense to know this is wrong. Here are some things that are as wrong (yes, I calculated):
  1. "I'm Native American, because my mom is Native American. Well not my mom, but my great-great-great-great-great-great-great-great-great-great-great-great-great-great-great-great-great-great-great-grandmother."
  2. "We live in Manhattan, so I told my son he has to stay on this block if he rides his tricycle. I found him in Argentina. After he had already triked around the entire planet 7 times."
  3. "This recipe called for 1 tsp of salt, but I accidentally put all of the salt for sale in all of the Wal-marts in Texas in instead."
  4. "I thought this was a dinosaur bone, but actually it's my grandpa's baby tooth."
  5. "I'm prepping for a 10-year quarantine in case coronavirus gets really bad. I've got all my food supplies ready: one almond should do the trick."
  6. "I told you to meet me on the stairs of the Met museum, and you thought Perth was close enough?"
  7. "The richest man in the world could give everyone on the planet $18. An average guy in Somalia could quit working for 24,000 years with that $18."
  8. "Bloomberg spent $500 million on his campaign. That's the entire GDP of the USA for 20 years." 

Now imagine... If a professional journalist can be this wrong, how accurate do you think your political facebook memes are?

Tuesday, March 5, 2019

blank slate

Sociobiology was published in 1975. The Blank Slate was published in 2002. And yet in 2018, this is still the outcome of a survey of 335 professional social psychologists:


Thursday, February 7, 2019

United Arab Emirates

My vacations nowadays are opportunistic, prompted either by overseas visitors who give us an excuse to see something new in Australia or by a conference that may or may not be in a destination I would choose on my own, but hey, I'll never not make the most of a free flight somewhere new. So that's how I ended up with a 6 day vacation in United Arab Emirates, after an excellent APESA conference at NYU Abu Dhabi. Miscellaneous notes:
  1. The basics: In case you're not already aware, UAE is very modern and cosmopolitan; you can wear whatever not-too-scanty clothing you want and go about your life as you would in any western country, no problem. Emiratis are a small minority of the population; the rest come from all over, which means English is the lingua franca. Almost everything is in both Arabic and English, and among what isn't, more of it is in only English than in only Arabic. What surprised me is that not only is it the modal language, everyone speaks at least some English. The only Arabic word I know is "al" ("the") and I had no trouble at all. It's a perfectly reasonable place to live, if you place less emphasis on climate than I do (most people do). My indifference curve would probably pass through Abu Dhabi and, say, Oslo.
  2. There are seven emirates that differ about as much as U.S. states from what I can tell; in one you can't buy alcohol, for example. These united to form the country in 1971. I don't know the prior history but the map indicates some complicated geopolitics; most emirates are in several pieces, and there are also two Omani enclaves in the mix, one of which has its own Emirati en-enclave.
    The mess of Emirates and Omani enclaves
  3. Abu Dhabi (and presumably other places, but this is where I wandered around downtown aimlessly) has a bizarre vibe. Because of the climate, it's quite pedestrian-unfriendly. The streets are empty of people before dark, the buildings look shut down from the outside since most have shades or reflective layers on all surfaces, and entrances are often hard to identify (I don't have an explanation for that one, but it was consistent). Add to that construction everywhere and I felt like I was trespassing in a deserted work site pretty much everywhere.
  4. If you're not into malls or ridiculously expensive theme parks, there's not much to do in Abu Dhabi. The mosque is the main attraction, and it is indeed impressive. Muslims really figured out art and architecture (obviously) and this particular example puts it on a grand scale. It has the largest hand-made carpet in the world, with over 2.5 billion knots. As an ignorant western tourist, I also appreciated the fact that it was constructed intentionally to be an Attraction - it's overtly welcoming of anyone regardless of nationality, race, etc (as emphasized repeatedly by the guides); they provide appropriate layers in case you come without sufficiently respectful attire (my standard-issue Grey Silicon Valley Start-up Hoodie was enough cover); they have hourly free tours in English; and they have clear signage and locations for taking pictures so you don't have to guess whether you're doing something accidentally offensive.
    Sheikh Zayed Grand Mosque
  5. Dubai, on the other hand, is a surreal alternate universe version of Las Vegas if Nevada were a Muslim country, right down to the desolate desert setting. The Dubai fountains, a giant choreographed-to-music attraction in front of the Burj Khalifa, was even designed by the same person who did the Bellagio fountains. Instead of casinos there are uber-fancy malls, and instead of prostitutes there are Victoria's Secrets with no softly-pornographic ads or any lingerie on display at all, but in either place you will be blinded by opulence and easily parted with your money.
  6. The one novelty I was willing to part with so much cash for was to visit the top of the Burj Khalifa. I was surprised to learn that not only is it the tallest building in the world, it's over 30% (almost 200 meters!) taller than 2nd place. Standing on the 125th floor skydeck it looks like there's still a normal skyscraper above you, and all the skyscrapers below you are babies. Plus, it's very cool design, definitely no regular box.
    Looking down and then up from the observation deck of the Burj Khalifa
  7. Getting out of the two major cities is when things get more interesting (if, like me, you're mostly interested in learning about local history/culture/nature). On the other hand, there's just not that much there... I recommend a desert safari to the empty quarter (most of the country is a never-ending empty sea of sand dunes, which are gorgeous and unlike any other desert I've been to, but you can experience it in a few hours), the Mleiha archeological site (which contains the oldest known human remains outside of Africa, from the late Paleolithic at least 130,000 years ago), and the traditional camel market in Al Ain (where you will be accosted by friendly men who want to show you around to take pictures and will then ask for money at the end, but that's par for the course, and it's worth it - just bring only a few dollars with you).
  8. I visited an assortment of historic sites in Ras al Khaimah, Fujairah, and Al Ain, which were all small and quick and at most $2 and I was usually the only one there. It's like what you would expect to see in an underdeveloped country that's taking a first stab at a tourism industry (they collectively reminded me of tourist attractions in Gabon or East Timor or Haiti, for example). I say this all descriptively rather than judgmentally. It was a fascinating contrast with the ultra-ritzy attractions in Dubai.
  9. Speaking now judgmentally in addition to descriptively, the outdoorsy attractions (except for the desert safari) were really not worth going out of your way for. I had some nice views and fun experiences but you can have just as much fun in a better location almost anywhere else. One interesting natural phenomenon is that there is so much dust blown into the air that the cities look like they're extremely polluted. It took me a few days to conclude for sure that it is dust rather than smog. The sun gradually fades behind the muck instead of setting, and even way out in the desert the night sky is underwhelming.
    The desert of the empty quarter; hundreds of miles of surprisingly colorful dunes
  10. Culturally, great emphasis is put on hospitality. All of the history museums mentioned hospitable customs such as offering the largest camel and fanciest bedroom to guests, and this clearly persists today, to such an extent that it feels downright oppressive to an extreme introvert like me. I ironically started avoiding going into businesses and keeping my phone out to credibly avoid eye contact to avoid unnecessary backs-and-forth of "no really, I'm fine, I don't need anything, I don't want anything, I know where I'm going". Yes, I realize that this approach is entirely contradictory to the goal of getting to know local culture.
  11. My research ahead of time made me pretty nervous about driving there, since drivers are described as FAST and they have one of the highest traffic fatality rates in the world. It honestly wasn't bad at all, pretty similar to the 5 between LA and SF. The exception is that people tailgate really scarily closely until you get out of their way. The exception in terms of really stupid design is that there are highways where the speed limit for cars is 140 and the speed limit for trucks is 80. There's nothing wrong with 140 on a desert straightaway (that's the same as the speed limit in west Texas on the 10, for example) but good god don't mix the two.
  12. Did you know that the black band that Emirati men wear to hold the ghutra (the white head covering) in place are also, traditionally, camel locks? You put one loop around a front leg and the other loop around a back leg, and they can't run away. Also, did you know that camels have clear third eyelids to wipe away sand, and can drink 200 liters of water in 3 minutes (more than one liter per second)! You can also breed a male Bactrian (two-humped) camel with a Dromedary (one-humped) and get a bigger, stronger, and faster camel with a single giant hump. Kinda like a mule of the Artiodactylae.
  13. I did have a few strange encounters with people that I still don't understand. In a fort that I was alone in, a guy came in and without saying anything indicated he wanted a selfie with me. And then 15 more selfies. And then an extra dozen with me holding the phone, for good measure. White tourists aren't that uncommon, even though this was in a smaller city, and he wasn't hitting on me, so what the heck. More nerve-wrackingly, the last day I was about to get in my car in a dark parking lot at 6am to drive up a mountain for sunrise. An SUV pulled up behind me, blocking me in, driven by two Emirati men. One rolled down the window and asked where I was going, I said I'm going to Abu Dhabi, he seemed to ask if I wanted him to come along, so I said no thanks and quickly got in my car and locked the doors just in case (it's a very safe country, for the record). I waited for them to unblock me, but instead he came to the window and kept up with the "you go Abu Dhabi? I go for you! What you need?" and other equally unintelligible phrases until I said "no thank you, bye" enough times for him to give up and let me out of my parking spot. Bizarre. If you can tell what this was about, let me know for future reference...
  14. Another funny but interesting incident happened in a museum in another fort. I was the only tourist there, among many workers doing some renovations, and I took some pictures in the courtyard and then went upstairs for a bit. About 15 minutes later an employee came up and asked if I was a regular visitor. Yes? "Why were you taking photos of the fire extinguisher?" I had no idea what he was talking about, so he led me to the extinguisher and I realized it was next to a windowsill where there had been an enormous grasshopper that I spent several minutes photographing. That would understandably look pretty suspicious if you didn't know what was there. I showed him the photos and he seemed quite relieved and then apologized profusely and made a show of telling his colleague about it as well, who also apologized and said I could photograph anything I wanted to, even them :)
  15. One last random story, from the airport on the way out of Abu Dhabi: If you travel anywhere touristy you've surely encountered the quintessential Chinese Tour Group, the organized tours for people who don't speak enough of the local language to be able to get around on their own, usually-but-not-always from China (you can call me racist if you can suggest a better name :). These groups are characterized by the practice of obnoxiously taking over an area without any regard for other visitors, by getting in the way of traffic, cutting in front of other traffic, taking an hour to take every possible selfie before moving on, ignoring signs asking for quiet or for no photographs, etc. When multiple CTMs collide they usually just kinda merge since they have aligned goals, but have you ever wondered what happens in a true collision? This is approximately what happened when I stepped into the airport bathroom and encountered a pack of around 15 Indian women all wearing identical abayas and identical hot pink "travel mart, Kerala India" headscarves and stashing their identical photo-personalized travel packs in the sinks while they took turns using the stalls. This seemed innocent enough at first, if annoying for sheer numbers. But the toilets in UAE all have shower sprayers attached to them, presumably as a hand-held bidet system, and it turns out they were going in the stalls and using them as showers, leaving the floor and seat and everything else drenched, and then taking up residence at the sinks to slowly dry off with paper towels. This gradually became apparent to the team of Chinese cleaning ladies at the other end of the bathroom, who started shouting at them "down, down, you go down! not allowed!" to the non-English-speaking Indian ladies. One by one they were physically moved back out of the sink area while one or two stragglers snuck into the stalls between the remaining Chinese cleaners who were playing defense. Since by now there's of course a line backed up out the door, they can barely squeeze out at all. This is about when the rest of the line finally realized that the stalls were in an unusable state, so we joined in the cluster@#*$ of the exodus. Good grief...

Wednesday, January 2, 2019

let's not emphasize behavioral economics *in introductory economics courses*

Scott Sumner has a great post (with an inappropriately truncated title, which I have corrected) on why introductory courses in microeconomics should not emphasize behavioral economics. But he's biased against behavioral economics, so lest that cause anyone to discount Scott's opinion, I will state that I, as a behavioral economist, wholeheartedly agree.

Scott says with dismay that when speaking with non-economists, they perk up when the topic drifts towards behavioral economics. I of course love it when non-economists perk up when they find out what kind of economics I work on, but the dismay usually follows shortly thereafter when I'm encouraged with "That's great! There's so much wrong in introductory economics; you can teach it right!" This says it all:
Non-economists also tend to reject the central ideas of basic economics, and for reasons that are not well justified.  For the economics profession, our “value added” comes not from spoon feeding behavioral theories that the public is already inclined to accept, rather it is in teaching well-established basic principles of which the public is highly skeptical.
The list of common well-established myths that follows is so great I'm going to steal it for use in my upcoming microeconomics course.

With that said, his dismissal of the field more generally reads a bit like an astrophysist saying "What use is microbiology? People claim it can be useful for the study of extraterrestrial life but I see little evidence for this." Come on...

Thursday, December 20, 2018

Kahneman and approaches to social science

This conversation with Daniel Kahneman is so great.

First of all, I love that he persistently pushes back against the notion that everything vaguely "behavioral" is a "bias". It's a terrible misconception of behavioral economics that we study the myriad ways in which people make bad decisions. Nonstandard preferences are not biases and many "biases" are sensible heuristics and error management strategies in disguise.
COWEN: Well, sports. You’re consuming bias, right? You don’t actually think your team is better.
KAHNEMAN: No, but you identify. There are emotions over which you have very little control. It’s a fact that you feel pride when your team wins. In fact, you feel pride if a stranger who lives on your street gets a prize. That tendency to identify with what’s around us, and with things that we are connected to, is very powerful. We derive a lot of emotion from it. I wouldn’t call that a bias because you can call any emotion a bias.
COWEN: If you think of the literature on what are called cognitive disabilities — ADHD — do you think of that as bias or somehow in a different logical category? Or . . . ?
KAHNEMAN: I don’t think it’s a bias, no. I think it’s an attention deficit. It means that people have difficulty controlling their attention, focusing on what they want to focus on, and staying focused. That’s neither bias nor noise.
COWEN: If you think about the issue of, when people think about the world, they find some kind of transactions repugnant. Sometimes they just don’t like to sell what they have. Other times, they seem to object to markets, say, in kidneys or kidney transplants. Do you view that as bias? Or where does that come from?
KAHNEMAN: In the sense that this is a norm, and there are things that we’re trained or socialized to find disgusting, to find repugnant. So there are repugnant transactions. And you have to treat them as you treat every other moral feeling. We have lots of moral feelings, things that we find unacceptable without any ability to really explain why they are unacceptable. There is such a thing as moral emotion. There is such a thing as indignation, as moral disgust. And that’s what we’re talking about here.
COWEN: A society such as Argentina that relies so heavily on psychoanalysis — as a psychologist, do you see that as bias? Is it a placebo? Is there a placebo effect in psychoanalysis?
KAHNEMAN: You seem to attribute . . . You seem to think that I think of bias all the time.
Secondly, I love Kahneman's unwillingness to speculate / unwillingness to engage in ex post rationalization. E.g.
COWEN: But the idea of attention-switching costs — so Israeli bus drivers, it takes time for them to switch attention from one event to another. Is that not an underlying micro foundation of your, say, 1980s papers on bias?
COWEN: That people aren’t switching their attention to the new problem?
KAHNEMAN: It’s not. We didn’t think of it. That really happens a great deal, and quite often, it happens in a different way. It happens when somebody’s insulted because you didn’t cite him. He looks at your work, and he says, “That’s just the same as what I’ve said before.” And in some way, it may be true. There may be some resemblance. It may be true, and yet you were completely uninfluenced by that. And it’s the same thing. I was uninfluenced by my earlier work, I think.
COWEN: Do you think of those in functionalist terms? Some people might argue, “Well, Israelis, they have a tendency to speak directly because they’ve had a lot of crisis situations, where you can’t beat around the bush. You need to say what you think.” Or we don’t know?
KAHNEMAN: I don’t like those kinds of explanations. They look facile to me.
COWEN: Do you have thoughts on the potential cognitive advantages of bilingualism or trilingualism?
KAHNEMAN: It’s an empirical matter. It’s not a matter of thinking. And I don’t know enough. It appears to be advantageous, but I don’t know the literature.
This approach, in contrast with a more freely speculative approach, reminds me of another issue I've thought a bit about. There's kind of a divide in behavioral economics (and economics more generally, but I'm going to write what I know best...) between what you might call the Matthew Rabin camp and the Ariel Rubinstein camp. The former says that economic theory is a slave to empirical results; regarding the evaluation of theories their motto would be Kahneman's great line "It's an empirical matter. It's not a matter of thinking." The latter says that economic theory consists of informative fables, regardless of empirics.

In my heart, I'm on team Rubinstein. I love math because it's beautiful and abstract and I honestly couldn't care less if it has anything to do with the world; I love economic theory because it's beautiful and clear and it teaches important lessons regardless of how it performs empirically in a particular situation. And in discussions like this one I'm more like a Camille Paglia (whose interview I also love), wildly theorizing from intuition and experience*, not because I think that's a valid way to go about producing knowledge but because it's at least a valid way to hypothesize and more importantly it's fun**.

But cognitively, as a scientist, I have to be on team Rabin. No matter how aesthetically pleasing a theory is, it doesn't mean anything if it isn't empirically useful. You can't think your way to the truth, even in the social sciences, and in these fields where it is so tempting to proceed on intuition, it's all the more important to insist on a rigorous scientific method.


* I realize there's more going on than wild speculation in the humanities, and that's what a lot of Paglia's theorizing is based on, but I can't tell the difference. The point is, don't take this characterization as a criticism of Paglia; I'm quite a fan, actually.

** Case in point: there's probably not a solid connection between interview style and scientific approach, but that isn't stopping me from analogizing freely.

Monday, December 17, 2018

beliefs-based altruism

I wrote a thing about my research for The Conversation, which is a great news outlet where the stories are written by academics and researchers.

The most entertaining thing about it is that the comments so far all call me out for screwing up my translation from American to Australian. I got halfway there - I translated "girl scouts" to "girl guides" (I even consulted with an Australian friend!) but I accidentally left "biscuits" as "cookies".

Whoops, my citizenship will never get approved now :)

Monday, November 12, 2018

Rising to expectations

A new working paper says:
We develop and estimate a joint model of the education and teacher-expectation production functions that identifies both the distribution of biases in teacher expectations and the impact of those biases on student outcomes via self-fulfilling prophecies. Our approach leverages a unique feature of a nationally representative dataset: two teachers provided their educational expectations for each student. Identification of causal effects exploits teacher disagreements about the same student, an idea we formalize using lessons from the measurement error literature. We provide novel, arguably causal evidence that teacher expectations affect students' educational attainment: Estimates suggest an elasticity of college completion with respect to teachers' expectations of about 0.12. On average, teachers are overly optimistic about students' ability to complete a four-year college degree. However, the degree of over-optimism of white teachers is significantly larger for white students than for black students. This highlights a nuance that is frequently overlooked in discussions of biased beliefs: less biased (i.e., more accurate) beliefs can be counterproductive if there are positive returns to optimism or if there are socio-demographic gaps in the degree of teachers' optimism; we find evidence of both.
I love this (because it fits my prior so nicely :) I went to a 2-year public math and science boarding school for 11th and 12th grade, and the education I got there was fantastic and clearly ahead of anything else available in the state. Of course many factors made that possible, including the selective admissions process, hiring of teachers with PhDs, having control over our entire daily schedules rather than just classroom hours, etc. But what stood out to me was the high expectations. The history teacher used to say "It's easy to raise expectations; what's difficult is raising performance" but on the contrary, I think raising expectations was the single most important trick OSSM pulled off, and it was able to do so credibly because of its unique position as an alternative school. Anyone who wasn't able to meet those expectations or who didn't like the inevitable slaughter of their GPA was free to go back to their home high school (and many did).

After OSSM I went to Caltech, another school that has no sympathy for those who can't keep up. The 4 year graduation rate at the time was only 77% (compared to 86% for Harvard) and 6 year rate was only 88% (compared to 98% for Harvard). This notoriously cost them the #1 position on the US News and World Report rankings after the weighting function was adjusted to put more emphasis on graduation rates. This practice certainly harms many students who were at the top of their high school classes, would have been at the top of their classes at other great universities, but struggled at Caltech. But on the flip side, this is a necessary consequence of having credibly high expectations, which are in turn critical for motivating educational achievement. I don't expect US News to quantify this nuance, but it's surely recognizeable, no?

Sunday, October 14, 2018

Which language has a word for this?

English is lacking an important word; Matt and I have had at least three whole conversations defining this and trying to figure out if there's a word for it and we haven't come up with anything. Help!

What we're trying to capture is the phenomenon where you're living your life, probably traveling but not necessarily, nothing unexpected is happening, but you're suddenly hit by a big picture perspective of what you're seeing/doing and it blows your mind. More specifically, you suddenly realize that long ago you had an abstract notion of the situation you're in, broadly speaking, and now you're actually in that situation, and your previous self never would have imagined it really happening.

This has happened to me countless times since moving to Australia but a few memorable times before then as well. When I was 12 I briefly lived in Germany and we visited the church where Martin Luther nailed his 95 Theses, and although I'd been seeing amazing historical things for months, this one really hit me for some reason as I realized, wow, a year ago I was learning about this place in a history class, and I didn't really think about it beyond as an abstract story about a place that may as well not actually exist. And now here it is, physically in front of me, one stop on a meander during a gap in the train schedule. When I moved to LA for college, the first time I drove through Hollywood and passed the exits for Sunset or Venice or Santa Monica Boulevards, all those Beach Boys songs and thousands of other movies and TV shows etc, suddenly became real. And one day when I was driving along the New Jersey Turnpike back to Brooklyn, I suddenly felt the tangible connection through time to the zeitgeist of my favorite Simon and Garfunkel songs.

Most often this is triggered by particular landmarks. The first time I walked through Circular Quay in Sydney and saw that image of the opera house and bridge over the harbor that had been on my TV every New Year's Eve of my childhood. Hiroshima, as a whole. The African tropical rainforests. The Great Barrier Reef. Uluru. But it's certainly possible for this feeling to hit you in pretty unremarkable places or places you didn't even know about before as well; in fact sometimes it's even more surprising then. Camping in some random national park I've never heard of and waking up to a field full of wallabies and I'm suddenly struck by the realization that I live in Australia and here I'm am just camping in the bush with a bunch of kangaroos. Watching sunrise from Mount Ramelau with a bunch of monks-in-training who are just as entertained to be there with two white people as we are to be there at all, and I suddenly realize, East Timor?? How did I get here again?

What is it called? What language has a word for it?

Edit: Actually, the pleasure of such experiences is similar to the pleasure of connecting visible reality with abstract, much more complex, understanding of what it is, e.g. celestial mechanics.

Thursday, September 13, 2018

An Ode To Free Trade

or, Things That I've Received In Thankfully Nontransparent Tiny Packages From Hong Kong Via Ebay (with Free Shipping!)

10 sawtooth frame hooks (+20 screws) $1.90
1 purple breakaway cat collar, $1.89
1 green breakaway cat collar with engraved tag, $5.99
1 dust brush universal vacuum attachment (as seen on TV) $6.83
1 instant hair bun maker donut, $1.58
10 heavy-duty D-ring frame hooks with screws, $6.99
10 CR2016 batteries, $8.99
10 CR2032 batteries, $7.89
1 77mm UV filter for Nikon cameras, $11.19
8 yards cheese cloth, $3.97
1 citrus press squeezer, $2.19
1 thin hard shell for Nexus 5X phone, $1.93
5 telescoping BBQ roasting skewers, $16.35
3 3-in-1 knife-fork-spoons, $5.58
2 braided micro USB cords, $5.90
1 77mm circular polarizing filter for Nikon cameras, $7.03
1 77mm center pinch lens cap, $3.89
3 2A 2-port USB wall chargers, $16.80
1 red leather coin purse wallet, $5.99
3 77mm neutral density filters for Nikon cameras, $15.59
1 100-LED string of solar powered fairy lights, $15.71
3 50-LED strings of solar powered fairy lights, $34.35
1 1080P HDMI Male to VGA Female adapter, $6.99
2 4K HDMI cables, $15.98
500g pharmaceutical-grade sucralose, $149.87
3 oil filters for Suzuki VL250 Intruder, $28.62
1 large motorcycle cover, $16.89
2 braided USB-C cables, $7.90
1 replacement touch screen LCD for Nexus 5X phone, $67.19
1 silicone dish washing sponge, $3.98
1 pull-up bar for doorways, $14.50
1 personalized engraved pet ID tag, $4.90
20 scopolamine motion sickness ear patches, $29.41
1 mini-DP to HDMI adapter, $4.99
1 Swiss Tech Utili-key multi-tool, $1.87
1 replacement backlit keyboard for Lenovo thinkpad, $39.64
1 tempered glass screen protector for Nexus 5X, $1.00
50 meters heat-resistant double-sided tape, $8.87
4 1600mAh batteries plus charger for GoPro 4, $23.99
1 stainless steel mesh sink strainer, $1.69
10 hair bun spiral claws, $2.29
100g sumac, $6.95
1 large tea infuser ball, $6.00
50 N35 neodymium magnets, $4.90
3 A3 black picture frames, $29.97
20 3M Command small poster strips, $12.98
250g andydrous caffeine, $19.07
1 replacement wrist band with metal buckle for Fitbit Flex, $1.67
100 cinnarizine motion sickness tablets, $30.80
1 pair boot laces, $1.80
1 remote shutter release for Nikon cameras, $19.99
8 mandolin strings, $3.99
1 rocket air duster for camera lenses, $4.47
150 packets of Emergen-C, $30.82
2 aluminum bicycle water bottle holders, $2.38
100 colored push pins, $3.16
4 power cable adapters for Lenovo thinkpads, $6.80


Australia is really expensive, compared to the U.S. But with ebay, given a little bit of patience, I can save ludicrous amounts of money (even more than with Amazon Prime, which I do miss for the things that need to be higher quality than you can count on from Chinese Ebay sellers) and skip the hassle of shopping. I have no idea how sellers are making any money on some of these things. That's the magic of the invisible hand.

Sunday, June 3, 2018

I really love Brisbane

I love it here so much.

Friday afternoon I was supposed to go to the hospital for a blood test but got delayed at work. Driving from UQ to South Brisbane at 5:30pm is not something I expected to go well, but apparently my expectations are still partially calibrated on SF/NY/LA data, so it was absolutely no problem. I lane filtered through the one or two blocks of red zone traffic and got there in 15 minutes.

Parking in South Brisbane during the dinner hour? No sweat, free motorcycle parking right by the entrance.

The pathology lab was closed by this point (but how awesome is that, that you can show up with no appointment for a blood test at a giant public hospital and wait about ten minutes total?!), so I followed the after-hours instructions to the emergency department desk. They told me I should go to the emergency department at the private hospital on the next block.

Uh-oh, this is where I get screwed... I've been carefully sticking to Mater Public for absolutely everything related to this freak knee infection debacle, so as to avoid the cluster#@*$ of dealing with multiple medical bureaucracies and getting charged out the wazoo the second I inch away from what I know is covered by medicare, but I guess I have to bite the bullet. I walk over to the gleamingly empty private hospital and try to ignore the impending bills.

They tell me they'll have to page someone to come do the test. Uh-oh, this is where I have to settle in for a four hour wait. Ah well, I didn't have any Friday evening plans anyway. I pull out my laptop and headphones and get psyched for a few hours of uninterrupted work.

Approximately 43 seconds later, my name is called.

Approximately 5 minutes after that, I'm back at the front desk. The ladies look at me quizzically. "Uh, I'm finished with the blood test." [More quizzical looks.] "Is there anything I have to do?" "Oh no, you're all set, have a good weekend!" [Amused looks at the silly American.]


[Previously, after an overnight hospital stay for four rounds of IV antibiotics that cost exactly 0 dollars and 0 cents (plus very high taxes but if this kind of logistical functionality is what you get for the public funding it's worth it ten times over), I was discharged with prescription antibiotics to take at home. Prescriptions are subsidized but not covered by medicare, so I was pretty nervous about getting screwed on that front. In fact, the doctor brought them up and asked me hopefully "Do you have a concession card or anything...? There's an invoice in here but it didn't have any discounts applied." My stomach immediately tied itself up around the expectation of hundreds of dollars of fancy-pants drugs and I didn't even dare look at the invoice until two days later when the fever subsided enough to give me the energy to deal with a new hit of bad news. The total? $27.30.]

I've now spent a total of about 20 minutes in both hospitals, so my bike is still warm when I take it back around the river under all the prettily lit bridges. It's the dead of winter, but with my regular jacket it's still a great evening for riding. Then after a cosy night in with hot cocoa and Bailey's, all weekend it's cloudlessly sunny and 70 degrees. [Recently a colleague from the UK visited UQ for a couple weeks and after four days I said "It's a little grey lately, hopefully it'll clear up for your weekend." He replied "I've been in the UK for twelve years and haven't had four days this nice." I guess my reference point has moved a bit after all.]

Maybe more relevantly, being done with lecturing for the year puts me in a really great mood :)

Thursday, February 1, 2018

MTurk demo

This may be useful to other researchers, so I'll share publicly. If you use MTurk for research, you most likely will save a lot of time if you set up the command line tools. These allow you to add assignments to HITs, grant qualifications, pay bonuses, and much more, all automatedly. I did a demo of setting up and using them, and a screencast is available [see update below]. (If you get an error that the file is corrupted, your browser is lying. Download it.)

Disclaimer: I am very far from being an expert on MTurk, I've just figured out how to do the things I want to do as I go. Also, cut from the beginning of the demo: If you use mac, pretty much everything works exactly as I show even though I'm using linux. If you use windows, things will be very slightly different, but the online documentation gives instructions for both platforms so refer to that as needed.

Update: The video and its followup are now on youtube.

Wednesday, August 23, 2017

the eclipse

It is very disconcerting to point your telescope directly at the sun, take the filter off, and immediately look through the eyepiece. I've spent most of my life learning to be very careful not to do just that. But that sight, with true-color bright pink prominences, glowing tendrils of the corona against a royal blue background, and the enormous, terrific void in the center, was hands down the single most incredible thing I have ever seen.

I've seen thousands of photos of the sun and its various types of surface features, and I've seen thousands of photos of eclipses, and read hundreds of elated accounts, and I've seen lunar eclipses and an annular solar eclipse, several partial eclipses and a Venus transit, but this was the difference between reading a textbook about string theory and sneaking a glance through a fleeting crack in spacetime itself. It's the difference between kissing a man and marrying him; the difference between a ferris wheel and skydiving; between a bathtub whirlpool and a tornado. No matter how much you objectively know what is coming, it's impossible to be adequately prepared. I am not at all surprised that ancient humans thought they were staring into the end of times, and they didn't even have a magnified view.

Eclipse progression from Madras, OR

And if anyone should have been prepared, it's me. I've been looking forward to this eclipse for the last 20 years (ironically because it was supposed to be the first eclipse I wouldn't have to fly around the world for. Whoops.) This was even before I owned my first telescope and was still contenting myself with learning constellations, hunting for Hyakutake and Hale-Bopp, measuring latitude with a protractor, and waiting 45 minutes on dial-up internet to download photos of Neptune. (I was not the most popular kid...) I can't adequately describe the eclipse directly, but surely you can infer from that kind of history that I had pretty high hopes and expectations, so if I'm still paralyzed by awe 36 hours later, it really had to have been pretty great.

Partial phase, with (unusual for this time during the solar cycle) sunspots. The grainy surface appearance is due to temperature variation (not digital noise).

Why exactly is it so incredible? I wish I had the faintest idea. I've experienced sunset and the associated sky colors, drop in temperature, and darkness. It's "just" a black circle in the sky. I don't even believe in anything supernatural that could have turned it into a spiritual experience. Sure, seeing things with your own eyes is often much more powerful than in photos, but nothing else purely visual, not even looking into the Grand Canyon, or seeing supernovae in other galaxies with my own eyes, or swimming with manta rays, has come close to giving me an adrenaline rush that took half an hour to dissipate. I've watched my own little gopro video of totality (below), in its gloriously crappy quality,  a dozen times already and it gives me chills just from remembering it. Vividly remembering bungee jumping also gives me a bit of an adrenaline rush, but that's a memory of jumping off a bridge and repeated freefall, not ... looking up. Matt's knees were shaking ages after totality was over, I had trouble firmly taping the solar filters back on the telescope with shaking hands, people around us were crying. I think if we could figure out what primal nerve these things hit and why, we would understand humanity much better.

Matt and I drove up to Oregon to see it with two friends who live in the Bay Area. On the way there I read up on photographing it, which I hadn't had time to think about and which I'd assumed I would only make a very cursory attempt at since there is no spare time during 2 minutes and 5 seconds of totality to waste fiddling with equipment instead of looking at it directly. But I found out that it's really easy to automate the shooting by plugging the camera into a laptop and scheduling a series of bracketed captures in darktable (the open source imitation of lightroom). I set my camera to F4, ISO200, 200mm (the maximum of my good quality lenses), 1/30 basic exposure, but then 10 exposures around that value ranging from 3 seconds to 1/3200 in order to capture different levels of detail. All during the partial phase I was doing something similar with manually tweaked settings as the conditions changed, but I planned on these values ahead of time for totality based on a bit of experimentation the day before and recommendations from the internet. Luckily I wrote them down, because with 2nd contact rapidly approaching I was already losing it over watching the tiny sliver in the telescope break into individual Bailey's beads and then disappear. With 30 seconds until totality or so I hurriedly punched in the settings, pulled off the filter, and pressed start (telling it to cycle through those exposures repeatedly until I stopped it) and looked up just in time to see the spectacular diamond ring, more clearly than I could have ever expected, more clearly than anything I was able to capture photographically (due to not controlling which exposure it was taking at the exact second needed), and I swear more clearly than I've seen in almost any other photo either. Unbelievable.

Outer corona during totality

Shorter exposure showing the inner corona and three pink prominences (at 11:30, 1, and 3).

And just that fast, it was gone, and we plunged into darkness. Venus showed up like a spotlight, and sunset colors spanned all 360 degrees of the horizon. People cheered; Matt played Pink Floyd's "Eclipse", which at 2 minutes and 1 second, almost had to have been specifically written for this time and place... I was completely taken aback by the naked eye sight overhead, but Matt had the sense to look through the telescope and exclaimed about the bright pink prominences. I took a look, and did a double take, and then a triple take, and now that image is permanently seared in my memory. I obviously can't stop myself from continuing to fail to describe it, so one more try: it's not just more beautiful than you expect, it's not just surprisingly moving, it's like staring straight at something you know in your core you're not supposed to be allowed to see, something that may have dire consequences, but that you most certainly can't look away from. And again, I have no idea why. Is it the strongly conditioned hesitancy about looking straight at the sun through a telescope? Is it the hole in the fabric in the universe that looks like a tunnel to the afterlife? Is it simply too alien to process with existing neural connections? Perhaps all of the above would begin to come close to explaining the adrenaline rush it caused. I wish I could at least share a picture, but nothing I can find online matches that view.

Even faster than it began, it ended. A neighbor set off fireworks; skydivers landed at the airport across the street; I flipped the filter back on the telescope as fast as possible with some dubious bits of scotch tape that Matt sensibly reinforced and watched the second set of Bailey's beads form and merge into a larger crescent, and when I looked up ten seconds later it was once again hot and brilliantly sunny, even with only a few percent of the sun uncovered. As soon as the moon fell behind the sun on the tail end we hit the road, which turned into an Oregon crossing at an average of 12 miles per hour. And it was worth every single second.

See you in Oklahoma in 2024.

Any photoshop experts want to help me make a better exposure stack than this one...?

Friday, August 11, 2017

MTurk tips

2 easy hacks to make MTurk/Web/Qualtrics data collection/management easier:

1. isn't actually a hack, just a recommendation to send users to an experiment that is either hosted on your own server or goes through your own server as a landing page (before redirecting to qualtrics or whatever). This serves a couple of purposes. You have complete control and complete records of absolutely every interaction anyone has with the website. You can see the IP and entry point and time of everyone who loads the site, you can record exactly when and where they click, you can manage random assignment to treatment groups yourself in a way that keeps samples balanced or meets any other constraint you have, you can see how many people open the survey and decide not to do it, etc. I also send users back to my server (with an automatic redirect from qualtrics) so I can mark them off as having completed the study.

2. is a hack assuming you do #1: make sure your server keeps all access logs from a long enough time span that at the end of the study you can keep a full record of all interaction. I cannot tell you how many times this data has been useful to have. With the timestamps and IPs I can tell when someone in the lab changed computers because he entered the wrong url and got an error on his first computer, I can tell you who used their phone, who restarted, who finished but just didn't click "submit" at the end, etc etc etc. There are always a few mystery people in the data who didn't do things the right way or had technical issues and I've always been able to track down what happened this way.

3. is a hack assuming you do #1 and #2 in combination with using qualtrics (or any similar third-party platform). Instead of hosting photos or other imported media on the third-party platform, host them on your own server and load them via url on the other platform. Every access of this kind will show up in the access logs, giving you better timing information and IP tracking than those platforms will usually tell you. Even qualtrics, which is phenomenal and and will give you timing information for every page of every survey, only records the time of the first and last click, but if the page is loaded and immediately exited you won't be able to tell, and if something is clicked that opens a sub-question and then closed to avoid the sub-question you won't be able to tell, and if someone fails a verification check and has to re-do a page you can't tell, and so on and so forth. You can make hacks of this kind arbitrarily fancy with custom code.

Sunday, January 29, 2017

biggest unanswered question

I'd like to hear people's answers from other fields and from a variety of economic subfields as well. Go, mysterious meme powers...

My answer for economics generally and behavioral economics specifically is exactly Tyler's: "culture".

To try to be slightly more specific: I basically see cultures as collections of norms and methods of norm enforcement.  There are various types of norms (but note that particular norms do not often fall clearly into exactly one bucket):

  1. Coordination norms, like which side of the street to drive on, are fairly well understood both in origin and enforcement (i.e. self-enforcing).
  2. Social enforcement of norms that provide public goods (e.g. littering) is getting to be fairly well understood. But which public goods are provided by norms and how this comes to be is utterly mysterious and of utmost importance. There are lots of pieces of the puzzle identified but not put together at all.
  3. Norms for division of goods (splitting the pie) is a sort of subcase of #2 but it's a weird area of study because these are some of the simplest and most universal norms yet the theories (e.g. Binmore's) of them are some of the most complex. It's really really interesting, but almost overkill.
  4. There are also many norms that are better described as heuristics for dealing with uncertainty and limited information (don't pick up hitchhikers; eat breakfast), and I don't think the selection and enforcement of these norms is at all well understood, nor given serious attention. I'm sure most are written off as individual heuristics, but the degree of social enforcement and arbitariness suggest to me something more is going on that just judgmentally inferring bad things about other people from their stupid decisions. I could be wrong. At the very least, the individual motivations for adhering to these norms are closely related to individual motivations for adhering to cooperative norms, so the literatures are intertwined.
  5. Lastly there are norms that are completely arbitrary, don't fit in any of the above categories, and exist solely for signaling value, like wearing ties to weddings. These are not very mysterious, in the same way that sexually selected traits are arbitrary but evolutionarily well understood.
In summary I'd say the most important and mysterious unanswered question of economics is the point from #2: which cooperative norms are chosen to be enforced and how does this come about?

Friday, July 1, 2016

is the referee process fair?

I'm at the Western Economic Association International annual conference in Portland and just saw a fascinating keynote address by David Card, on his work in progress with Stefano DellaVigna on "What Gets In" top economic journals. The paper is unfortunately not yet available online, so I can't excerpt or show any of the very interesting diagrams that go with the analysis, but in the meantime I can tell you to keep a (skeptical) eye out for it when it does get posted.

The paper aims to confirm or dispel the common belief that the editorial process is unfair because of some combination of three factors: 1) Referees aren't good at assessing quality, 2) the process is biased in favor of big name authors, and 3) the editors overweight their own priors relative to referee recommendations. The authors acquired data from QJE, JEEA, ReStud, and Restat* and looked at three stages of the editorial process in comparison to ex post citation rates (controlling for journal and time) as the measure of paper quality. The three stages of the referee process are 1) the decision to desk reject, 2) the decision to send the paper to a particular number of particular referees, and 3) the decision to reject or RR/accept after receiving the reports.

Bullet points:

  • Referees are good at assessing paper quality in the sense that their ratings (from 1-7; definitely reject, reject, no rating, weak R&R, R&R, strong R&R, accept) line up well with ex post citations.
  • Higher quality referees, measured either by citation counts or publication numbers in 35 top journals in the preceding 5 years (I can't remember which), aren't better at assessing paper quality.
  • Papers that are sent to a larger number of reviewers are cited more, so the number of reviewers is a proxy for the editor's prior belief about the paper.
  • Prolific authors (measured by publication numbers in 35 top journals in the preceding 5 years) get many more citations controlling for reviewer rating.
  • So do papers with more authors.
  • Editors increase the citations of published articles by publishing papers by prolific authors more often, conditional on reviewer rating, but they could go further and do even better.
  • Editors do not seem to take into account the number of authors, and could increase citations by publishing more of these articles.
  • Editors could also increase citations by putting larger weight on their own prior relative to reviewer ratings.

The conclusions David drew are that 1) referees are indeed good at assessing quality, 2) the process contains affirmative action for junior/less prolific authors, and 3) editors are not overconfident. Thus, the myth of unfairness is dispelled.

The assumption this story rests on is glaring and glaringly fragile: ex post citations is the relevant measure of paper quality when people assess whether papers are fairly treated.

From the perspective of editors, I completely understand why you would focus on citations. That's how your journal gains prominence. But as a scientist, what I want and what I believe is the gold standard for fairness is that papers are published and cited in proportion to their quality. Treating citation rates as quality assumes away half of the problem.

Are citation numbers just the best measure of quality that we're stuck with? Well I'm sure that was the reason for using it, and I'm sure citations are correlated with quality, but as they show, referee ratings are also correlated with citation numbers. Since the citation process is self-evidently biased in favor of prolific authors** (I'm sure you can prove this to yourself through introspection just as easily as I did), and since referees are several of a very small number of people who thoroughly study any given paper, it seems utterly bizarre that the former, and not the latter, would be treated as the primary proxy measure of quality (if the goal of the paper is in fact to assess fairness rather than to assess journal performance.)

If we consider referee ratings the better measure of quality, the conclusions exactly reverse and exactly confirm some of the common suspicions of the editorial process: 1) Citations are a good measure of quality but substantially biased in favor of prolific authors and multi-author papers, 2) editors are biased in favor of prolific authors, but not as much as citations are, and they are not biased in favor of multi-author papers, and 3) editors could reduce their bias by putting less weight on their personal priors.***

I do suspect citations are a better proxy for quality in the sense that they are less noisy (but more biased). I'm sure this noise is why people complain about the competence of referees, in fact. This does mean that saying a particular paper was treated unfairly based on the average of three wildly different referee ratings isn't going to be credible. But when we're looking at data from 30,000 paper submissions, the signal shines through the noise and bias is much more important to worry about.


*Iirc, which applies to the entire summary.

**and it certainly makes sense to me that it could be biased in favor of multi-author papers as well, since more authors are necessarily more in contact with potential citers. Then again it also makes sense to me that multi-author papers could be higher quality, since there are more eyes on every step of the process.

***I asked David about this at the end of the talk (and several people immediately thanked me for it), and he readily admitted the alternative interpretation. I appreciate that and don't wish to accuse him of any suspect interpretation of data when I can't even read the paper yet, but it's a point worth discussing even if the paper makes it much more clearly than he did in his talk.

Tuesday, May 17, 2016

a failure of inference

Some idiots put a baby bison in their car in Yellowstone National Park out of "misplaced concern" for its wellbeing. He imprinted on humans and cars so quickly that he could not be persuaded to rejoin its herd, and the herd rejected him as well, including his mother. The calf was causing a danger to cars in his insistence on returning to the road, and so for reasons detailed below, park staff were forced to euthanize the calf.

Cue 13,000 comments on Yellowstone's facebook page accusing them of being heartless murderers.

There are plenty of fact-based suggestions and objections to be made on both sides, and the NPS has responded to most of these comments with the form response "In order to ship the calf out of the park, it would have had to go through months of quarantine to be monitored for brucellosis. No approved quarantine facilities exist at this time, and we don't have the capacity to care for a calf that's too young to forage on its own. Nor is it the mission of the National Park Service to rescue animals: our goal is to maintain the ecological processes of Yellowstone. Even though humans were involved in this case, it is not uncommon for bison, especially young mothers, to lose or abandon their calves. Those animals typically die of starvation or predation."*

But that's beside the point. I don't have to know any of the facts involved in order to have an opinion on the matter, because of all people, the park service is staffed by the ones most likely to go to the end of the earth to care for wildlife, especially in this heartwrenching case of a baby calf rejected by its mother due to human interference. Not only do I know for certain that they are much better informed of the options and issues than I am, I know that they have infallible intentions when it comes to conservation as well. So, I don't even have to "trust" them to make the right decision (since "trust" connotes a leap of faith that the right thing will be done despite conflicting personal incentives), I can infer with high confidence that they will do, and did, the right thing. Because if there were any kinder option, I know the people involved would have wanted to take it.

I sure hope these 13,000 commentators aren't representative of humanity overall, because the signaling models I'm so fond of are doomed if they are. I know people underestimate the intentions of others when they disagree, but in this case everything lines up including intentions; there is no basis for doubt that the right thing was done.

*They probably could have left off the part about their mission, which is completely reasonable and accurate but doesn't help project a superficial image of compassion (emphasis on superficial).

Thursday, October 15, 2015

re-SMBC part 6

More accurate redo of SMBC number 3823:

This is a bit inaccurate, because of course an economist wouldn't even ask why...

Saturday, May 16, 2015

problems with measuring personality

At lunch we were discussing Big Five traits (a common 5-dimensional categorization of personalities, measuring openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism) and this reminded me of the strange phenomenon in my Myers-Briggs categorization that I've noticed over time. In 8th grade I first scored INTP - entirely Introverted vs Extroverted,  strongly iNtuitive vs Sensing,  entirely Thinking vs  Feeling, and borderline Perceiving vs Judging. Nowdays I'm only mostly I and slightly J. I've only taken the Big 5 test recently so I unfortunately don't know the trend, but I assume something similar would have happened.

I don't feel like my personality has changed so this has puzzled me. But now I think I understand the source, and it is concerning with regard to the use of Big 5 in research (mostly in psychology but more frequently as control variables in economics.)

If people are free to maximize their utility of time use,  the marginal utility of any activity should be equal at the bliss point. Obviously we work with many constraints, so I don't expect this to be exactly true*, but I expect it to be closer to true the more freedom of choice we have.

In 8th grade I had no choice but to spend 8 hours a day around people and to follow the strict schedule of activities in general. This was far too much human interaction for me and I would nearly always opt for alone time when given the chance. Nowdays I've carefully engineered my life to have as little compelled socializing as possible, and will usually opt to go to any party friends of mine might have. Similarly,  in San Francisco Matt used to not believe me when I said I was more introverted than him, because I was usually the one wanting to go out and do stuff after work and on weekends. But I mostly worked at home alone,  while he spent 10 hours a day in an open plan office. I would (/did) have a nervous breakdown in that environment.

Along with having more freedom to be alone, I also have more control over my schedule and environment. So while previously the rigid structure and organized environment imposed on me was more than enough, I now realize that I do generally like having a plan and an organized approach to things, hence P became J.

I can't think of what might have suddenly given me more freedom to trade off S/N or T/F, so I'm not surprised those have remained steady.

What does this mean for using these scores in regressions? My first thought is that asking about marginal preferences to measure averages will make people look less variable than they are and will understate the importance of personality. But that's on average. I bet there are plenty of circumstances in which the measure is actually biased.

*but maybe surprisingly close since we should really be equating the marginal present value of activities,  not immediate happiness, which makes work and sleep seem a lot more attractive.

Tuesday, May 12, 2015

awesome broken windows theory tests

This conference on "social norms and institutions", as you might expect, is quite cross-disciplinary. I'm not sure what the breakdown is, but there are plenty of behavioral economists, sociologists, psychologists, and even law professors and political scientists. So, I'm collecting quite a list of interesting new sources from lectures and conversations with these non-economists.

The most fun, and incredible, set of experiments I've learned about are the ones run by Siegwart Lindenberg and Linda Steg and colleagues at Groningen University. Despite having an unusually high prior*, I'm honestly stunned by their results, both in terms of the sheer size of the effects they're finding, and the stability of those large effects across a great number of slightly different situations.

They're studying, essentially, the broken windows theory. This theory hypothesizes that if there is evidence of some law/norm breaking behavior, that will cause people to break other laws/norms at a higher rate.** This is not due to inferences about levels of enforcement, because the findings are just as strong in purely normative (prescriptive, in addition to proscriptive) settings. In a world of marginally statistically significant results, it's a thrill to see such a long list of replications with huge and extremely statistically significant findings.
  1. Bikes are parked in a row next to a fence with a conspicuous "no graffiti" sign, and flyers are attached to each bicycle such that they must be removed to use the bike. If no graffiti is on the fence, 33% of subjects will litter their flyers. If graffiti is on the fence, 69% will. This was so surprising that a news station paid the researchers to replicate the study while they watched from rented rooms looking down on the area. The finding replicated very closely, and now the Netherlands requires immediate removal of graffiti.
  2. A temporary fence is set up around a parking lot. Two police ordinance signs on the gate state "no trespassing" and "no locking bikes to fence". People were directed 200 meters around to an alternative entrance, but the gate was left about 50cm ajar. If four bikes nearby were not locked to the fence, 27% of people squeezed through the gap in the main gate. If the four bikes were locked to the fence, 82% did.
  3. In a grocery store parking lot, a sign states "please return your shopping carts", and flyers are left under the driver-side windshield wipers on cars in the lot such that people had to remove them to safely drive away. When no stray carts were left in the lot, 30% of drivers littered their flyers. When several carts were improperly left in the lot, 58% of drivers littered.
  4. Bikes parked in a shed have flyers attached to them as in example 1. In the control condition, 52% of subjects littered their flyer. If fireworks were set off, within earshot but out of sight, which is a well-known offense prior to December 31 in the Netherlands, 80% littered.
  5. An envelope is left partially hanging out of a mailbox and visibly has a 5 Euro note inside, showing through a window in the envelope. In the control condition, 13% of people who passed the mailbox on foot stole the envelope. If the mailbox was covered with graffiti, 27% stole the envelope. If the mailbox was not covered in graffiti but litter was on the ground next to the mailbox, 25% stole the envelope.
  6. A stamped addressed envelope is on the ground next to a mailbox. In the control condition, 24% of passersby put the envelope in the mailbox. If bags of garbage are on the ground nearby (illegal in the Netherlands), 10% did.
  7. A bike is on the ground in an alley, having apparently accidentally fallen off its stand. If passersby have just entered the alley from an empty, clean street, 20% of individuals and 27% of groups right the bicycle. If garbage bags had been left on the street, then 6% of individuals and 5% of groups did so. If prior to entering the alley, passersby passed by someone who dropped an aluminum can and then picked it back up, 34% of individuals and 35% of groups picked up the bike.
  8. A person on the sidewalk accidentally drops some oranges just before meeting another pedestrian. Normally, 40% of passersby help the stranger pick up their oranges. If approximately 20 yards earlier, the passersby had witnessed someone drop an aluminum can and pick it up back up, 64% will help the stranger. If 20 yards earlier, the passerby had witnessed someone (a private citizen) sweeping the sidewalk, 82% helped the stranger.
  9. I can't find the numbers for this one, but in a mall where you have to walk down a hallway to get to a smoking zone, a certain percentage of people will start smoking before they reach the zone. If a group of typical looking Germans are standing around smoking prior to the zone, though, more passersby will light up early. If a group of goth teens are smoking prior to the zone, though, a higher than normal percentage of passersby will wait until they get to the smoking zone to light up.
Amazing. And while each individual study has many issues with interpretation and is targetting a different nuance of the phenomenon, the evidence from all 9 considered together is undeniably striking.

References: 1-8 come from here and here.


* Perceived unusuality is based on my reading of the literature (see next footnote). I've always been sure (without solid evidence) that the mechanism must be quite important.
** This motivated a famous crackdown on petty crime in NYC, which did undergo a drastic reduction in crime, but causality has not been convincingly established and the theory remains controversial.

Friday, January 2, 2015

Australian PC

Australia is much less PC than the US, which I love. It's just plain awkward when US TV shows have to hide jokes about groups of people in a layer of meta by, e.g., having an uncouth character make an uncouth comment so the joke is the character rather than the comment.

Shortly after I arrived in Brisbane, I was in a workshop and a chair collapsed underneath an Asian man. After it was clear he was ok and we all had a good laugh, someone asked "One too many dumplings, huh?" Imagine someone saying that in California!

That same week I was at a department event and introduced myself to someone as the new lecturer, and he said "Oh! I was wondering who you were, I thought maybe we had a new secretary." Even moreso, can you imagine someone saying that in California! Americans can't even call them secretaries anymore! I was certainly not offended* but had a hard time not losing my composure while I died from internal shocked laughter.

But the best one so far was a van Matt spotted earlier this week in Sydney. "You ling, we bling." We couldn't figure out what it was, maybe something involving chrome rims installation? Turned out to be a Chinese restaurant delivery van.

*I'll be offended the first time someone suggests, based on my gender, that I might be more suited to being a secretary, but nothing in my experience has come remotely close.