Here you go, Quirkybook.
This is pure speculation, unlike the rest of my posts, which are nothing but fact.
Was Cheney gambling by getting the National Academy of Sciences to review the Klamath River Biological Opinion? After all, the NAS is not known for being political hacks; they might well have supported the Biological Opinion.
Well, in the first place, Cheney had nothing to lose. A legitimately completed Biological Opinion had already been issued, saying that water had to remain in the river for the continued existence of endangered suckerfish in the lakes and threatened coho salmon in the Klamath River. Presumably, if that Biological Opinion could be challenged in court, it would have been. Cheney wasn’t willing to call in the God Squad, which is the appeal process outlined in the ESA. Since he’d already lost, calling in the National Academy of Sciences couldn’t make it worse.
Second, no matter how it turned out in the Klamath, Cheney had achieved a different goal. Cheney had introduced a whole new level of review of the ESA. That never existed before. Even if it changed nothing in the Klamath, it was a concept that could be used next time. Adding a new layer of review is one more chance for each decision to get reversed. Also, it was a snide dig at agency science and a way to pit scientists against each other. In the long run, the NAS supported the Fish and Wildlife Services analysis, but in the meantime it let people say that the quality of agency science is questionable*. So Cheney weakened the ESA and strengthened the idea of science "controversy". Since that worked so well for the Bush administration in evolution and global warming, why not throw it into the mix in species protection?
I have another idea teasing me, but I can’t get it to resolve fully. I wonder if Cheney wasn’t cynically taking advantage of science’s reliance on the null hypothesis and statistical significance. He knew that the NAS approach would be the null hypothesis, that water levels in the lake would not influence suckerfish, and that they would be trying to disprove that. That is a really high threshold and very hard to meet in the three months they had to spend on this. He knew those stuffy-ass scientists weren’t going to say, yes, it is perfectly obvious that low flows will get hot and cook the fish. Instead they would say, no, null hypothesis isn’t disproved. That was enough for him. He can take that to the public, count on them to not understand the null hypothesis and standard for significance and say “Fish and Wildlife Service makes shit up, but I have real scientists who say it isn’t proved yet.” He may have predicted that NAS would say what scientists say, and that was good enough.
I never get tired of saying bad things about Cheney, but I should spread that around some. I think NAS got used and that they weren’t clever enough see that they were doing damage. I wonder if there wasn’t some hubris involved in that, and I think they should feel ashamed of their participation. They didn’t mean to, but they caused harm by getting involved in political machinations they didn’t understand. They cheapened the NAS, if nothing else. Also, the top bureaucrats at Rclmtn shouldn’t have caved. I understand why they did; they are also political appointees and the Vice-President flew out to see them personally. But they failed their duty to the public, which was to stand by the results of the processes the public decided they should follow.
*Dudes. It is not perfect. It is fine to say that agency science should be reviewed. There are venues for that, like in the public process, where both sides pore over agency analyses before they are issued. (Yes, they do. Do not assert that public comment is a half-assed mechanism unless you yourself have submitted or read public comment on an agency document and can give concrete examples. I’m getting tired of this shit where you know things to be true about bureaucracies, and then that turns out to be based on academic articles or your ideological beliefs about agencies. In fact, if you can’t actually name the management agencies in the fields you care about, I do not think you know much about how agencies work.) Another venue for review of agency science is a court challenge, where competing experts tear the document apart. The ESA has a review process built into it - the God Squad. But agency science is experts doing a credible job, knowing full well that there are consequences to their analyses and that no matter what they write, someone will hate it and try to discredit them. You can be as skeptical of agency science as you are about any science, but there is no reason to assume a priori that agency science is not solid.
***********************
Francis knows what he is talking about, and corrects some points I made:
BEFORE YOU COMMENT:
Restricting the comments in the last post so that we didn't hear the same arguments repeated seemed to work. I'm going to do it again. If you want to weigh in on the quality of agency science, your first sentence must include the name of the agency document you personally have reviewed. It can be a Biological Opinion, an EIR, an EIS, a Habitat Conservation Plan, a Timber Harvest Plan, a General Plan, any substantive document put out by an agency. I do not want to hear how all agency staff are blinded by their biases so they do shoddy science because you just know it is true even though you've never read an agency document.
Got that? Name of document first and your interest in the document, or no assertions about agency science.
Update: If you want to comment on other aspects of this, go ahead. It is only for critiques of agency science that I want you to have actually been in contact with agency science.
Was Cheney gambling by getting the National Academy of Sciences to review the Klamath River Biological Opinion? After all, the NAS is not known for being political hacks; they might well have supported the Biological Opinion.
Well, in the first place, Cheney had nothing to lose. A legitimately completed Biological Opinion had already been issued, saying that water had to remain in the river for the continued existence of endangered suckerfish in the lakes and threatened coho salmon in the Klamath River. Presumably, if that Biological Opinion could be challenged in court, it would have been. Cheney wasn’t willing to call in the God Squad, which is the appeal process outlined in the ESA. Since he’d already lost, calling in the National Academy of Sciences couldn’t make it worse.
Second, no matter how it turned out in the Klamath, Cheney had achieved a different goal. Cheney had introduced a whole new level of review of the ESA. That never existed before. Even if it changed nothing in the Klamath, it was a concept that could be used next time. Adding a new layer of review is one more chance for each decision to get reversed. Also, it was a snide dig at agency science and a way to pit scientists against each other. In the long run, the NAS supported the Fish and Wildlife Services analysis, but in the meantime it let people say that the quality of agency science is questionable*. So Cheney weakened the ESA and strengthened the idea of science "controversy". Since that worked so well for the Bush administration in evolution and global warming, why not throw it into the mix in species protection?
I have another idea teasing me, but I can’t get it to resolve fully. I wonder if Cheney wasn’t cynically taking advantage of science’s reliance on the null hypothesis and statistical significance. He knew that the NAS approach would be the null hypothesis, that water levels in the lake would not influence suckerfish, and that they would be trying to disprove that. That is a really high threshold and very hard to meet in the three months they had to spend on this. He knew those stuffy-ass scientists weren’t going to say, yes, it is perfectly obvious that low flows will get hot and cook the fish. Instead they would say, no, null hypothesis isn’t disproved. That was enough for him. He can take that to the public, count on them to not understand the null hypothesis and standard for significance and say “Fish and Wildlife Service makes shit up, but I have real scientists who say it isn’t proved yet.” He may have predicted that NAS would say what scientists say, and that was good enough.
I never get tired of saying bad things about Cheney, but I should spread that around some. I think NAS got used and that they weren’t clever enough see that they were doing damage. I wonder if there wasn’t some hubris involved in that, and I think they should feel ashamed of their participation. They didn’t mean to, but they caused harm by getting involved in political machinations they didn’t understand. They cheapened the NAS, if nothing else. Also, the top bureaucrats at Rclmtn shouldn’t have caved. I understand why they did; they are also political appointees and the Vice-President flew out to see them personally. But they failed their duty to the public, which was to stand by the results of the processes the public decided they should follow.
*Dudes. It is not perfect. It is fine to say that agency science should be reviewed. There are venues for that, like in the public process, where both sides pore over agency analyses before they are issued. (Yes, they do. Do not assert that public comment is a half-assed mechanism unless you yourself have submitted or read public comment on an agency document and can give concrete examples. I’m getting tired of this shit where you know things to be true about bureaucracies, and then that turns out to be based on academic articles or your ideological beliefs about agencies. In fact, if you can’t actually name the management agencies in the fields you care about, I do not think you know much about how agencies work.) Another venue for review of agency science is a court challenge, where competing experts tear the document apart. The ESA has a review process built into it - the God Squad. But agency science is experts doing a credible job, knowing full well that there are consequences to their analyses and that no matter what they write, someone will hate it and try to discredit them. You can be as skeptical of agency science as you are about any science, but there is no reason to assume a priori that agency science is not solid.
***********************
Francis knows what he is talking about, and corrects some points I made:
Delegation to the NAS: there are two ways of looking at the reference to NAS. One is the way you did, in which the NAS review is an insult to NOAA scientists.
Another is that NAS review can add legitimacy to the government's view of the science in a hotly disputed area.
As usual, the devil is in the details. It is possible to stack NAS review boards (which may have happened). And it is certainly possible for a vice president to deliberately mis-read the conclusions of a NAS review (which appears to have happened).
Second, the God Squad is not an appeal process so much as a bypass process. The God Squad does not determine whether the bio. opinion is correct; instead it determines whether the national interest so outweighs the conclusions of the bio. opinion that the project should proceed.
Third, federal district court review of a bio. opinion does not, except in the rarest of cases, involve expert testimony. Courts leave to the agencies the weighing of expert testimony. Courts only determine whether the agency followed statutory law and applicable regulation in coming to the conclusion it did.
BEFORE YOU COMMENT:
Restricting the comments in the last post so that we didn't hear the same arguments repeated seemed to work. I'm going to do it again. If you want to weigh in on the quality of agency science, your first sentence must include the name of the agency document you personally have reviewed. It can be a Biological Opinion, an EIR, an EIS, a Habitat Conservation Plan, a Timber Harvest Plan, a General Plan, any substantive document put out by an agency. I do not want to hear how all agency staff are blinded by their biases so they do shoddy science because you just know it is true even though you've never read an agency document.
Got that? Name of document first and your interest in the document, or no assertions about agency science.
Update: If you want to comment on other aspects of this, go ahead. It is only for critiques of agency science that I want you to have actually been in contact with agency science.
posted by Megan at 12:16 PM
13 Comments:
I just read this whole thing and didn't understand a word of it. Now I feel dumb. I'm so glad I'm just a dumb guy who drinks coffee and tries to think of clever word plays and such. These things are too lofty for me.
Hello, Megan.
personal cred:
this is what i do for a living. As an attorney, I've co-written, reviewed and/or litigated EIRs (state), EISs (federal), bio. opinions, HCPs, HCP implementing agreements, THPs, 404 permits, WDRs, 1600 streambed alteration agreements, General Plans amendments, Specific Plans, zone changes and development agreements. (and i'm probably missing some.)
megan, I disagree with a few of things you wrote.
Delegation to the NAS: there are two ways of looking at the reference to NAS. One is the way you did, in which the NAS review is an insult to NOAA scientists.
Another is that NAS review can add legitimacy to the government's view of the science in a hotly disputed area.
As usual, the devil is in the details. It is possible to stack NAS review boards (which may have happened). And it is certainly possible for a vice president to deliberately mis-read the conclusions of a NAS review (which appears to have happened).
second, the God Squad is not an appeal process so much as a bypass process. The God Squad does not determine whether the bio. opinion is correct; instead it determines whether the national interest so outweighs the conclusions of the bio. opinion that the project should proceed.
third, federal district court review of a bio. opinion does not, except in the rarest of cases, involve expert testimony. Courts leave to the agencies the weighing of expert testimony. Courts only determine whether the agency followed statutory law and applicable regulation in coming to the conclusion it did.
Otherwise, as usual, very solid post.
Hello Scott. Now I feel terrible. I should have written that better. I read what you write every day and you're as smart as anyone I've read on the internet. Your posts go on some of the most interesting tangents I've ever seen and the love you send out is very often a rest and pause and comfort.
If you didn't follow that, I didn't write it right.
megan: men spell my name like this: Francis.
women spell it: Frances.
(here's the reminder -- the shape of the letter reflects the shape of the relevant anatomy.)
if you please, in the future refer to me as a "he".
Thank you!
Okay, where to start:
I have another idea teasing me, but I can’t get it to resolve fully. I wonder if Cheney wasn’t cynically taking advantage of science’s reliance on the null hypothesis and statistical significance.
This doesn't gel for me, either, and let me see if I can explain why. First of all, let's get the nitpick out of the way: I'm not sure statistical significance comes into play at all here; my impression, informed by this NAS representative's Congressional testimony, is that the NAS folk did a thorough literature review, NOT a meta-analysis or other quantitative aggregation of past results.
But more broadly speaking, it just doesn't make sense to me that Cheney would act in a way that presumes that he knows much more about the scientific process, or even the general NAS process for generating reports, than the general American population. He isn't, after all, educated as a scientist (whoops! wikipedia tells me he's a poly sci major -- and maybe they take a lot more stats classes at UWyoming than they did at my college, and maybe he retained all that through his Congress and DOD and Halliburton years). I guess Cheney might have suspected (IMO, correctly) that a group of scientists would be more inclined to maintain the correctness of the status quo line of thought(i.e., null hypothesis) than not. But my policy professor instructed told us that politicians never leave that kind of stuff to chance. I think Cheney, through whatever channels he has available, did exert some influence on the scientific panel. And let's be fair here: lit reviews, while an important part of scientific research, are hardly the basis off of which to make completely objective assessments.
I think NAS got used and that they weren’t clever enough see that they were doing damage.
My (mostly uninformed) theory: In the post-9/11 months, normal procedures to curtail governmental interference in NAS matters were not enforced, due to the heightened respect for the authority of the executive office. The Klamath River disaster might not now seem comparable to the Iraq war, but in late 2001/early 2002, I bet *everything* like this took on elevated importance in the guise of national security.
Francis: OK. I tend to assume people are women unless I have evidence otherwise. I'll fix that on the front page. I know other Francis/Frances people. I should have remembered.
Qbook:
You know, I was trying to stay away from assertions that Cheney actively tampered with or leaned on the panel; I think he could have gotten what he wanted without that. I was willing to give him credit for characterizing scientists as "not willing to say anything is true". But you're right, and it is a fine distinction. Still, you asked me to guess, and that was what came to mind.
It may be a formality, but I do not see the NAS study as introducing a new level of review or somehow violating or evading the ESA. When the NAS came to an interim conclusion different from the NOAA, there was pressure on the agency to review its own opinion, but I would expect that same sort of pressure to exist whenever a reputable researcher came to markedly different conclusion.
One question I still have is how often draft opinions get used in this way way. From your post, the NAS had only been studying things three months by the time the agency revised its opinion? Is it normal for a draft opinion to carry so much weight before much has been done on it?
HC:
There's no normal. It has never been done before or since. It was an extraordinary political maneuver, done to change the outcome of a legitimate ESA process.
Okay, first sentence: my experience with agency science is limited to having read an NRC report on information technology and counterterrorism.
I wonder if the prestige of the NAS played a role here? NAS is not a purely scientific organization. They're consciously interested in policy, and they use their prestige as a source of influence. I'm not saying this in a judgemental way; that's just how they operate, and it's pretty clean compared to other things in politics. But sometimes a NAS report carries more weight in an argument than it ought to.
In any case, this is a somewhat different group of people than in a university science department. They usually have some experience with how politicians will use and misuse their reports. In this case, I think they should have known better.
I understand that asking the NAS to study the subject of a biological opinion with an eye to getting a second opinion was novel, but the process whereby an agency incorporates new research into existing opinions seems like it would have to be business as usual as opposed to a brand new level of review. And, formally, that's what seems to have happened here - please do correct me where I err.
That's why I haven't seen this as introducing a new level of review, or evading the ESA, but rather taking existing procedures and twisting them into the desired shape.
but the process whereby an agency incorporates new research into existing opinions seems like it would have to be business as usual
OK, this was a Biological Opinion on an action that was imminent. That summer, Rclmtn was going to release water for fish or give it to farmers. Agency scientists had used the current science to complete a legitimate Biological Opinion (one that was proved accurate by a fish kill and then supported by the full NAS study).
Then Cheney took deliberate action to get it reversed so they could win a close election. It wasn't as if there were a new study about water temperature and salmon mortality that was shaking the fish science world. There was no new science that could have changed anything in the Biological Opinion. He didn't get a review to add anything new. He did it to undermine the legal process Congress requires the agencies to abide by.
Honestly, I don't know how agencies change their Biological Opinions to include new information. But I know this isn't it. They are not taking "existing procedures and twisting them into shape." There was no procedure that included the NAS OR high level politicians asking for reviews of legal outcomes they don't like, and Cheney refused the procedure that IS part of the law- the God Squad. It was a legal dodge, and that is bullshit from someone who is sworn to uphold the laws of this country.
Honestly, I don't know how agencies change their Biological Opinions to include new information.
ask and ye shall receive.
50 CFR 402.16 (Re-initiation of formal consultation).
tsk, tsk, Megan. You should know this considering that this is what's going on with the Delta smelt bio. opinion.
I don't work on Delta stuff. And I'm hired as an engineer these days. The law degree was a fluke.
Post a Comment
<< Home