How should you form your mental map of reality?

Feel free to skip the plodding generalities and jump to the local content of this post.


This question has been coming up a lot lately - with the CBS News-Dan Rather letters, with Iraq, with Swiftboats, with The Union.

Unfortunately, it doesn't have a simple answer. If you rely solely on your own experiences and perceptions, your reality map might end up being extremely accurate, but it will take form extremely slowly - this is like insisting on building the house yourself before you move in, cutting the trees for the logs for the beams yourself, forging your own tools...In short, it's neither desirable nor attainable. We need others to provide us with the parts we need - we need to outsource much of our decisionmaking, we need others to inform us.

When we put the job of "Information provider" out for bid, we'll have plenty of takers: we can hire consultants, ask friends, get told by busybodies, read weblogs, read newspapers...but then we're faced with a new problem - who will do the best job? Some of them aren't competent, and can't do as good a job as well as we would (if we had the time) (of these, some can hide it well, some can't); some - perhaps serving two masters - might have their own reasons for not doing it properly; on the other end the spectrum, many are experts, who know the lay of the land far better than we and who can effortlessly produce top quality results.

How do we judge who to hire? We want the ones who are honest and have a good track record, who are sharp, who use good tools, who are dedicated, who are straight with us about what they can and can't provide. While they might not be able to provide exactly what we want, they shouldn't engage in deliberate omissions or misinformation.
[Obvious disclaimer: this is an ideal, nobody's perfect, everything's relative, etc.]

How can we judge whether they have these qualities?

One way is to listen to how they say they'll inform us, and see if it sounds like what we want. This method has two drawbacks - first, what they may mean might not be what we hear, and second, they may have the words down but be less strong on the execution, i.e. inflate their qualities beyond what's real. (in the absence of an objective metric for evaluation - or a good referee - this is bound to happen, like grade inflation in universities.)

So the words are good, but they're not enough. We need to look at how they do, not just how they talk.

But how can we know how they do? they're the experts, we're not.

Here are three methods:

• Outsource the judging - in effect, hire a widely acknowledged (or at least disinterested) expert - perhaps an ombudsman, or a book reviewer, or an accredited institution - to judge the performance of our expert.
  
• Back-seat driving - try, through reading, to learn about what our expert's results should look like, and try to judge based on that. But here, we don't know if we've learned enough to judge properly.
  
• Calibrate - see what the expert says about, and how s/he handles, a case that you understand well. Here you can measure how well they perform, and extrapolate from this measurement to infer how well they're likely to do on cases that you don't understand well.

---

The problem I had with the North Star Controversy involved judging the experts - I didn't know who to believe.

On the one hand, Richard Somerville and Jeff Ackerman, editor and publisher of The Union respectively, appear steadfast in their conviction that Conklin behaved wrongly, and - I infer, since I haven't seen them retract it - in their conviction that he and the Land Trust hatched a deal ("How the plans were laid") long before Conklin got the job. If a newspaper editor is convinced, you'd think that this fact shouldn't be treated lightly.

On the other hand, Bruce Conklin (who I think I've only met once) appears - by reputation from people I respect, and by all actions that I know of - to have the utmost integrity.

On the other hand, money is a powerful motivator.

On the other hand (hoof?), politics as practiced by certain parties up here in Nevada County are far from clean; if some of Conklin's opponents had an opportunity to 'take him out' by smearing, they'd do it.

What initially tipped the balance for me was having an ideal opportunity to outsource the judging - if Andy Cassano, a 'widely acknowledged expert' (i.e. an ethical person who knows the participants well) says the Land Trust behaved completely ethically, that's convincing.

And yet, accepting Cassano's view would entail inferring that Somerville and Ackerman have been misled or are poorly informed or deceptive, and - having never met them at all, except via email - that's not a conclusion I would want to make.

So your correspondent would be an utter waffler, except that The Union's website appears to sheds some light, by enabling calibration.

Go to The Union's North Star Controversy 'front page'. Do you see a veritable plethora of articles, all - with the exception of Conklin's "I did nothing wrong""We made a wise decision" - forming a uniformly disapproving chorus?

Do you see the piece from Andy Cassano, headlined "Political naivete by Land Trust"?

Read the article. Does the headline in any sense match the thrust of Cassano's piece?

Answer: No. However, it's just one item, copy editors are rushed, let's not be so distrustful.


I asked The Union's editor in email:
>Do you feel that this representation of Cassano's article is appropriate?

was told:
...that was the headline on his column, based on his key statement...In this case, we wrote a head that we felt represented his key point

Look at the piece again, and see if you think that Cassano's key point was that the Land Trust was politically naive.

Everyone makes mistakes and omissions, but usually they don't influence - negatively - how a community perceives one of its most prominent members. I believe The Union seriously misrepresented Cassano's article in their overview page - which will mislead the casual reader into assuming that the condemnation is unanimous. And please use this opportunity to calibrate my credibility too- as Jay Rosen said today, "This is the great thing about blogging, seems to me. You can judge [for] yourself". And, if I'm way off base, you can infer that the rest of this post - and the weblog as a whole - is probably garbage.

Because of this misrepresentation, I find it harder to put a charitable spin on The Union's not showing any of the flood of anti-Union-coverage-of-Conklin letters on their "single day" pages (you can specify on the Search page that, for example, you just want to see - online - what was in the 'paper' paper on Sept 25, 2004; but if it's a letter about this coverage, you won't find it there). It turns out these are considered "election letters" - and, in a new setup, these letters go into a separate ghettosection over on the "Election" page, where the casual reader is extremely unlikely to find them. Plus, after they scroll off the bottom of the page, there's no "next page" link (AFAIK) by which you can reach them; so, unless you do a Search for keywords (which fortunately still works, at least for recent items) you won't be able to find them.

Editor R.S. says: "All election letters can be found online under the opinion page, through the election page, or by typing [keywords] in the search engine."

Will look into this; I could be wrong, but this (availability under the Opinion page) might be new. In any case though,
It appears that the "Union criticism" letters are still only visible for a couple of days even on the Opinion page, after that being invisible unless you know what keywords to search for.

Maybe the fact that letters critical of The Union's coverage get hidden away more than 'normal' letters do is a feature. Maybe their becoming invisible to the casual browser after a few days is an oversight. Maybe the fact that the standard Archive Search doesn't bring up salient articles is (as I'm told ) a technical problem. Maybe for The Union to tell the Search users that it's broken would be "too much information". Maybe the title of Cassano's piece just happened to get written in a negative way, because The Union staff misread the column.

Maybe they don't think it's important to fix.

Maybe it's forming a pattern.


I don't want to be claiming that The Union's management sit around in the back room smoking cigars and laughing diabolically as they plot how to bring down their customers' political opponents - it's more that the absence of consideration for fairness, as shown by their disinterest in rectifying the misimpression that their oversights have produced, is likely to have affected (or effected) the original series of articles as well.

Oct 16: OK, not effected - the rapid slide into invisibility of critical letters was probably not deliberate, but to be complacent about this situation and allow it to remain thus, when a man's reputation is in the balance, is not responsible behavior.


Oct 13: to belatedly mention the most important point: we the readers have yet to see a clear, concise account - with evidence to support it - as to exactly what "unethical behavior" was engaged in.