TL;DR: Prediction & calibration parties are an exciting way for your EA/rationality group to practice rationality skills and celebrate the new year.
On December 30th, Seattle Rationality had a prediction party. Around 15 people showed up, brought snacks, brewed coffee, and spent several hours making predictions for 2017, and generating confidence levels for those predictions.
This was heavily inspired by Scott Alexander’s yearly predictions. (2014 results, 2015 results, 2016 predictions.) Our move was to turn this into a communal activity, with a few alterations to meet our needs and make it work better in a group.
Each person individually writes a bunch of predictions for the upcoming year. They can be about global events, people’s personal lives, etc.
If you use Scott Alexander’s system, create 5+ predictions for fixed confidence levels (50%, 60%, 70%, 80%, 90%, 95%, etc.)
Save your predictions and put it aside for 12 months.
Open up your predictions and see how everyone did.
To make this work in a group, we recommend the following:
Don’t share your confidence intervals. Avoid anchoring by just not naming how likely or unlikely you think any prediction is.
Do share predictions. Generating 30+ predictions is difficult, and sharing ideas (without confidence levels) makes it way easier to come up with a bunch. We made a shared google doc, and everyone pasted some of their predictions into it.
Make predictions that, in a year, will verifiably have happened or not. (IE, not “the academic year will go well”, which is debatable, but “I will finish the year with a 3.5 GPA or above”.)
It’s convenient to assume that unless stated otherwise predictions that end by the next year (IE, “I will go to the Bay Area” means “I will go to the Bay Area at least once in 2017.”) It’s also fine to make predictions that have other end dates (“I will go to EA Global this summer.”)
Make a bunch of predictions first without thinking too hard about how likely they are, then assign confidence levels. This post details why. You could also generate a group list of predictions, and everyone individually lists their own confidence levels.
This makes a good activity for rationality/EA groups for the following reasons:
Practicing rationality skills:
Making accurate predictions
Using confidence intervals
It’s open to many different knowledge levels. Even if you don’t know a thing about geopolitics, you can still give predictions and confidence intervals about media, sports, or your own life.
More free-form and less intimidating than using a prediction market. You do not have to know about the details of forecasting to try this.
Natural time and recurring activity
You could do this at any point during the year, but doing it at the start of the year seems appropriate for ringing in the new year.
In twelve months, you have an automatic new activity, which is coming back together and checking everybody’s predictions from last year. Then you make a new set of predictions for next year. (If this falls through for some reason, everyone can, of course, still check their predictions on their own.)
Fostering a friendly sense of competitiveness
Everyone wants to have the best calibration, or the lowest Brier score. Everyone wants to have the most accurate predictions!
Brier values and graphs of ‘perfect’ vs. actual scores will give you different information. Alexander writes about the differences between these. Several of us did predictions last year using the Scott Alexander method (bins at fixed probabilities), although this year, everybody seems to have used continuous probabilities. The exact method by which we’ll determine how well-calibrated we were will be left to Seattle Rationality of 2018, but will probably include Brier values AND something to determine calibration.
As I write this, it’s 4:24 PM in 2016, twelve days before the darkest day of the year. The sun has just set, but you’d be hard-pressed to tell behind the heavy layer of marbled gray cloud. There’s a dusting of snow on the lawns and the trees, and clumps on roofs, already melted off the roads by a day of rain. From my window, I can see lights glimmering in Seattle’s International District, and buildings of downtown are starting to glow with flashing reds, neon bands on the Colombia Tower, and soft yellow on a thousand office windows. I’m starting to wonder what to eat for dinner.
It’s the eve before Seattle Effective Altruism’s Secular Solstice, a somewhat magical humanist celebration of our dark universe and the light in it. This year, our theme is global agriculture – our age-old answer to the question of “what are we, as a civilization, collectively going to eat for dinner?” We have not always had good answers to this question.
Civilization, culture, and the super-colony of humanity, the city, started getting really big when agriculture was invented, when we could concentrate a bunch of people in one place and specialize. It wasn’t much specialization, at first. Farmers or hunter-gatherers were the vast majority of the population and the population of Ur, the largest city on earth, was around 65,000 people in 3000 BC. Today, farmers are 40% of the global population, and 2% in the US. In the 1890’s, the city of Shanghai had half a million people. Today, it’s the world’s largest city, with 34 million residents.
What happened in those 120 years, or even the last 5000?
I’m a scientist, so the people I know of are scientists, and science is what’s shaped a lot of our agriculture in the last hundred years. When I think of the legacy of science and global agriculture, of people trying to figure out how we feed everyone, I think of three people, and I’ll talk about them here. I’ll go in chronological order, because it’s the order things go in already.
Fritz Haber, 1868-1934
Haber was raised in a Jewish family in Prussia, but converted to Lutheranism after getting his doctorate in chemistry – possibly to improve his odds of getting high-ranking academic or military careers. At the University of Kulroch in Germany, Haber and his assistant Robert Le Rossignol did the work that won them a Nobel prize: they invented the Haber-Bosch process.
The chemistry of this reaction is pretty simple – it was a fact of chemistry at the time that if you added ammonia to a nickel catalyst, the ammonia decomposed into hydrogen and nitrogen. Haber’s twist was to reverse it – by adding enough hydrogen and nitrogen gas at a high pressure and temperature, the catalyst operates in reverse and combines the two into ammonia. Hydrogen is made from natural gas (CH4, or methane), and nitrogen gas is already 80% of the atmosphere.
Here’s the thing – plants love nitrogen. Nitrogen is, largely, the limiting factor in land plants’ growth – when you see that plants aren’t growing like mad, it’s because they don’t have sufficient nitrogen to make new proteins. When you give a plant nitrogen in a form it can assimilate, like ammonia, it grows like mad. The world’s natural solid ammonia deposits were being stripped away to nothing, applied to crops to feed a growing population.
When Haber invented his process in 1909, ammonia became cheap. A tide was turning. The limiting factor of the world’s agriculture was suddenly no longer limiting.
Other tides were turning too. In 1914, Germany went to war, and Haber went to work on chemical weapons.
During peace time a scientist belongs to the World, but during war time he belongs to his country. – Fritz Haber
He studied deploying chlorine gas, thinking that it would shorten the war. Its effect is described as “drowning on dry land”. After its first use on the battlefield, he received a promotion on the same night his wife killed herself. Clara Immerwahr, a fellow chemist, was a pacifist, and had shot herself with Haber’s military pistol. Haber continued his work. Scientists in his employ also eventually invented Zykkon B. First designed as a pesticide, after his death, the gas would be used to murder his extended family (along with many others) in the Nazi gas chambers.
Anti-Jewish sentiment was growing in the last few years of his life. In 1933, he wasn’t allowed through the doors of his institute. The same year, his friend, and fellow German Jewish scientist, Albert Einstein, went to the German Consulate in Belgium and gave them back his passport – renouncing his citizenship of the Nazi-controlled government. Haber left the country, and then died of a heart attack, in the next year.
I don’t know if Fritz Haber’s story has a moral. Einstein wrote about his colleague that “Haber’s life was the tragedy of the German Jew – the tragedy of unrequited love.” Haber was said to ‘make bread from air’ and said to be the father of chemical weapons. He certainly created horrors. What I might take from it more generally is that the future isn’t determined by whether people are good or bad, or altruistic or not, but by what they do, as well as what happens to the work that they do.
Nikolai Vavilov – 1887-1943
We shall go into the pyre, we shall burn… But we shall not abandon our convictions. – Nikolai Vavilov
As a young but wildly talented agronomist in Russia, the director of the Lenin All-Union Academy of Agricultural Sciences for over a decade, the shrewd and charismatic Nikolai Vavilov, wanted to make Russia unprecedented experts in agriculture. He went on a series of trips to travel the globe and retrieve samples. He observed that in certain parts of the world, one would find a much greater variety of a given crop species, with a wider range of characteristics and traits not seen elsewhere. This lead to his breakthrough theory, his Vavilov centers of diversity, that the greatest genetic diversity could be found where a species originated.
What has this told us about agriculture? This morning for breakfast, I had coffee (originally from Ethiopia) with soy milk (soybeans originally from China), toast (wheat from the Middle East) with margarine (soy oil, China, palm oil, West and Southwest Africa), and chickpeas (Central Asia) with black bean sauce (central or possibly South America) and pepper (India). One fairly typical vegan breakfast, seven centers of diversity.
He traveled to twelve Vavilov centers, regions where the world’s food species were originally cultivated. He traveled in remote regions of the world, gathering unique wheat and rye in the Hindu Kush, Spain, and Portugal, teff in Somalia, sugar beet and flax in the Mediterranean, potatoes in Peru, fava beans and pomegranates and hemp in Herat. He was robbed by bandits in Eritrea, and nearly died riding horseback along deep ravines in the Pamirs. The seeds he gathered were studied carefully back in Russia, tested in fields, and most importantly, cataloged and stored – by gathering a library of genetic diversity, Vavilov knew he was creating a resource that could be used to grow plants that would suit the country’s needs for decades to come. If a pest decimates one crop, you can find a resistant crop and plant it instead. If drought kills your rice, all you need to do is find a drought-tolerant strain of rice. At the Pavlovsk Experimental Research Station, Vavilov was building the world’s first seed bank.
In Afghanistan, he saw wild rye intermingled with wheat in the fields, and used this as evidence of the origin of cultivated rye: that it wasn’t originally grown intentionally the way wheat or barley had been, but that it was a wheat mimic that had slipped into farms and taken advantage of the nurturing protection of human farmers, and had, almost accidentally, become popular food plants at the same time. Other Vavilovian mimics are oats and Camelina sativa.
While he travelled the world and became famous around the burgeoning global scientific community, Russia was changing. Stalin had taken over the government. He was collectivizing the farms of the country, and in the scientific academies, were dismissing staff based on bourgeois origin and increasing the focus on practical importance of work for the good of the people. A former peasant was working his way up through agricultural institutions: Trofim Lysenko, whose claimed that his theory of ‘vernalization’, or adapting winter crops to behave more like summer crops by treating the seeds with heat, would grow impossible quantities of food and solve hunger in Russia. Agricultural science was politicized in a way that it never had been – Mendelian genetics and the existence of chromosomes were seen as unacceptably reactionary and foreign. Instead, a sort of bastardized Lamarckism was popular – aside from being used by Lysenko to justify outrageous promises of future harvests that never quite came in, it said that every organism could improve its own position – a politically popular implication, but one which failed to hold up to experimental evidence.
Vavilov’s requests to leave the country were denied. His fervent Mendelianism and the way he fraternized with Western scientists were deeply suspicious to the ruling party. As his more resistant colleagues were arrested around him, his institute filled up with Lysenkoists, and his work was gutted. Vavilov refused to denounce Darwinism. Crops around Russia were failing under the new farming plans, and people starved as Germany invaded.
Vavilov’s devoted colleagues and students kept up his work. In 1941, the German Army reached the Pavlovsk Experimental Research Station, interested in seizing the valuable samples within – only to find it barren.
Vavilov’s colleagues had taken all 250,000 seeds in the collection by train into Leningrad. There, they hid them in the basement of an art museum and watched them in shifts all throughout the Siege of Leningrad. They saw themselves as protecting Russia’s future in agriculture. When the siege lifted in 1944, twelve of Vavilov’s scientists had starved to death rather than eat the edible seeds they guarded. Vavilov’s collection survived the war.
Gardening has many saints, but few martyrs. – T. Kingfisher
In 1940, Vavilov was arrested, and tortured in prison until he confessed to a variety of crimes against the state that he certainly never committed.
He survived for three years in the gulag. The German army advanced on Russia and terrorized the state. Vavilov, the man who had dreamed of feeding Russia, starved to death in prison in the spring of 1943. His seed bank still exists.
Vavilov’s moral, to me, is this: Science can’t be allowed to become politicized. Whatever the facts are, we have to build our beliefs around them, never the other way around.
Norman Borlaug, 1914-2009
Borlaug was raised on a family farm to Norwegian immigrants in Iowa. He studied crop pests, and had to take regular breaks from his education to work: He worked in the Civilian Conservation Corps during the dustbowl alongside starving men, and for the Forest Service in remote parts of the country. In World War 2, he worked on adhesives and other compounds for the US MIlitary. In 1944, he worked on a project sponsored by the Rockefeller Foundation and the Mexican Ministry of Agriculture to improve Mexico’s wheat yields and stop it from having to import most of its grain. The project faced opposition from local farmers, mostly because wheat rust had been killing their crops. This wasn’t an entirely unique problem – populations were growing globally. Biologist Paul Erlich wrote in 1968, “The battle to feed all of humanity is over … In the 1970s and 1980s hundreds of millions of people will starve to death in spite of any crash programs embarked upon now.”
Borlaug realized that by harvesting seeds in one part of the country and quickly moving them to another, the government could take advantage of the country’s two growing seasons and double the harvest.
By breeding many wheat strains together, farmers could make crops resistant to many more diseases.
He spread the use of Haber’s ammonia fertilizers, and bred special semi-dwarf strains of wheat that held up to heavy wheat heads without bending, and grew better in nitrogen fertilizers.
Nine years later, Mexico’s wheat harvest was six times larger than it had been in 1944, and it had enough wheat to export.
Borlaug was sent to India in 1962, and along with Mankombu S. Swaminathan, they did it again. India was at war, dealing with famine and starvation, and was importing necessary grain for survival. They used Borlaug’s strains, and by 1968, were growing so much wheat that the infrastructure couldn’t handle it. Schoolhouses were converted into granaries.
His techniques spread. Wheat yields doubled in Pakistan. Wheat yields in the world’s least developed countries doubled. Borlaug’s colleagues used the same process on rice, and created cultivars that were used all over Asia. Borlaug saw a world devastated by starvation, recognized it for what it was, and treated it as a solvable problem. He took Haber’s mixed legacy and put it to work for humanity. Today, he’s known as the father of the Green Revolution, and his work is estimated to have saved a billion lives.
We would like his life to be a model for making a difference in the lives of others and to bring about efforts to end human misery for all mankind. – Statement from Borlaug’s children following his death
When I think of modern global agriculture, this is who I think of. I’ve been trying to find something connecting Vavilov and the Green Revolution, and haven’t turned up much – although it’s quite conceivable there is, given Vavilov’s inspirational presence and the way he shared his samples throughout the globe. Borlaug’s prize wheat strain that saved those billion lives, Norin 10-Brevor 14, was a cross between Japanese and Washingtonian wheat. Past that, who knows?
One of the organizations protecting crop diversity today is the Consultative Group for International Agricultural Research (CGIAR), which was founded in 1971 by the Rockefeller Foundation as the Green Revolution was in full swing. They operate a variety of research stations worldwide, mostly at Vavilov Centers in the global south where crop diversity is highest. Their mission is to reduce global poverty, improve health, manage natural resources, and increase food security.
They must have been inspired by Vavilov’s conviction that crop diversity is essential for a secure food supply. If a legacy that’s saved literally a billion human lives can be said to have a downside, it’s that diets were probably more diverse before, and now 12 species make up 75% of our food plant supply. Monocultures are fragile, and if conditions change, a single disease is more likely to take out all of a crop.
In 2008, CGIAR brought the first seed samples into the Svalbard Seed Vault – a concrete structure buried in the permafrost. It’s constructed as a refuge against whatever the world might throw. If electricity goes out, the permafrost will keep the seeds cool. If sea levels rise, the vault is built on a hill. The land it’s on is geologically stable and very remote. And it stores 1,500,000 seeds – six times more than Vavilov’s 250,000 – at no cost to countries that use it.
Let it be known: starvation is on its last legs. We have a good thing going here. Still, with global warming and worse things still looming over the shoulder of this tentative victory, let’s give thanks to the movers and shakers of global agriculture for tomorrow: the people ensuring that whatever happens next, we are going to be fed.