How much of a collective intelligence we are? The case of cities and agricultural land


I continue to work on the role of cities in our civilisation and on the changes that the current COVID-19 pandemic can possibly bring to our ways of living in cities. Initially, when I started writing this update, on June 6th, I intended to explore the connection between technological change and the civilizational role of cities. Further in this update, I do go down that avenue, yet for now, in those initial paragraphs, I want to share another strand of my thinking, which I already signalled last time, namely my impressions from reading Daniel Defoe’s ‘Journal of The Plague Year’, published in 1665. That book was published in 1665 and gives the account of events which took place in London, in 1664, during the epidemic outbreak of plague. It was 356 years ago, and yet, when I read it, especially the initial chapters, I have the impression of going through news feeds from the last 4 months, like from February until now, of course in relation to the COVID-19 pandemic. The sequence of events described by Daniel Defoe, the patterns of human reactions to the epidemic disease – all that is so incredibly similar to what we experience today that I have hard times to realize that what Daniel Defoe described took place 18 generations ago (if we count 25 years for one generational shift, by sociological standards).

That striking similarity gives tons of hope. Eighteen generations ago, people had just a small fraction of science and technology that we have today, and yet they pushed themselves through that deep shit, and there was another sunrise. It was plague, not COVID-19. It was a monster. Yet, there was another sunrise. What impressed me the most, I think, is the very end of that book, and I allow myself to quote it: “It was a common thing to meet people in the street that were strangers, and that we knew nothing at all of, expressing their surprise. Going one day through Aldgate, and a pretty many people being passing and repassing, there comes a man out of the end of the Minories, and looking a little up the street and down, he throws his hands abroad, ‘Lord, what an alteration is here! Why, last week I came along here, and hardly anybody was to be seen.’ Another man—I heard him—adds to his words, ‘’Tis all wonderful; ’tis all a dream.’ ‘Blessed be God,’ says a third man, and and let us give thanks to Him, for ’tis all His own doing, human help and human skill was at an end.’ These were all strangers to one another. But such salutations as these were frequent in the street every day; and in spite of a loose behaviour, the very common people went along the streets giving God thanks for their deliverance. It was now, as I said before, the people had cast off all apprehensions, and that too fast; indeed we were no more afraid now to pass by a man with a white cap upon his head, or with a cloth wrapt round his neck, or with his leg limping, occasioned by the sores in his groin, all which were frightful to the last degree, but the week before. But now the street was full of them, and these poor recovering creatures, give them their due, appeared very sensible of their unexpected deliverance; and I should wrong them very much if I should not acknowledge that I believe many of them were really thankful”.  (Excerpt From: Daniel Defoe. “A Journal of the Plague Year / Written by a Citizen Who Continued All the While in London”. Apple Books.”)

As I am rereading that book by Daniel Defoe, and as I meditate over it, I realize how bloody tough we, humans, are. The city of London (where the events described by Daniel Defoe take place) is still there. It is thriving. We moan, we bicker, we take grand moral stances over events we don’t even have full knowledge about, and yet, at the bottom line, when the shit hits the fan, we just clench our teeth, dig our heels into the sand, and survive. Wonderful.

I am going into a slightly different path of thinking, as compared to my recent updates. The initial hypothesis of that entire thread of research is that technological change that has been going on in our civilisation at least since 1960 is oriented on increasing urbanization of humanity, and more specifically on effective, rigid partition between urban areas and rural ones. I focus on the connection between cities and the countryside, at the aggregate level. In Figure 1, below, you can see indexed trends in three aggregate variables: a) density of urban population denominated in units of general density in population (which I will further designate, for the sake of presentational convenience, as [DU/DG]) b) cereal yield in kg per hectare, and c) total surface of agricultural land. In order to assure comparability, I represented all those three metrics as constant-base indexes, where values from the year 2000 make 1.

As you can see, I provided direct links (to the database of the World Bank) as regards two variables out of the three. I did it because the first variable is a compound construct of my own, made out of primary data supplied by the World Bank. I took the numbers regarding aggregate urban population, and I divided it by the aggregate surface of urban land, which yields the coefficient of density in urban population. In the next step, I want to use that coefficient so as to measure the relative social difference between cities and the countryside. In order to do so, I divide the coefficient of density in urban population by the coefficient of general density in population. In other words, I check how many general densities of population we need in order to have one unit of density in urban population.

Since 1961 through 2016, the relative social distance between cities and the countryside, measured at the planetary level, has been growing steadily, almost in a straight line. As a matter of fact, that line is so straight that it is hardly believable. When you find a straight line of trend, which sort of cuts across waves and bumps in other variables, you are either completely wrong or deeply right. Linear change over time is a rare beast in the realm of measurable phenomena. However, as I measure local growth rates in that [DU/DG] metric, they keep sticking to 1% a year. Yes, since 1961, the average social distance between cities and the countryside has been growing at a nearly constant rate of 1% a year.

Against that almost suspiciously consistent change in the density of urban populations across the planet, agriculture has been changing at two different speeds. Cereal yield per hectare has grown, at the end of the day, yet its growth has been happening at a much more familiar, bumpy rate, sort of two steps forward, one step back. The aggregate surface of agricultural land presents a stairway type of change: two plateaus separated by a sudden jump in the beginning of the 1990ies.

Summing up, as social density in cities has been hyper-consistently drifting away and above general social density, agriculture kept adapting, mostly by consistent growth in agricultural productivity. Interestingly, all three trends, although different in shape, are strongly correlated, which is shown in Table 1, below Figure 1. Those correlations are so strong that it all looks like one compound phenomenon, with just a little entropy inside.  

As local expansions of agricultural land have kept happening, yet they also kept being compensated, at the global scale, by decreases in other parts of the world. On the long run, between 1961 and 2016, the total surface of agricultural land in the world has grown by 11,4 millions of square kilometres. Apparently, more than 55% of that aggregate growth happened in the short window between 1989 and 1992 seems to be only moment since 1961 when the total global surface of agricultural land unequivocally went up. That big leap in agricultural land, by about 6,3 millions of additional square kilometres, happened mostly in countries classified as ‘Middle Income’, and was prevalently concentrated in two of them: Kazakhstan and Russian Federation. The long-term geography of change in agricultural land, between 1961 and 2016, is shown in the form of a map in Figure 2. Kazakhstan, Russian Federation and China keep the podium. A freakish idea comes to my mind. Between 1989 and 1992, a dramatic increase happened in the surface of agricultural land on the planet. It happened mostly in the former Soviet Union, which, precisely then, was dissolving. Are the two phenomena connected? Is it possible that the dissolution of the biggest country in the world was a collectively intelligent response of our planetary human species to the necessity of having more land to grow food?  

Figure 1

Table 1 – Pearson correlation between density of urban population, agricultural land, and cereal yield per hectare

 Density of urban population, denominated in units of general density in population: World
Surface of agricultural land, km2 : World0,927149105
Cereal yield, kg per hectare of arable land: World0,984881004

Figure 2

Now, I focus on the ‘technological change’ part and I formulate two other hypotheses. Firstly, I claim that technological change manifests collective intelligence in human societies. Secondly, Artificial Intelligence, the development of which marks technological change of the last two decades, emulates collective intelligence much more than individual one.

Why do I claim at all that technological change manifests collective human intelligence? Isn’t it rather individual intelligence saying, at some point in time, something like ‘Enough! Enough of those stupid sleighs. We need wheels!’? It is true to some extent, more specifically to the extent that individually expressed ideas really push technology forward. Still, those ideas work similarly to the way that a ball is being played in a team game. When we play basketball, most individual actions with the ball are effective and efficient only to the extent of cooperation from the part of other players in the team. An innovative idea is like that ball: its needs to be passed around and collectively played.

Collective intelligence can be described as the ability to collectively figure out what to do when we collectively have no clue what to do. This is a very synthetic description of mechanisms which require a deeper insight. We collectively experience problems when we share collective beliefs, acceptably grounded in empirical facts, that something happens the way most of us doesn’t want it to happen. This is the gap between expectations and reality. Collective experience is that something doesn’t work as we would like it to work.

Now, let’s introduce the distinction between simple discomfort with reality, on the one hand, and the experience of inefficiency in our behaviour, on the other hand. Life is brutal, in general. Yes, it is beautiful as well, and yet we experience beauty largely by opposition to ugliness. We perceive the brutal beauty of existence mostly as gradients of change, and not really as absolute states of things (see, for example: We really don’t see small change). We are uncomfortable with some changes in reality, and sometimes that discomfort triggers collectively coordinated action. That’s the first moment of assessment as regards us being collectively smart: can we coordinate to take action, or cannot we? The next level is being efficient in that action. Have we achieved the results we expected to achieve?

We have two levels of collective ambition, whence two possible levels of collective frustration, namely with the failure to coordinate, or with the insufficient outcomes of coordination. Both failures incite to do something about that imperfect social coordination of ours. When we dig a bit into the depth of the problem, we usually discover at least one of the two things: we are either too rigid or too random in coupling individual behaviours into the beautiful dance of well-rounded teamwork. Too rigid means that person A always does what person B expects them to do, whence nearly perfect a stationarity of their common action. Too much randomness manifests as the person A hardly ever doing what person B expects them to do, whence a well-understandable frustration in the person B and a lack of trust in coordination.

Good coordination relies on a behavioural pattern called correlated coupling, which manifests as the person A responding flexibly and yet predictably to signals sent by person B, and vice versa. Being both flexible and predictable in my response to other people’s signals means that my own action takes a recurrent form – which sends other people the reassuring signal ‘I get it, guys, carry on’ – and yet that form is somehow scalable. When I am an engineer and my boss asks me ‘to give that engine a bit of nerve’, he or she can trust – if my behaviour is correlatedly coupled with theirs – that I will come up with a range of possible solutions for said nerve, and I will select the most appropriate.

We are collectively intelligent when, as a collective, we have the ability to spot recurrent cases of too rigid behavioural coupling, or too much randomness in collective coordination, and transform those situations into correlated coupling. Let’s take the example of a simple, old technology: the use of windmills to power querns, instead of grinding cereal grain by hand. I think it was some 20 years ago: I messed around a bit with a reconstructed, man-powered quern (you now, two flat stones on a common rotating axis), just to see how it felt, centuries ago. When I gave it a try, I understood why the baking of bread became really widespread across Europe only with the diffusion of windmills and watermills. Grinding grain into flour by the sheer force of human muscle is, at the end of the day, a zero-sum activity, energy-wise. You burn approximately as much energy when grinding as you can have from the flour you obtain.    

Technologies give us flexibility and predictability. The wind-powered quern, back in the day, as compared to the man-powered one, assured smooth grinding of grain and scalability: faster or slower, greater quantity per day or a smaller one etc. Technologies allow replacing fixed coupling in behaviour, or a random one, with the functional elegance of correlated coupling.

Now, let’s get into the process of implementing new technologies. When I do it individually, it is a sequence of trials and errors. I try something, and it works smoothly, it works just sort of, or it doesn’t work at all. Depending on the exact outcome, either I say ‘Hooray! Nailed it!’, or I go ‘Well, it needs some improvement’, or, finally, I say things I could be ashamed, later on, of having said at all. When I need improvement, it slows me down, obviously. Still, when my new contrivance seems to be working just perfectly, it can slow me down even more. I am satisfied with immediate outcomes, and a prolonged chain of satisfactory results can prevent me from seeing an entirely different, alternative way of doing things.

When lots of ‘I’ do the same thing – after all, each human is an ‘I’ – it is different. Each ‘I’ comes up with somehow different results, and these can be instantaneously compared. The ‘I’s which do the best job stick out of the crowd. Their ways are likely to be reproduced by other people, whilst the clearly suboptimal methods fall into oblivion. Many humans experimenting with solutions to the same problem are like as many living organisms attempting to mutate in the presence of an exogenous stressor. The more organisms experiment with themselves, the greater the likelihood of successful mutations. In biology, this mechanism is called ‘adaptive walk in rugged landscape’ and can be applied in social sciences. When many social entities experiment with themselves in order to cope with an exogenous pressure, such as the pressure to survive or to climb the ladder of social hierarchy, some of those entities (e.g. persons, businesses, political parties) are more successful than others. Best practices are retained and reproduced in the future. This is collective intelligence in solving collective problems.

Cities facilitate technological change because they reinforce that sort of next-to-my-neighbour innovation. In cities, due to high density of population, it is simply easier to observe others, to emulate their successes or steer clear of the way they failed. It is easier to navigate through the muddy waters between conformism and individuation. Cities are instances of enhanced collective intelligence.

I use simple neural networks to emulate the way that our human collective intelligence works. I used it in this specific thread of research (see The perfectly dumb, smart social structure), you can find it in a published article on energy efficiency, and in another, unpublished paper. As I keep meddling with neural networks, I am more and more convinced that artificial intelligence emulates the collective intelligence of human societies much more than individual intelligence of one human being. Why do I make such a claim? Because neural networks work well when they can experiment with quasi-random combinations of weights assigned to many input variables, i.e. many different phenomena. With just one input variable, a neural network usually goes completely bananas. No learning whatsoever. The necessity of multiple input makes me think about many social entities trying to do something, rather than just one human.     

Discover Social Sciences is a scientific blog, which I, Krzysztof Wasniewski, individually write and manage. If you enjoy the content I create, you can choose to support my work, with a symbolic $1, or whatever other amount you please, via MY PAYPAL ACCOUNT.  What you will contribute to will be almost exactly what you can read now. I have been blogging since 2017, and I think I have a pretty clearly rounded style.

In the bottom on the sidebar of the main page, you can access the archives of that blog, all the way back to August 2017. You can make yourself an idea how I work, what do I work on and how has my writing evolved. If you like social sciences served in this specific sauce, I will be grateful for your support to my research and writing.

‘Discover Social Sciences’ is a continuous endeavour and is mostly made of my personal energy and work. There are minor expenses, to cover the current costs of maintaining the website, or to collect data, yet I want to be honest: by supporting ‘Discover Social Sciences’, you will be mostly supporting my continuous stream of writing and online publishing. As you read through the stream of my updates on , you can see that I usually write 1 – 3 updates a week, and this is the pace of writing that you can expect from me.

Besides the continuous stream of writing which I provide to my readers, there are some more durable takeaways. One of them is an e-book which I published in 2017, ‘Capitalism And Political Power’. Normally, it is available with the publisher, the Scholar publishing house ( ). Via , you can download that e-book for free.

Another takeaway you can be interested in is ‘The Business Planning Calculator’, an Excel-based, simple tool for financial calculations needed when building a business plan.

Both the e-book and the calculator are available via links in the top right corner of the main page on .

You might be interested Virtual Summer Camps, as well. These are free, half-day summer camps will be a week-long, with enrichment-based classes in subjects like foreign languages, chess, theater, coding, Minecraft, how to be a detective, photography and more. These live, interactive classes will be taught by expert instructors vetted through Varsity Tutors’ platform. We already have 200 camps scheduled for the summer. .

Leave a Reply