The possible Black Swans

I am re-digesting, like a cow, some of the intellectual food I figured out recently. I return to the specific strand of my research to be found in the unpublished manuscript ‘The Puzzle of Urban Density And Energy Consumption’, and I want to rummage a bit inside one specific issue, namely the meaning which I can attach to the neural activation function in the quantitative method I use.

Just to give a quick sketch of the landscape, I work through a general hypothesis that our human civilization is based on two factories: the factory of food in the countryside, and the factory of new social roles in cities. The latter produce new social roles by creating demographic anomalies, i.e. by packing humans tightly together, in abnormally high density. Being dense together makes us interact more with each other, which, whilst not always pleasant, stimulates our social brains and makes us figure out new interesting s**t, i.e. new social roles.

I made a metric of density in population, which is a coefficient derived from the available data of the World Bank. I took the coefficient of urbanization (World Bank 1[1]), and I multiplied it by the headcount of population (World Bank 4[2]). This is how I got the number of people living in cities. I divided it by the surface of urban land (World Bank 2[3]), and I got the density of population in cities, which I further label as ‘DU’. Further, I gather that the social difference between cities and the countryside, hence the relative impact of cities as breeding ground for new social roles, is determined by the difference in the depth of demographic anomalies created by the urban density of population. Therefore, I took the just-calculated coefficient DU and I divided it by the general density of population, or ‘DG’ (World Bank 5[4]). This is how I ended up the with the coefficient ‘DU/DG’, which, mathematically, denominates the density of urban population in units of general density in population.

I simulate an artificial reality, where we, humans, optimize the coefficient ‘DU/DG’ as our chief collective orientation. We just want to get it right. Enough human density in cities to be creative, and yet enough space for each human being able to practice mindfulness when taking a #2 in the toilet. We optimize ourselves being dense together in cities on the base of 7 input characteristics of ours, namely:

Population – this is a typical scale variable. The intuition behind it is that size matters, and that’s why in most socio-economic research, when we really mean business in quantitative terms, we add such variables, pertinent to the size of the social entity studied. Urbanization occurring in a small country, like Belgium (with all my due respect for Belgians), is likely to occur differently from urbanization in India or in the U.S. In this specific case, I assume that a big population, like hundreds of millions of people, has to move more resources around to accommodate people in cities, as compared to a population counted in dozens of millions.

Urban population absolute – same tune, a scale variable, more specifically pertinent to the headcount of urban populations.

Gross Domestic Product (GDP, constant 2010 US$) – scale variable, once again, but this time it is about the real output of the economy. In my approach, the GDP is not exactly a measure of the wealth produced, but more of an appraisal of total productive activity in the humans living around. This is why I use constant prices. That shaves off the price-and-relative-wealth component, and leaves GDP as a metric pertinent to how much tradable surpluses do humans create in a given place and time.

Broad money (% of GDP) – this is essentially the opposite to the velocity of money, and it corresponds to another strand in my research. I discovered and I keep studying the fact that in the presence of quick technological change, human societies stuff themselves up with abnormally high amounts of cash (or cash equivalents, for that matter). It holds for entire countries as well as for individual businesses. You can find more on that in my article ‘Technological change as a monetary phenomenon’. I guess that when humans make more new social roles in cities, technologies change faster.

Energy use (kg of oil equivalent per capita) – this is one of the fundamental variables I frequently work with. I guess I included it in this particular piece of research just in case, in order to be able to connect with my research on the market of energy.

Agricultural land (km2) – the surface of agricultural land available is a logical correlate of urban population. A given number of people in cities need a given amount of food, which, in turn, can be provided by a given surface of agricultural land.

Cereal yield (kg per hectare) – logically complementary to the surface of agricultural land. Yield per hectare in France is different from what an average hectare can contribute in Nigeria, and that is likely to be correlated with urbanization.

You can get the raw data I used UNDER THIS LINK. It covers Australia, Brazil, Canada, China, Colombia, France, Gabon, Germany, Ghana, India, Malaysia, Mexico, Mozambique, Namibia, New Zealand, Nigeria, Norway, Poland, Russian Federation, United Kingdom, and the United States. All that lot observed over the window in time stretching from 1961 all the way to 2015.

I make that data into a neural network, which means that I make h(t_j) = x₁(t_j)*R* E[x_i(t_j-1)] + x₂(t_j)*R* E[x₂(t_j-1)] + … + x_n(t_j)*R* E[x_n(t_j-1)], as explained in my update titled ‘Representative for collective intelligence’, with x₁, x₂,…, x₇ input variables described above, grouped in 21 social entities (countries), and spread over 2015 – 1961= 54 years. After the curation of data for empty cells, I have m = 896 experimental rounds in the (alleged) collective intelligence, whose presence I guess behind the numbers. I made that lot learn how to squeeze the partly randomized input, controlled for internal coherence, into the mould of the desired output of the coefficient x_o = DU/DG. I ran the procedure of learning with 4 different methods of estimating the error of optimization. Firstly, I computed that error the way we do it in basic statistics, namely e₁ = x_o – h(t_j). The mixed-up input is simply subtracted from expected output. In the background, I assume that the locally output x_o is an expected value in statistical terms, i.e. it is the mean value of some hypothetical Gaussian distribution, local and specific to that concrete observation. With that approach to error, there is no neural activation as such. It is an autistic neural network, which does not discriminate input as for its strength. It just reacts.

As I want my collective intelligence to be smarter than your average leech, I make three more estimations of errors, with the input h(t_j) passing through a neural activation function. I start with the ReLU rectifier, AKA max[0, h(t_j)], and, correspondingly, with e₂ = x_o – ReLU[h(t_j)]. Then I warm up, and I use neural activation via hyperbolic tangent tanh[h] = (e^2h – 1) / (e^2h + 1), and I compute e₃ = x_o – tanh[h(t_j)]. The hyperbolic tangent is a transcendental number generated by periodical observation of a hyperbola, and that means that hyperbolic tangent has no functional correlation to its input. Neural activation with hyperbolic tangent creates a projection of input into a separate, non-correlated space of states, like cultural transformation of cognitive input into symbols, ideologies and whatnot. Fourthly and finally, I use the sigmoid function (AKA logistic function) sig(h) = 1 / (1 + e^-h) which can be read as smoothed likelihood that something happens, i.e. that input h(t_j) has full power. The corresponding error is e₄ = x_o – sig[h(t_j)].

From there, I go my normal way. I create 4 artificial realities out of my source dataset. Each of these realities assumes that humans strive to nail down the right social difference between cities and the countryside, as measured with the DU/DG coefficient. Each of these realities is generated with a different way of appraising how far we are from the desired DU/DG, this with four different ways of computing the error: e₁, e₂, e₃, and e₄. The expected states of both the source empirical dataset, and sets representative for those 4 alternative realities, are given by their respective vectors of mean values, i.e. mean DU/DG, mean population etc. Those vectors of means are provided in Table 1 below. The source dataset shows a mean DU/DG = 41,14, which means that cities in this dataset display, on average across countries, 41 times greater a density of population than the general density of population. Mean empirical population is 149,6 million people, with mean urban population being 67,34 million people. Yes, we have China and India in the lot, and they really pump those scale numbers up.

Table 1 – Vectors of mean values in the source empirical set and in the perceptrons simulating alternative realities, optimizing the coefficient DU/DG

		Perceptrons pegged on DU/DG
Variable	Source dataset	*error = xo – h*	*error = xo – ReLu(h)*	*error = xo – tanh(h)*	*error = xo – sigmoid(h)*
DU/DG	41,14	36,38	4,91	61,56	324,29
Population	149 625 587,07	125 596 355,00	(33 435 417,00)	252 800 741,00	1 580 356 431,00
GDP (constant 2010 US$)	1 320 025 624 972,08	1 025 700 000 000,00	(922 220 000 000,00)	2 583 780 000 000,00	18 844 500 000 000,00
Broad money (% of GDP)	57,50	54,13	31,80	71,99	258,38
Urban population absolute	67 349 480,42	54 311 459,20	(31 977 590,00)	123 331 287,00	843 649 729,00
Energy use (kg of oil equivalent per capita)	2 918,69	2 769,76	1 784,11	3 558,16	11 786,15
Agricultural land km2	1 227 301,86	1 135 064,25	524 611,51	1 623 345,71	6 719 245,69
Cereal yield (kg per hectare)	3 153,31	3 010,54	2 065,68	3 766,31	11 653,77

One of the first things which jumps to the eye in Table 1 – at least to my eye – is that one of the alternative realities, namely that based on the ReLU activation function, is an impossible reality. There are negative populations in this one, and this is not a livable state of things. I don’t know about you, my readers, but I would feel horrible knowing that I am a minus. People can’t be negative by default. By the way, in this specific dataset, the ReLU looks like almost identical to the basic difference e₁ = x_o – h(t_j). Yet, whilst making an alternative reality with no neural transformation of quasi-randomized input, thus making it with e₁ = x_o – h(t_j), creates something pretty close to the original empirics.

Another alternative reality which looks sort of sketchy is the one based on neural activation via the sigmoid function. This one transforms the initial mean expected values into their several-times-multiples. Looks like the sigmoid is equivalent, in this case, to powering the collective intelligence of societies studied with substantial doses of interesting chemicals. That particular reality is sort of a wild dream, like what it would be like to produce almost 4 times more cereal yield per hectare, having more than 4 times more agricultural land, and over 10 times more people in cities. The surface of available land being finite as it is, 4 times more agricultural land and 10 times more people in cities would mean cities tiny in terms of land surface, probably all in height, both under and above ground, with those cities being 324 times denser with humans than the general landscape. Sounds familiar, a bit like sci fi movies.

Four different ways of pitching input variables against the expected output of optimal DU/DG coefficient produce four very different alternative realities. Out of these four, one is impossible, one is hilarious, and we stay with two acceptable ones, namely that based on no proper neural activation at all, and the other one using the hyperbolic tangent for assessing the salience of things. Interestingly, errors estimated as e₁ = x_o – h(t_j) are essentially correlated with the input variables, whilst those assessed as e₃ = x_o – tanh[h(t_j)] are essentially uncorrelated. It means that in the former case one can more or less predict how satisfied the neural network will be with the local input, and that prediction can be reliably made a priori. In the latter case, with the hyperbolic tangent, there is no way to know in advance. In this case, neural activation is a true transformation of reality.

Table 2 below provides the formal calculation of standardized Euclidean distance between all the 4 alternative realities and the real world of tears we live in. By standardized Euclidean I mean: E = {[(mean_X – mean_S)²]^0,5} / mean_X. The ‘/ mean_X’ part means that divide the basic Euclidean distance by the mean value which serves me as benchmark, i.e. the empirical one. That facilitates subsequent averaging of those variable-specific Euclidean distances into one metric of mathematical similarity between entire vectors of values.

Table 2 – Vectors of standardized Euclidean distances between the source set X and the perceptrons simulating alternative realities, optimizing the coefficient DU/DG

	error = xo – h	error = xo – ReLu(h)	error = xo – tanh(h)	error = xo – sigmoid(h)
DU/DG]	0,115597874	0,88065496	0,496346621	6,882843342
Population	0,160595741	1,223460557	0,68955555	9,562073386
GDP (constant 2010 US$)	0,22296963	1,698637953	0,957371093	13,27585923
Broad money (% of GDP)	0,058672324	0,446981172	0,251923403	3,493424228
Urban population absolute	0,193587555	1,474800842	0,831213637	11,52644748
Energy use (kg of oil equivalent per capita)	0,051026181	0,388730845	0,219092892	3,038163202
Agricultural land km2	0,075154787	0,572548914	0,322694736	4,474810971
Cereal yield (kg per hectare)	0,045275	0,344916834	0,194398841	2,695730596
Average	*0,115359886*	*0,87884151*	*0,495324596*	*6,868669054*

Interestingly, whilst alternative reality based on neural activation through the ReLU function creates impossibly negative populations, its overall Euclidean similarity to the source dataset is not as big as it could seem. The impossible alternative is specific just to some variables.

Now, what does it all have to do with anything? How is that estimation of error representative for collective intelligence in human societies? Good question. I am doing my best to give some kind of answer to it. Quantitative socio-economic variables represent valuable collective outcomes, and thus are informative about alternative orientations in collective action. The process of learning how to nail those valuable outcomes down consumes said orientation in action. Assuming that figuring out the right proportion of demographic anomaly in cities, as measured with DU/DG, is a valuable collective outcome, four collective orientations thereon have been simulated. One goes a bit haywire (negative populations), and yet it shows a possible state of society which attempts to sort of smooth out the social difference between cities and the countryside, with DU/DG being ten times lower than reality. Another one goes fantasque, with huge numbers and a slightly sci-fi-ish shade. The remaining two look like realistic alternatives, one essentially predictable with e₁ = x_o – h(t_j), and another one essentially unpredictable, with e₃ = x_o – tanh[h(t_j)].

I want my method to serve as a predictive tool for sketching the possible scenarios of technological change, in particular as regards the emergence and absorption of radically new technologies. On the other hand, I want my method to be of help when it comes to identifying the possible Black Swans, i.e. the rather unlikely and yet profoundly disturbing states of nature. As I look at those 4 alternative realities my perceptron has just made up (it’s not me, its him! Well, it…), I can see two Black Swans. The one made with the sigmoid activation function shows a possible direction which, for example, African countries could follow, should they experience rapid demographic growth. This particular Black Swan is a hypothetical situation, when population grows like hell. This automatically puts enormous pressure on agriculture. More people need more food. More agriculture requires more space and there is fewer left for cities. Still, more people around need more social roles, and we need to ramp up the production thereof in very densely packed urban populations, where the sheer density of human interaction makes our social brains just race for novelty. This particular Black Swan could be actually a historical reconstruction. It could be representative for the type of social change which we know as civilisational revival: passage from the nomad life to the sedentary one, like a dozen of thousands of years ago, reconstruction of social tissue after the fall of the Western Roman Empire in Europe, that sort of stuff.

Another Black Swan is made with the ReLU activation function and simulates a society, where cities lose their function as factories of new social roles. It is the society in downsizing. It is actually a historical reconstruction, too. This is what must have happened when the Western Roman Empire was collapsing, and before the European civilization bounced back.

Well, well, well, that s**t makes sense… Amazing.

[1] World Bank 1: https://data.worldbank.org/indicator/SP.URB.TOTL.IN.ZS

[2] World Bank 4: https://data.worldbank.org/indicator/SP.POP.TOTL

[3] World Bank 2: https://data.worldbank.org/indicator/AG.LND.TOTL.UR.K2

[4] World Bank 5: https://data.worldbank.org/indicator/EN.POP.DNST

Representative for collective intelligence

I am generalizing from the article which I am currently revising, and I am taking a broader view on many specific strands of research I am running, mostly in order to move forward with my hypothesis of collective intelligence in human social structures. I want to recapitulate on my method – once more – in order to extract and understand its meaning.

I have recently realized a few things about my research. Firstly, I am using the logical structure of an artificial neural network as a simulator more than an optimizer, as digital imagination rather than functional, goal-oriented intelligence, and that seems to be the way of using AI which hardly anyone else in social sciences seems to be doing. The big question which I am (re)asking myself is to what extent are my simulations representative for the collective intelligence of human societies.

I start gently, with variables, hence with my phenomenology. I mostly use the commonly accessible and published variables, such as those published by the World Bank, the International Monetary Fund, STATISTA etc. Sometimes, I make my own coefficients out of those commonly accepted metrics, e.g. the coefficient of resident patent applications per 1 million people, the proportion between the density of population in cities and the general one, or the coefficient of fixed capital assets per 1 patent application.

My take on any variables in social sciences is very strongly phenomenological, or even hermeneutic. I follow the line of logic which you can find, for example, in “Phenomenology of Perception” by Maurice Merleau-Ponty (reprint, revised, Routledge, 2013, ISBN 1135718601, 9781135718602). I assume that any of the metrics we have in social sciences is an entanglement of our collective cognition with the actual s**t going on. As the actual s**t going on encompasses our way of forming our collective cognition, any variable used in social sciences is very much like a person’s attempt to look at themselves from a distance. Yes! This is what we use mirrors for! Variables used in social sciences are mirrors. Still, they are mirrors made largely by trial and error, with a little bit of a shaky hand, and each of them shows actual social reality in slightly disformed a manner.

Empirical research in social sciences consists, very largely, in a group of people trying to guess something about themselves on the basis of repeated looks into a set of imperfect mirrors. Those mirrors are imperfect, and yet they serve some purpose. I pass to my second big phenomenological take on social reality, namely that our entangled observations thereof are far from being haphazard. The furtive looks we catch of the phenomenal soup, out there, are purposeful. We pay attention to things which pay off. We define specific variables in social sciences because we know by experience that paying attention to those aspects of social reality brings concrete rewards, whilst not paying attention thereto can hurt, like bad.

Let’s take inflation. Way back in the day, like 300 years ago, no one really used the term of inflation because the monetary system consisted in a multitude of currencies, mixing private and public deeds of various kinds. Entire provinces in European countries could rely on bills of exchange issued by influential merchants and bankers, just to switch to other type of bills 5 years later. Fluctuations in the rates of exchange in those multiple currencies very largely cancelled each other. Each business of respectable size was like a local equivalent of the today’s Forex exchange. Inflation was a metric which did not even make sense at the time, as any professional of finance would intuitively ask back: ‘Inflation? Like… inflation in which exactly among those 27 currencies I use everyday?’.

Standardized monetary systems, which we call ‘FIAT money’ today, steadied themselves only in the 19^th century. Multiple currencies progressively fused into one, homogenized monetary mass, and mass conveys energy. Inflation is loss of monetary energy, like entropy of the monetary mass. People started paying attention to inflation when it started to matter.

We make our own social reality, which is fundamentally unobservable to us, and it makes sense because it is hard to have an objective, external look at a box when we are staying inside the box. Living in that box, we have learnt, over time, how to pay attention to the temporarily important properties of the box. We have learnt how to use maths for fine tuning in that selective perception of ours. We learnt, for example, to replace the basic distinction between people doing business and people not doing business at all with finer shades of how much business are people doing exactly in a unit of time-space.

Therefore, a set of empirical variables, e.g. from the World Bank, is a collection of imperfect observations, which represent valuable outcomes social outcomes. A set of N socio-economic variables represents N collectively valuable social outcomes, which, in turn, correspond to N collective pursuits – it is a set of collective orientations. Now, my readers have the full right to protest: ‘Man, just chill. You are getting carried away by your own ideas. Quantitative variables about society and economy are numbers, right? They are the metrics of something. Measurement is objective and dispassionate. How can you say that objectively gauged metrics are collective orientations?’. Yes, these are all valid objections, and I made up that little imaginary voice of my readers on the basis of reviews that I had for some of my papers.

Once again, then. We measure the things we care about, and we go to great lengths in creating accurate scales and methods of measurement for the things we very much care about. Collective coordination is costly and hard to achieve. If we devote decades of collective work to nail down the right way of measuring, e.g. the professional activity of people, it probably matters. If it matters, we are collectively after optimizing it. A set of quantitative, socio-economic variables represents a set of collectively pursued orientations.

In the branch of philosophy called ethics, there is a stream of thought labelled ‘contextual ethics’, whose proponents claim that whatever normatively defined values we say we stick to, the real values we stick to are to be deconstructed from our behaviour. Things we are recurrently and systematically after are our contextual ethical values. Yes, the socio-economic variables we can get from your average statistical office are informative about the contextual values of our society.

When I deal with a variable like the % of electricity in the total consumption of energy, I deal with a superimposition of two cognitive perspectives. I observe something that happens in the social reality, and that phenomenon takes the form of a spatially differentiated, complex state of things, which changes over time, i.e. one complex state transitions into another complex state etc. On the other hand, I observe a collective pursuit to optimize that % of electricity in the total consumption of energy.

The process of optimizing a socio-economic metric makes me think once again about the measurement of social phenomena. We observe and measure things which are important to us because they give us some sort of payoff. We can have collective payoffs in three basic ways. We can max out, for one. Case: Gross Domestic Product, access to sanitation. We can keep something as low as possible, for two. Case: murder, tuberculosis. Finally, we can maintain some kind of healthy dynamic balance. Case: inflation, use of smartphones. Now, let’s notice that we don’t really do fine calculations about murder or tuberculosis. Someone is healthy or sick, still alive or already murdered. Transitional states are not really of much of a collective interest. As it comes to outcomes which pay off by the absence of something, we tend to count them digitally, like ‘is there or isn’t there’. On the other hand, those other outcomes, which we max out on or keep in equilibrium, well, that’s another story. We invent and perfect subtle scales of measurement for those phenomena. That makes me think about a seminal paper titled ‘Selection by consequences’, by the founding father of behaviourism, Burrhus Frederic Skinner. Skinner introduced the distinction between positive and negative reinforcements. He claimed that negative reinforcements are generally stronger in shaping human behaviour, whilst being clumsier as well. We just run away from a tiger, we don’t really try to calibrate the right distance and the right speed of evasion. On the other hand, we tend to calibrate quite finely our reactions to positive reinforcements. We dose our food, we measure exactly the buildings we make, we learn by small successes etc.

If a set of quantitative socio-economic variables is informative about a set of collective orientations (collectively pursued outcomes), one of the ways we can study that set consists in establishing the hierarchy of orientations. Are some of those collective values more important than others? What does it even mean ‘more important’ in this context, and how can it be assessed? We can imagine that each among the many collective orientations is an individual pursuing their idiosyncratic path of payoffs from interactions with the external world. By the way, this metaphor is closer to reality than it could appear at the first sight. Each human is, in fact, a distinct orientation. Each of us is action. This perspective has been very sharply articulated by Martin Heidegger, in his “Being and Time”.

Hence, each collective orientation can be equated to an individual force, pulling the society in a specific direction. In the presence of many socio-economic variables, I assume the actual social reality is a superimposition of those forces. They can diverge or concur, as they please, I do not make any assumptions about that. Which of those forces pulls the most powerfully?

Here comes my mathematical method, in the form of an artificial neural network. I proceed step by step. What does it mean that we collectively optimize a metric? Mostly by making it coherent with our other orientations. Human social structures are based on coordination, and coordination happens both between social entities (individuals, cities, states, political parties etc.), and between different collective pursuits. Optimizing a metric representative for a collectively valuable outcome means coordinating with other collectively valuable outcomes. In that perspective, a phenomenon represented (imperfectly) with a socio-economic metric is optimized when it remains in some kind of correlation with other phenomena, represented with other metrics. The way I define correlation in that statement is a broad one: correlation is any concurrence of events displaying a repetitive, functional pattern.

Thus, when I study the force of a given variable as a collective orientation in a society, I take this variable as the hypothetical output in the process (of collective orientation, and I simulate that process as the output variable sort of dragging the remaining variables behind it, by the force of functional coherence. With a given set of empirical variables, I make as many mutations thereof as I have variables. Each mutated set represents a process, where one variable as output, and the remaining ones as input. The process consists of as many experiments as there are observational rows in my database. Most socio-economic variables come in rows of the type “country A in year X”.

Here, I do a little bit of mathematical cavalry with two different models of swarm intelligence: particle swarm and ants’ colony (see: Gupta & Srivastava 2020[1]). The model of particle swarm comes from the observation of birds, which keeps me in a state of awe about human symbolic creativity, and it models the way that flocks of birds stay collectively coherent when they fly around in the search of food. Each socio-economic variable is a collective orientation, and in practical terms it corresponds to a form of social behaviour. Each such form of social behaviour is a bird, which observes and controls its distance from other birds, i.e. from other forms of social behaviour. Societies experiment with different ways of maintaining internal coherence between different orientations. Each distinct collective orientation observes and controls its distance from other collective orientations. From the perspective of an ants’ colony, each form of social behaviour is a pheromonal trace which other forms of social behaviour can follow and reinforce, or not give a s**t about it, to their pleasure and leisure. Societies experiment with different strengths attributed to particular forms of social behaviour, which mimics an ants’ colony experimenting with different pheromonal intensities attached to different paths toward food.

Please, notice that both models – particle swarm and ants’ colony – mention food. Food is the outcome to achieve. Output variables in mutated datasets – which I create out of the empirical one – are the food to acquire. Input variables are the moves and strategies which birds (particles) or ants can perform in order to get food. Experimentation the ants’ way involves weighing each local input (i.e. the input of each variable in each experimental round) with a random weight R, 0 < R < 1. When experimenting the birds’ way, I drop into my model the average Euclidean distance E from the local input to all the other local inputs.

I want to present it all rolled nicely into an equation, and, as noblesse oblige, I introduce symbols. The local input of an input variable x_i in experimental round t_j is represented with x_i(t_j), whilst the local value of the output variable x_o is written as x_o(t_j). The compound experimental input which the society makes, both the ants’ way and the birds’ way, is written as h(t_j), and it spells h(t_j) = x₁(t_j)*R* E[x_i(t_j-1)] + x₂(t_j)*R* E[x₂(t_j-1)] + … + x_n(t_j)*R* E[x_n(t_j-1)].

Up to that point, this is not really a neural network. It mixes things up, but it does not really adapt. I mean… maybe there is a little intelligence? After all, when my variables act like a flock of birds, they observe each other’s position in the previous experimental round, through the E[x_i(t_j-1)] Euclidean thing. However, I still have no connection, at this point, between the compound experimental input h(t_j) and the pursued output x_o(t_j). I need a connection which would work like an observer, something like a cognitive meta-structure.

Here comes the very basic science of artificial neural networks. There is a function called hyperbolic tangent, which spells tanh = (e^2x – 1)/(e^2x + 1) where x can be whatever you want. This function happens to be one of those used in artificial neural networks, as neural activation, i.e. as a way to mediate between a compound input and an expected output. When I have that compound experimental input h(t_j) = x₁(t_j)*R* E[x_i(t_j-1)] + x₂(t_j)*R* E[x₂(t_j-1)] + … + x_n(t_j)*R* E[x_n(t_j-1)], I can put it in the place of x in the hyperbolic tangent, and I bet tanh = (e^2h – 1)/(e^2h + 1). In a neural network, error in optimization can be calculated, generally, as e = x_o(t_j) – tanh[h(t_j)]. That error can be fed forward into the next experimental round, and then we are talking, ‘cause the compound experimental input morphs into:

>> input h(t_j) = x₁(t_j)*R* E[x_i(t_j-1)]*e(t_j-1) + x₂(t_j)*R* E[x₂(t_j-1)] *e(t_j-1) + … + x_n(t_j)*R* E[x_n(t_j-1)] *e(t_j-1)

… and that means that each compound experimental input takes into account both the coherence of the input in question (E), and the results of previous attempts to optimize.

Here, I am a bit stuck. I need to explain, how exactly the fact of computing the error of optimization e = x_o(t_j) – tanh[h(t_j)] is representative for collective intelligence.

[1] Gupta, A., & Srivastava, S. (2020). Comparative analysis of ant colony and particle swarm optimization algorithms for distance optimization. Procedia Computer Science, 173, 245-253. https://doi.org/10.1016/j.procs.2020.06.029

Cœur de réflexion

Je me concentre sur un aspect particulier de la révision finale de mon article pour « International Journal of Energy Sector Management » – sous le titre « Climbing the right hill – an evolutionary approach to the European market of electricity » – notamment sur le rapport entre ma méthodologie et celle de MuSIASEM, soit « Multi-scale Integrated Analysis of Societal and Ecosystem Metabolism ».

Je me réfère plus particulièrement à trois articles que je juge représentatifs pour ce créneau de recherche :

>> Al-Tamimi and Al-Ghamdi (2020), ‘Multiscale integrated analysis of societal and ecosystem metabolism of Qatar’ Energy Reports, 6, 521-527, https://doi.org/10.1016/j.egyr.2019.09.019

>> Andreoni, V. (2020). The energy metabolism of countries: Energy efficiency and use in the period that followed the global financial crisis. Energy Policy, 139, 111304. https://doi.org/10.1016/j.enpol.2020.111304

>> Velasco-Fernández, R., Pérez-Sánchez, L., Chen, L., & Giampietro, M. (2020), A becoming China and the assisted maturity of the EU: Assessing the factors determining their energy metabolic patterns. Energy Strategy Reviews, 32, 100562. https://doi.org/10.1016/j.esr.2020.100562

De parmi ces trois, je choisis subjectivement le travail de prof. Andreoni (2020[1]) comme le plus solide en termes de théorie. L’idée de base de MuSIASEM est d’étudier l’efficience énergétique des sociétés humaines comme un métabolisme, donc comme un système complexe qui se soutient et se développe à travers la transformation d’énergie et de ressources matérielles.

J’essaie de comprendre et présenter la logique de base de MuSIASEM en explorant les avantages que professeur Andreoni attribue à cette méthode. Je me permets de traduire fidèlement un passage de l’article (2020[2]) : « […] l’approche MuSIASEM présente des avantages par rapport aux autres méthodologies utilisées pour étudier le métabolisme des sociétés, telles que ‘emergy’, empreinte écologique et l’analyse entrée-sortie […]. En fournissant des descriptions intégrées à travers des niveaux d’analyse différents, l’approche MuSIASEM ne réduit pas l’information en un index quantitatif unique et analyse l’énergie utilisée par rapport aux structures socio-économiques concrètes. Qui plus est, l’inclusion de dimensions multiples (telles que le PIB, temps humain et consommation d’énergie) en combinaison avec des échelles différentes d’analyse (telles que le niveau sectoriel et le niveau national) rend possible de fournir l’information pertinente aux processus à l’intérieur du système ainsi que d’analyser la façon dont les variables externes (telles que la crise économique et la pénurie des ressources) peuvent affecter l’allocation et l’utilisation des ressources ».

Je me dis que si quelqu’un se vante d’avoir des avantages par rapport à quoi que ce soit d’autre, ces avantages reflètent les aspects les plus importants des phénomènes en question, selon le même quelqu’un. Ainsi donc, prof. Andreoni assume que MuSIASEM permet d’étudier quelque chose d’important – l’efficience énergétique des sociétés comme un métabolisme – toute en ayant l’avantage de déconstruction des variables agrégées en des variables composantes ainsi que celui de multi-dimensionnalité d’analyse.

Les variables étudiées semblent donc être la base de la méthode. Parlons donc des variables. Professeur Andreoni présente dans son article trois variables essentielles :

>> L’activité humaine totale, calculée comme le produit de : [la population] x [24 heures] x [365 jours]

>> Transformation totale d’énergie, calculée comme la somme de : [consommation finale d’énergie] + [Consommation interne d’énergie dans le secteur d’énergie] + [Pertes d’énergie dans sa transformation]

>> Produit Intérieur Brut

Ces trois variables fondamentales sont étudiées à trois niveaux différents d’agrégation. Le niveau de base est celui d’économie(s) nationale(s), à partir d’où on décompose, tout d’abord, entre les secteurs macroéconomiques de : ménages par opposition à celui d’activité payée (entreprises plus secteur public). Ensuite, ces secteurs macroéconomiques sont tous les deux désagrégés en l’agriculture, l’industrie et les services.

A chaque niveau d’agrégation, les trois variables fondamentales sont mises en relation entre elles pour calculer deux coefficients : intensité énergétique et métabolisme d’énergie. Celui d’intensité énergétique est calculé comme quantité d’énergie utilisée pour produire un euro de Produit Intérieur Brut et c’est donc l’inverse de l’efficience énergétique (cette dernière est calculée comme quantité de PIB produite à partir d’une unité d’énergie). Le coefficient métabolique, en revanche, est calculé comme la quantité d’énergie par heure d’activité humaine.

J’ai quelques remarques critiques par rapport à ces variables, mais avant de développer là-dessus je contraste rapidement avec ma méthode. Les variables de professeur Andreoni sont des transformations des variables utilisées dans des bases de données publiquement accessibles. Professeur Andreoni prend donc une méthode générale d’observation empirique – donc par exemple la méthode de calculer la consommation finale d’énergie – et transforme cette méthode générale de façon à obtenir une vue différente de la même réalité empirique. Cette transformation tend à agréger des variables « communes ». Moi, de mon côté, j’utilise un éventail large des variables communément formalisées et présentées dans des bases de données publiquement accessibles plus un petit zest des coefficients que je calcule moi-même. En fait, dans la recherche sur l’énergie, j’utilise juste deux coefficients originaux, soit le nombre moyen de demandes de brevet nationales par 1 million d’habitants, d’une part, et la quantité moyenne de capital fixe d’entreprise par une demande nationale de brevet. Quant au reste, j’utilise des variables communes. Dans cet article que je suis en train de finir pour « International Journal of Energy Sector Management » j’utilise les quarante et quelques variables de Penn Tables 9.1. (Feenstra et al. 2015[3]) plus des variables de la Banque Mondiale au sujet d’énergie (consommation finale, participation des sources renouvelables, participation d’électricité) plus des données Eurostat sur les prix d’électricité, plus ces deux coefficients relatifs aux demandes nationales de brevets.

La différence entre ma méthode et celle de MuSIASEM est donc visible déjà au niveau phénoménologique. Moi, je prends la phénoménologie généralement acceptée – donc par exemple la phénoménologie de consommation d’énergie ou celle d’activité économique – et ensuite j’étudie le rapport entre les variables correspondantes pour en extraire un tableau plus complexe. Je sais déjà que dans ma méthode, la quantité et la diversité des variables est un facteur clé. Mes résultats deviennent vraiment robustes – donc cohérents à travers des échantillons empiriques différents – lorsque j’utilise une panoplie riche de variables. Chez MuSIASEM, en revanche, ils commencent par construire leur propre phénoménologie au tout début en ensuite ils raisonnent avec.

Il semble y avoir un terrain commun entre ma méthode et celle de MuSIASEM : on semble être d’accord que les variables macroéconomiques telles qu’elles sont accessibles publiquement donnent l’image imparfaite d’une réalité autrement plus complexe. A partir de là, toutefois, il y différence. Moi, j’assume que si je prends beaucoup d’observations imparfaites distinctes – donc beaucoup de variables différentes, chacune un peu à côté de la réalité – je peux reconstruire quelque chose à propos de ladite réalité en transformant ces observations imparfaites avec un réseau neuronal. J’assume donc que je ne sais pas d’avance de quelle manière exacte ces variables sont imparfaites et je m’en fiche par ailleurs. C’est comme si reconstruisais un crime (j’adore les romans policiers) à partir d’un grand nombre des dépositions faites par des témoins qui, au moment et en présence du crime en question étaient soit ivres, soit drogués soit ils regardaient un match de foot sur leur portable. J’assume qu’aussi peu fiables soient tous ces témoins, je peux interposer et recombiner leurs dépositions de façon à cerner le mécréant qui a tué la vieille dame. J’expérimente avec des combinaisons différentes et j’essaie de voir laquelle est la plus cohérente. Chez MuSIASEM, en revanche, ils établissent d’avance une méthode de mettre en concours des dépositions imparfaites des témoins en état d’ébriété et ensuite ils l’appliquent de façon cohérente à travers tous les cas de tels témoignages.

Jusqu’à ce point-là, ma méthode est garnie d’assomptions moins fortes que celle de MuSIASEM. De manière générale je préfère des méthodes avec des assomptions faibles. Lorsque je mets en question des idées reçues, tout simplement en les suspendant et en vérifiant si elles tiennent le coup (de suspension), j’ai la chance de trouver plus de trucs nouveaux et intéressants. Maintenant, je m’offre le plaisir pervers de passer au peigne fin les assomptions fortes de MuSIASEM, juste pour voir où bien puis-je leur enfoncer une épingle. Je commence par l’activité humaine totale, calculée comme le produit de : [la population] x [24 heures] x [365 jours]. Première remarque : le produit 24 heures fois 365 jours = 8760 heures est une constante. Si je compare deux pays aux populations différentes, leur activités humaines totales respectives seront différentes uniquement à travers leurs démographies différentes. Le produit [24 heures] x [365 jours] est donc une décoration redondante du point de vue mathématique. Toutefois, c’est une redondance astucieuse. Le produit 24 heures fois 365 jours = 8760 c’est le facteur de multiplication communément utilisé pour transformer la capacité énergétique en énergie effectivement accessible. On prend la puissance d’une bombe atomique, en joules, on la recalcule en kilowatts, on la multiplie par 24 heures fois 365 jours et boum : on obtient la quantité d’énergie accessible à la population générale si cette bombe explosait continuellement tout le long de l’année. On ajoute toutefois 24 heures supplémentaires d’explosion pour les années bissextiles.

Bombe atomique ou pas, le produit 24 heures fois 365 jours = 8760 est donc utile lorsqu’on veut faire une connexion élégante entre la démographie et la transformation d’énergie, ce qui semble judicieux dans une méthode de recherche qui se concentre précisément sur l’énergie. La multiplication « population x 8760 heures dans l’année » est-elle donc pertinente comme mesure d’activité humaine ? Hmmouiais… peut-être, à la rigueur… Je veux dire, si nous avons des populations très similaires en termes de style de vie et de technologie, elles peuvent démontrer des niveaux d’activité similaires par heure et donc des niveaux d’activité humaine totales distincts uniquement sur la base de leurs démographies différentes. Néanmoins, il nous faut des populations vraiment très similaires. Si nous prenons une portion essentielle de l’activité humaine – la production agricole par tête d’habitant – et nous la comparons entre la Belgique, l’Argentine et Botswana, nous obtenons des coefficients d’activité tout à fait différents.

Je pense donc que les assomptions qui maintiennent l’identité phénoménologique l’activité humaine totale = [la population] x [24 heures] x [365 jours] sont des assomptions tellement fortes qu’elles en deviennent dysfonctionnelles. J’assume donc que la méthode MuSIASEM utilise en fait la taille de la population comme une variable fondamentale, point à la ligne. Moi je fais de même, par ailleurs. Je trouve la démographie jouer un rôle injustement secondaire dans la recherche économique. Je vois que beaucoup de chercheurs utilisent des variables démographiques comme « calibrage » ou « facteurs d’ajustement ». Tout ce que je sais sur la théorie générale des systèmes complexes, par exemple le créneau de recherche sur la théorie d’automates cellulaires (Bandini, Mauri & Serra 2001[4] ; Yu et al. 2021[5]) ou bien la théorie d’essaims (Gupta & Srivastava (2020[6]), suggère que la taille des populations ainsi que leur intensité d’interactions sociales sont des attributs fondamentaux de chaque civilisation.

Je trouve donc que l’identité phénoménologique l’activité humaine totale = [la population] x [24 heures] x [365 jours] dans la méthode MuSIASEM est donc une sorte de ruse, un peu superflue, pour introduire la démographie au cœur de la réflexion sur l’efficience énergétique. Par conséquent, le coefficient métabolique de MuSIASEM, calculé comme la quantité d’énergie par heure d’activité humaine, est équivalent à la consommation d’énergie par tête d’habitant. Le métabolisme énergétique d’une société humaine est donc défini par la consommation d’énergie par tête d’habitant (https://data.worldbank.org/indicator/EG.USE.PCAP.KG.OE ) ainsi que le coût énergétique de PIB (https://data.worldbank.org/indicator/EG.USE.COMM.GD.PP.KD ). Les liens hypertexte entre parenthèses renvoient à des bases de données correspondantes de la Banque Mondiale. Lorsque je regarde ces deux coefficients à travers le monde et je fais un truc absolument simpliste – je discrimine les pays et les régions en une liste hiérarchique – deux histoires différentes émergent. Le coefficient de consommation d’énergie par tête d’habitant raconte une histoire de hiérarchie pure et simple de bien-être économique et social. Plus ce coefficient est élevé, plus le pays donné est développé en termes non seulement de revenu par tête d’habitant mais aussi en termes de complexité institutionnelle, droits de l’homme, complexité technologique etc.

Lorsque j’écoute l’histoire dite par le coût énergétique de PIB (https://data.worldbank.org/indicator/EG.USE.COMM.GD.PP.KD ), c’est compliqué comme une enquête policière. Devinez donc les points communs entre Panama, Sri Lanka, la Suisse, l’Irlande, Malte et la République Dominicaine. Fascinant, non ? Eh bien, ces 6 pays sont en tête de la course planétaire à l’efficience énergétique, puisqu’ils sont tous les six capables de produire 1000 dollars de PIB avec moins de 50 kilogrammes d’équivalent pétrole en énergie consommée. Pour placer leur exploit dans un contexte géographique plus large, les États-Unis et la Serbie sont plus de deux fois plus bas dans cette hiérarchie, tout près l’un de l’autre, à 122 kilogrammes d’équivalent pétrole par 1000 dollars de PIB. Par ailleurs, ça les place tous les deux près de la moyenne planétaire ainsi que celle des pays dans la catégorie « revenu moyen inférieur ».

Si je récapitule mes observations sur la géographie de ces deux coefficients, les sociétés humaines différentes semblent avoir une capacité très idiosyncratique d’optimiser le coût énergétique de PIB à des niveaux différents de la consommation d’énergie par tête d’habitant. C’est comme s’il y avait une façon différente d’optimiser l’efficience énergétique en étant pauvre, par rapport à celle d’optimiser la même efficience lorsqu’on est riche et développé.

Nous, les homo sapiens, on peut faire des trucs vraiment bêtes dans le quotidien mais dans le long terme nous sommes plutôt pratiques, ce qui pourrait notre capacité actuelle de transformer quelque 30% de l’énergie totale à la surface de la planète. Si hiérarchie il y a, cette hiérarchie a probablement un rôle à jouer. Difficile à dire quel rôle exactement mais ça semble important d’avoir cette structure hiérarchique d’efficience énergétique. C’est un autre point où je diverge de la méthode MuSIASEM. Les chercheurs actifs dans le créneau MuSIASEM assument que l’efficience énergétique maximale est un impératif évolutif de notre civilisation et que tous les pays devraient aspirer à l’optimiser. Hiérarchies d’efficiences énergétique sont donc perçues comme un accident historique dysfonctionnel, probablement effet d’oppression des pauvres par les riches. Bien sûr, on peut demander si les habitants de la République Dominicaine sont tellement plus riches que ceux des États-Unis, pour avoir une efficience énergétique presque trois fois supérieure.

[1] Andreoni, V. (2020). The energy metabolism of countries: Energy efficiency and use in the period that followed the global financial crisis. Energy Policy, 139, 111304. https://doi.org/10.1016/j.enpol.2020.111304

[2] Andreoni, V. (2020). The energy metabolism of countries: Energy efficiency and use in the period that followed the global financial crisis. Energy Policy, 139, 111304. https://doi.org/10.1016/j.enpol.2020.111304

[3] Feenstra, Robert C., Robert Inklaar and Marcel P. Timmer (2015), “The Next Generation of the Penn World Table” American Economic Review, 105(10), 3150-3182, available for download at http://www.ggdc.net/pwt 

[4] Bandini, S., Mauri, G., & Serra, R. (2001). Cellular automata: From a theoretical parallel computational model to its application to complex systems. Parallel Computing, 27(5), 539-553. https://doi.org/10.1016/S0167-8191(00)00076-4

[5] Yu, J., Hagen-Zanker, A., Santitissadeekorn, N., & Hughes, S. (2021). Calibration of cellular automata urban growth models from urban genesis onwards-a novel application of Markov chain Monte Carlo approximate Bayesian computation. Computers, environment and urban systems, 90, 101689. https://doi.org/10.1016/j.compenvurbsys.2021.101689

[6] Gupta, A., & Srivastava, S. (2020). Comparative analysis of ant colony and particle swarm optimization algorithms for distance optimization. Procedia Computer Science, 173, 245-253. https://doi.org/10.1016/j.procs.2020.06.029

Living next door to such small success

Just two updates ago, I was trying to combine my work on the technological concept which I labelled ‘Energy Ponds’ AKA ‘Project Aqueduct’, with more theoretical a strand of research on collective intelligence in human societies. A third component thread has come into the game, a bit as a surprise. The editor of ‘International Journal of Energy Sector Management’ has just asked me to give a final revision to the manuscript which I am about to publish with them, titled ‘Climbing the right hill – an evolutionary approach to the European market of electricity’. More specifically, the editor asks me to refine the style of the paper, so as to make it more accessible to non-initiated readers.

I think I am smart. Many people think they are. I know I tend to overestimate my work capacity, though. I need an intellectual synthesis for all the three things: ‘Energy Ponds’, research on collective intelligence, and the final revision of my article. I need some kind of common denominator over which I could put and denominate all that intellectual stuff. I focus on the phenomenon of technological change. My most fundamental intuition about technological change is that it happens as a by-product of us, humans, collectively pursuing some other outcomes. I perceive technology as an emergence (not to confound with emergency) which happens when human societies reach a given level of complexity. Technologies are complex ways human interaction with the broadly spoken natural environment, i.e. with both natural resources and natural constraints.

I am rummaging in my most personal cases of technological change, namely my idea of ‘Energy Ponds’, and my investment decisions in the stock market. Non-linearity of change keeps floating to the surface. When the most obvious path of development in a technology is tight optimization through a sequence of small incremental improvements in efficiency, that technology is close to maturity in its lifecycle, and is not much of a big deal anymore. The truly promising technologies, those able to wake up the neighbours, are those with yet unclear prospects for optimization, with different alternative paths of experimentation in view.

Deep technological change occurs as non-linear path of experimentation in collective human interaction with both natural resources and natural constraints. Non-linearity means uncertainty, and uncertainty implies alternative states of nature, spread over a broad spectrum of outcomes. Them Black Swans are just waiting around the street corner. Deep technological change can play out according to different scenarios. We tend to think about scenarios as sequences, only with technological change the sequence is highly speculative, and the more uncertain the further we go from the starting point. There is another way of defining a scenario, namely as an orientation, a social force which pushes in a specific direction.

I start connecting the dots. Deep, break-through technological change practically never happens as a clearly purposeful collective strategy. It is always a disruption, and it takes us by surprise. Technological change happens as a sudden shortcut to achieve whatever collective outcomes we are after. People who invented the wheel probably didn’t want to invent the wheel as such, they were after a better way of transportation by land. Internet was invented because scientists started to work in large, dispersed networks of labs and needed a fast communication system for a lot of content.

Thus, we are after something, and, accidentally, we invent something else, which makes ripples across the social structure. We use the transformational force conveyed in those ripples to keep pursuing the same collective outcomes. It is interesting to notice that a new technology is practically never superior per se to the incumbent solutions. Social improvement happens only when human societies wrap themselves around that novel stuff and learn how to use it. Let’s suppose that a benevolent and very advanced alien race hands out to us a technology to travel between parallel universes. Looks cool, at the first sight. When we think about it longer, though, questions arise. What are the practical benefits of travelling between parallel universes? It is only when we figure out those benefits that we start absorbing that otherwise revolutionary technology.

I double back a bit on my own words. Deep technological change is essentially disruptive and surprising, and yet there is more to technological change than just the deep and disruptive kind. Periods of grinding, progressive optimization come after and between deep technological ripples. Here, I ask: why? Why the hell having all that business of technological change? It is interesting to notice that rapid technological change makes rifts in space just as it does in time. There are places on this planet where humans have been living for quite a few millennia without inventing s**t. It is even more interesting to notice that some among those no-progress lands used to be quite the opposite in the past. Amazonian jungle is a good example. Pre-Colombian people (i.e. people who used to live there before they learnt they had just been discovered) had a thriving civilization, with a lot of innovations up their sleeve, such as altitude specific agriculture in terraced fields, or written communication using pieces of string. Afghanistan (hic!) is another example. Centuries before Pythagoras figured out his angles and them square roots from sums of square powers, the place which we call ‘Afghanistan’ today used to be a huge mining hub, providing tin to all of the Bronze Age civilization in the Mediterranean and the Levant.

My point is that we, humans, need a good kick where it really hurts, plus some favourable conditions to recover when it really hurts, and then we start inventing stuff. Still, many of us can pass entire epochs (literally epochs) without figuring out anything new. As I like surfing through literature as I write, a few quotes come to my mind, out of the books I am reading now. Out of ‘The Black Swan. The impact of the highly improbable’ by Nassim Nicolas Taleb , Penguin, 2010, I have that passage from page 114: “Consider the following: of all the colorful adventurers who have lived on our planet, many were occasionally crushed, and a few did bounce back repeatedly. It is those who survive who will tend to believe that they are indestructible; they will have a long and interesting enough experience to write books about it. Until, of course … Actually, adventurers who feel singled out by destiny abound, simply because there are plenty of adventurers, and we do not hear the stories of those down on their luck”. The point is that we mostly know about technological change we know, as it were. The folds of history, which we tend to smooth out ex post, cover thousands of episodes when inventions simply didn’t work. I wonder how many people got mauled to death before someone finally nailed down the right way to make big, strong oxen pull heavy carts on wheels.

The adventure of technological change plays out favourably just sometimes, and yet we keep trying. Here come two quotes from another book: ‘The Knowledge Illusion. Why we never think alone’ by Steven Sloman and Philip Fernbach, RIVERHEAD BOOKS (An imprint of Penguin Random House LLC, Ebook ISBN: 9780399184345, Kindle Edition). On page page 133 thereof a whole new chapter starts under the provocative title: Technology as an Extension of Thought. It goes: ‘The mastery of new technology has gone hand in hand with the evolution of our species. According to Ian Tattersall, curator emeritus with the American Museum of Natural History in New York, “cognitive capacity and technology reinforced each other” as civilization developed. Genetic evolution and technological change have run in tandem throughout our evolutionary history. As brains increased in size from one hominid species to its descendants, tools became more sophisticated and more common. Our predecessors started using rocks with sharp edges. Later generations discovered fire, stone axes, and knives, followed by harpoons and spears, then nets, hooks, traps, snares, and bows and arrows, and eventually farming. Each of these technological changes was accompanied by all the other changes that led to the modern human being: cultural, behavioral, and genetic changes’. A few pages further, p. 150, the same authors write about the modern technology of crowdsourcing: ‘The power of crowdsourcing and the promise of collaborative platforms suggest that the place to look for real superintelligence is not in a futuristic machine that can outsmart human beings. The superintelligence that is changing the world is in the community of knowledge’.

It seems that we, humans, invent new things just because we can, just because we are biologically wired for it. Still, that creative interaction with our environment is full of failures, which, from time to time, produce some timid successes. The local humans, living next door to such small success, have the drive and the capacity to put a big fire up, starting from such a small spark. Once it has worked, deep technological rift happens, which transforms civilizations.

As I return to the final revision of the manuscript which I am about to publish with them, titled ‘Climbing the right hill – an evolutionary approach to the European market of electricity’, for the ‘International Journal of Energy Sector Management’, I wonder how to describe the kind of technological change which I write about in that paper, namely the development of renewable energies and the transition to electricity from the straightforward use of fossil thermal energy, in the European market. What I see in the empirical data is a historically short window of progress which, whilst being a bit bumpy, generally follows an upward trend. As I look at all of my so-far research on collective intelligence, it is largely the same. I have been studying historically short windows of technological change which generally looks like progress with some minor accidents on the way. On the other hand, when I refer to my ‘Energy Ponds’ concept and to the feasibility studies I am running for it, it is the deep-ripple type. I propose to implement a complex solution whose outcomes will be more environmental (water management and landscape management) more than straightforwardly financial. Yes, the whole thing has a chance to earn a living by selling electricity from hydroelectric turbines, but this is like Nicola Tesla earning a living by repairing people’s house equipment.

Is there any theoretical way I can use my toolbox of collective intelligence – tested on incremental technological change – to represent the socio-economic absorption of ‘Energy Ponds’? Good question. It is about social structures reacting to something disturbing. The general intuition I have in that respect, and which I developed through simulations described in my draft paper: ‘Behavioral absorption of Black Swans: simulation with an artificial neural network’ is that social structures tend to smooth out disturbances, for one. New things enter the game easier and faster than old things get pushed out of it, for two. I think that both cases, namely technological change in the European market of electricity and the possible development of ‘Energy Ponds’ are the kind of story, when new technologies sort of pile up on the top of old ones. Increased complexity is created. Increasing complexity means the build-up of some kind of non-equilibrium, which either gest smoothed out, and the corresponding technological change is nicely absorbed, or it doesn’t, and we have the Schumpeterian creative destruction.

I pretty much know how social structures wrap themselves around new power installations. There is one Black Swan, though, swimming surreptitiously around: the nuclear. In Europe, we have a keen interest in passing from combustion engines to electric vehicles. Combustion engines run on gasoline or on diesel, which all boils down to oil, which we don’t have and need to import. Transportation based on electricity makes us much less dependent on imported fuels, and that means more strategic security. Still, I think we will need to come back to developing nuclear power plants if we want to have enough juice for all those batteries on wheels.

As regards ‘Energy Ponds’, the big question is how will urban and peri-urban structures get along with swamp-like reservoirs of water. That is really a deep question. For centuries, cities in Europe have been developing by drying out and draining down swamps. Swamps and buildings do not really like each other. Do we have the technologies to make their mutual neighbourhood liveable?

The red-neck-cellular automata

I continue revising my work on collective intelligence, and I am linking it to the theory of complex systems. I return to the excellent book ‘What Is a Complex System?’ by James Landyman and Karoline Wiesner (Yale University Press, 2020, ISBN 978-0-300-25110-4, Kindle Edition). I take and quote their summary list of characteristics that complex systems display, on pages 22 – 23: “ […] which features are necessary and sufficient for which kinds of complexity and complex system. The features are as follows:

1. Numerosity: complex systems involve many interactions among many components.

2. Disorder and diversity: the interactions in a complex system are not coordinated or controlled centrally, and the components may differ.

3. Feedback: the interactions in complex systems are iterated so that there is feedback from previous interactions on a time scale relevant to the system’s emergent dynamics.

4. Non-equilibrium: complex systems are open to the environment and are often driven by something external.

5. Spontaneous order and self-organisation: complex systems exhibit structure and order that arises out of the interactions among their parts.

6. Nonlinearity: complex systems exhibit nonlinear dependence on parameters or external drivers.

7. Robustness: the structure and function of complex systems is stable under relevant perturbations.

8. Nested structure and modularity: there may be multiple scales of structure, clustering and specialisation of function in complex systems.

9. History and memory: complex systems often require a very long history to exist and often store information about history.

10. Adaptive behaviour: complex systems are often able to modify their behaviour depending on the state of the environment and the predictions they make about it”.

As I look at the list, my method of simulating collective intelligence is coherent therewith. Still, there is one point which I think I need to dig a bit more into: that whole thing with simple entities inside the complex system. In most of my simulations, I work on interactions between cognitive categories, i.e. between quantitative variables. Interaction between real social entities is most frequently implied rather than empirically nailed down. Still, there is one piece of research which sticks out a bit in that respect, and which I did last year. It is devoted to cities and their role in the human civilisation. I wrote quite a few blog updates on the topic, and I have one unpublished paper written thereon, titled ‘The Puzzle of Urban Density And Energy Consumption’. In this case, I made simulations of collective intelligence with my method, thus I studied interactions between variables. Yet, in the phenomenological background of emerging complexity in variables, real people interact in cities: there are real social entities interacting in correlation with the connections between variables. I think the collective intelligence of cities the piece of research where I have the surest empirical footing, as compared to others.

There is another thing which I almost inevitably think about. Given the depth and breadth of the complexity theory, such as I start discovering it with and through that ‘What Is a Complex System?’ book, by James Landyman and Karoline Wiesner, I ask myself: what kind of bacon can I bring to that table? Why should anyone bother about my research? What theoretical value added can I supply? A good way of testing it is talking real problems. I have just signalled my research on cities. The most general hypothesis I am exploring is that cities are factories of new social roles in the same way that the countryside is a factory of food. In the presence of demographic growth, we need more food, and we need new social roles for new humans coming around. In the absence of such new social roles, those new humans feel alienated, they identify as revolutionaries fighting for the greater good, they identify the incumbent humans as oppressive patriarchy, and the next thing you know, there is systemic, centralized, government-backed terror. Pardon my French, this is a system of social justice. Did my bit of social justice, in the communist Poland.

Anyway, cities make new social roles by making humans interact much more abundantly than they usually do in a farm. More abundant an interaction means more data to process for each human brain, more s**t to figure out, and the next thing you know, you become a craftsman, a businessperson, an artist, or an assassin. Once again, being an assassin in the countryside would not make much sense. Jumping from one roof to another looks dashing only in an urban environment. Just try it on a farm.

Now, an intellectual challenge. How can humans, who essentially don’t know what to do collectively, can interact so as to create emergent complexity which, in hindsight, looks as if they had known what to do? An interesting approach, which hopefully allows using some kind of neural network, is the paradigm of the maze. Each individual human is so lost in social reality that the latter appears as a maze, which one ignores the layout of. Before I go further, one linguistic thing is to nail down. I feel stupid using impersonal forms such as ‘one’, or ‘an individual’. I like more concreteness. I am going to start with George the Hero. George the Hero lives in a maze, and I stress it: he lives there. Social reality is like a maze to George, and, logically, George does not even want to get out of that maze, ‘cause that would mean being lonely, with no one around to gauge George’s heroism. George the Hero needs to stay in the maze.

The first thing which George the Hero needs to figure out is the dimensionality of the maze. How many axes can George move along in that social complexity? Good question. George needs to experiment in order to discover that. He makes moves in different social directions. He looks around what different kinds of education he can possibly get. He assesses his occupational options, mostly jobs and business ventures. He asks himself how he can structure his relations with family and friends. Is being an asshole compatible with fulfilling emotional bonds with people around?

Wherever George the Hero currently is in the maze, there are n neighbouring and available cells around him. In each given place of the social maze, George the Hero has n possible ways to move further, into those n accessible cells in the immediate vicinity, and that is associated with k dimensions of movement. What is k, exactly? Here, I can refer to the theory of cellular automata, which attempts to simulate interactions between really simple, cell-like entities (Bandini, Mauri & Serra 2001[1]; Yu et al. 2021[2]). There is something called ‘von Neumann neighbourhood’. It corresponds to the assumption that if George the Hero has n neighbouring social cells which he move into, he can move like ‘left-right-forward-back’. That, in turn, spells k = n/2. If George can move into 4 neighbouring cells, he moves in a 2-dimensional space. Should he be able to move into 6 adjacent cells of the social maze, he has 3 dimensions to move along etc. Trouble starts when George sees an odd number of places to move to, like 5 or 7, on the account of these giving half-dimensions, like 5/2 = 2.5, 7/2 = 3.5 etc. Half a dimension means, in practical terms, that George the Hero faces social constraints. There might be cells around, mind you, which technically are there, but there are walls between George and them, and thus, for all practical purposes, the Hero can afford not to give a f**k.

George the Hero does not like to move back. Hardly anyone does. Thus, when George has successfully moved from cell A to cell B, he will probably not like going back to A, just in order to explore another cell adjacent thereto. People behave heuristically. People build up on their previous gains. Once George the Hero has moved from A to B, B becomes his A for the next move. He will choose one among the cells adjacent to B (now A), move there etc. George is a Hero, not a scientist, and therefore he carves a path through the social maze rather than discovers the maze as such. Each cell in the maze contains some rewards and some threats. George can get food and it means getting into a dangerously complex relation with that sabre-tooth tiger. George can earn money and it means giving up some of his personal freedom. George can bond with other people and find existential meaning and it means giving up even more of what he provisionally perceives as his personal freedom.

The social maze is truly a maze because there are many Georges around. Interestingly, many Georges in English give one Georges in French, and I feel this is the point where I should drop the metaphor of George the Hero. I need to get more precise, and thus I go to a formal concept in the theory of cellular automata, namely that of a d-dimensional cellular automaton, which can be mathematically expressed as A = (Z^d, S, N, Sⁿ⁺¹ -> S). In that automaton A, Z^d stands for the architecture of the maze, thus a lattice of d – tuples of integer numbers. In plain human, Z^d is given by the number of dimensions, possibly constrained, which a human can move along in the social space. Many people carve their paths across the social maze, no one likes going back, and thus the more people are around, and the better they can communicate their respective experiences, the more exhaustive knowledge we have of the surrounding Z^d.

There is a finite set S of states in that social space Z^d, and that finitude is connected to the formally defined neighbourhood of the automaton A, namely the N. Formally, N is a finite ordered subset of Z^d, and, besides the ‘left-right-forward-back’ neighbourhood of von Neumann, there is a more complex one, namely the Moore’s neighbourhood. In the latter, we can move diagonally between cells, like to the left and forward, to the right and forward etc. Keeping in mind that neighbourhood means, in practical terms, the number n of cells which we can move into from the social cell we are currently in, the cellular automaton can be rephrased as as A = (Z^d, S, n, Sⁿ⁺¹ -> S). The transition Sⁿ⁺¹ -> S, called the local rule of A, makes more sense now. With me being in a given cell of the social maze, and there being n available cells immediately adjacent to mine, that makes n +1 cells where I can possibly be in, and I can technically visit all those cells in a finite number of Sⁿ⁺¹ combinatorial paths. The transition Sⁿ⁺¹ -> S expresses the way which I carve my finite set S of states out of the generally available Sⁿ⁺¹.

If I assume that cities are factories of new social roles, the cellular automaton of an urban homo sapiens should be more complex than the red-neck-cellular automaton in a farm folk. It might mean greater an n, thus more cells available for moving from where I am now. It might also mean more efficient a Sⁿ⁺¹ -> S local rule, i.e. a better way to explore all the possible states I can achieve starting from where I am. There is a separate formal concept for that efficiency in the local rule, and it is called configuration of the cellular automaton AKA its instantaneous description AKA its global state, and it refers to the map Z^d -> S. Hence, the configuration of my cellular automaton is the way which the overall social space Z^d mapes into the set S of states actually available to me.

Right, if I have my cellular automaton with a configuration map Z^d -> S, it is sheer fairness that you have yours too, and your cousin Eleonore has another one for herself, as well. There are many of us in the social space Z^d. We are many x’s in the Z^d. Each x of us has their own configuration map Z^d -> S. If we want to get along with each other, our individual cellular automatons need to be mutually coherent enough to have a common, global function of cellular automata, and we know there is such a global function when we can collectively produce a sequence of configurations.

According to my own definition, a social structure is a collectively intelligent structure to the extent that it can experiment with many alternative versions of itself and select the fittest one, whilst staying structurally coherent. Structural coherence, in turn, is the capacity to relax and tighten, in a sequence, behavioural coupling inside the society, so as to allow the emergence and grounding of new behavioural patterns. The theory of cellular automata provides me some insights in that respect. Collective intelligence means the capacity to experiment with ourselves, right? That means experimenting with our global function Z^d -> S, i.e. with the capacity to translate the technically available social space Z^d into a catalogue S of possible states. If we take a random sample of individuals in a society, and study their cellular automatons A, they will display local rules Sⁿ⁺¹ -> S, and these can be expressed as coefficients (S / Sⁿ⁺¹), 0 ≤ (S / Sⁿ⁺¹) ≤ 1. The latter express the capacity of individual cellular automatons to generate actual states S of being out of the generally available menu of Sⁿ⁺¹.

In a large population, we can observe the statistical distribution of individual (S / Sⁿ⁺¹) coefficients of freedom in making one’s cellular state. The properties of that statistical distribution, e.g. the average (S / Sⁿ⁺¹) across the board, are informative about how intelligent collectively the given society is. The greater the average (S / Sⁿ⁺¹), the more possible states can the given society generate in the incumbent social structure, and the more it can know about the fittest state possible. That looks like a cellular definition of functional freedom.

[1] Bandini, S., Mauri, G., & Serra, R. (2001). Cellular automata: From a theoretical parallel computational model to its application to complex systems. Parallel Computing, 27(5), 539-553. https://doi.org/10.1016/S0167-8191(00)00076-4

[2] Yu, J., Hagen-Zanker, A., Santitissadeekorn, N., & Hughes, S. (2021). Calibration of cellular automata urban growth models from urban genesis onwards-a novel application of Markov chain Monte Carlo approximate Bayesian computation. Computers, environment and urban systems, 90, 101689. https://doi.org/10.1016/j.compenvurbsys.2021.101689

The collective of individual humans being any good at being smart

I am working on two topics in parallel, which is sort of normal in my case. As I know myself, instead of asking “Isn’t two too much?”, I should rather say “Just two? Run out of ideas, obviously”. I keep working on a proof-of-concept article for the idea which I provisionally labelled “Energy Ponds” AKA “Project Aqueduct”, on the one hand. See my two latest updates, namely ‘I have proven myself wrong’ and ‘Plusieurs bouquins à la fois, comme d’habitude’, as regards the summary of what I have found out and written down so far. As in most research which I do, I have come to the conclusion that however wonderful the concept appears, the most important thing in my work is the method of checking the feasibility of that concept. I guess I should develop on the method more specifically.

On the other hand, I am returning to my research on collective intelligence. I have just been approached by a publisher, with a kind invitation to submit the proposal for a book on that topic. I am passing in review my research, and the available literature. I am wondering what kind of central thread I should structure the entire book around. Two threads turn up in my mind, as a matter of fact. The first one is the assumption that whatever kind of story I am telling, I am actually telling the story of my own existence. I feel I need to go back to the roots of my interest in the phenomenon of collective intelligence, and those roots are in my meddling with artificial neural networks. At some point, I came to the conclusion that artificial neural networks can be good simulators of the way that human societies figure s**t out. I need to dig again into that idea.

My second thread is the theory of complex systems AKA the theory of complexity. The thing seems to be macheting its way through the jungle of social sciences, those last years, and it looks interestingly similar to what I labelled as collective intelligence. I came by the theory of complexity in three books which I am reading now (just three?). The first one is a history book: ‘1177 B.C. The Year Civilisation Collapsed. Revised and Updated’, published by Eric H. Cline with Princeton University Press in 2021[1]. The second book is just a few light years away from the first one. It regards mindfulness. It is ‘Aware. The Science and Practice of Presence. The Groundbreaking Meditation Practice’, published by Daniel J. Siegel with TarcherPerigee in 2018[2]. The third book is already some sort of a classic; it is ‘The Black Swan. The impact of the highly improbable’ by Nassim Nicolas Taleb with Penguin, in 2010.

I think it is Daniel J. Siegel who gives the best general take on the theory of complexity, and I allow myself to quote: ‘One of the fundamental emergent properties of complex systems in this reality of ours is called self-organization. That’s a term you might think someone in psychology or even business might have created—but it is a mathematical term. The form or shape of the unfolding of a complex system is determined by this emergent property of self-organization. This unfolding can be optimized, or it can be constrained. When it’s not optimizing, it moves toward chaos or toward rigidity. When it is optimizing, it moves toward harmony and is flexible, adaptive, coherent, energized, and stable’. (Siegel, Daniel J.. Aware (p. 9). Penguin Publishing Group. Kindle Edition).

I am combining my scientific experience with using AI as social simulator with the theory of complex systems. I means I need to UNDERSTAND, like really. I need to understand my own thinking, in the first place, and then I need to combine it with whatever I can understand from other people’s thinking. It started with a simple artificial neural network, which I used to write my article ‘Energy efficiency as manifestation of collective intelligence in human societies’ (Energy, 191, 116500, https://doi.org/10.1016/j.energy.2019.116500 ). I had a collection of quantitative variables, which I had previously meddled with using classical regression. As regression did not really bring much conclusive results, I had the idea of using an artificial neural network. Of course, today, neural networks are a whole technology and science. The one I used is the equivalent of a spear with a stone tip as compared to a battle drone. Therefore, the really important thing is the fundamental logic of neural networking as compared to regression, in analyzing quantitative data.

When I do regression, I come up with a function, like y = a₁*x₁ + a₂*x₂ + …+ b, I trace that function across the cloud of empirical data points I am working with, and I measure the average distance from those points to the line of my function. That average distance is the average (standard) error of estimation with that given function. I repeat the process as many times as necessary to find a function which both makes sense logically and yields the lowest standard error of estimation. The central thing is that I observe all my data at once, as if it was all happening at the same time and as if I was observing it from outside. Here is the thing: I observe it from outside, but when that empirical data was happening, i.e. when the social phenomena expressed in my quantitative variables were taking place, everybody (me included) was inside, not outside.

How to express mathematically the fact of being inside the facts measured? One way is to take those empirical occurrences one by one, sort of Denmark in 2005, and then Denmark in 2006, and then Germany in 2005 etc. Being inside the events changes my perspective on what is the error of estimation, as compared to being outside. When I am outside, error means departure from the divine plan, i.e. from the regression function. When I am inside things that are happening, error happens as discrepancy between what I want and expect, on the one hand, and what I actually get, on the other hand. These are two different errors of estimation, measured as departures from two different functions. The regression function is the most accurate (or as accurate as you can get) mathematical explanation of the empirical data points. The function which we use when simulating the state of being inside the events is different: it is a function of adaptation.

Intelligent adaptation means that we are after something: food, sex, power, a new Ferrari, social justice, 1000 000 followers on Instagram…whatever. There is something we are after, some kind of outcome we try to optimize. When I have a collection of quantitative variables which describe a society, such as energy efficiency, headcount of population, inflation rates, incidence of Ferraris per 1 million people etc., I can make a weak assumption that any of these can express a desired outcome. Here, a digression is due. In science and philosophy, weak assumptions are assumptions which assume very little, and therefore they are bloody hard to discard. On the other hand, strong assumptions assume a lot, and that makes them pretty good targets for discarding criticism. In other words, in science and philosophy, weak assumptions are strong and strong assumptions are weak. Obvious, isn’t it? Anyway, I make that weak assumption that any phenomenon we observe and measure with a numerical scale can be a collectively desired outcome we pursue.

Another assumption I make, a weak one as well, is sort of hidden in the word ‘expresses’. Here, I relate to a whole line of philosophical and scientific heritage, going back to people like Plato, Kant, William James, Maurice Merleau-Ponty, or, quite recently, Michael Keane (1972[3]), as well as Berghout & Verbitskiy (2021[4]). Very nearly everyone who seriously thought (or keeps thinking, on the account of being still alive) about human cognition of reality agrees that we essentially don’t know s**t. We make cognitive constructs in our minds, so as to make at least a little bit of sense of the essentially chaotic reality outside our skin, and we call it empirical observation. Mind you, stuff inside our skin is not much less chaotic, but this is outside the scope of social sciences. As we focus on quantitative variables commonly used in social sciences, the notion of facts becomes really blurred. Have you ever shaken hands with energy efficiency, with Gross Domestic Product or with the mortality rate? Have you touched it? No? Neither have I. These are highly distilled cognitive structures which we use to denote something about the state of society.

Therefore, I assume that quantitative, socio-economic variables express something about the societies observed, and that something is probably important if we collectively keep record of it. If I have n empirical variables, each of them possibly represents collectively important outcomes. As these are distinct variables, I assume that, with all the imperfections and simplification of the corresponding phenomenology, each distinct variable possibly represents a distinct type of collectively important outcome. When I study a human society through the lens of many quantitative variables, I assume they are informative about a set of collectively important social outcomes in that society.

Whilst a regression function explains how many variables are connected when observed ex post and from outside, an adaptation function explains and expresses the way that a society addresses important collective outcomes in a series of trials and errors. Here come two fundamental differences between studying a society with a regression function, as opposed to using an adaptation function. Firstly, for any collection of variables, there is essentially one regression function of the type: y = a₁*x₁ + a₂*x₂ + …+ a_n*x_n + b. On the other hand, with a collection of n quantitative variables at hand, there is at least as many functions of adaptation as there are variables. We can hypothesize that each individual variable x is the collective outcome to pursue and optimize, whilst the remaining n – 1 variables are instrumental to that purpose. One remark is important to make now: the variable informative about collective outcomes pursued, that specific x, can be and usually is instrumental to itself. We can make a desired Gross Domestic Product based on the Gross Domestic Product we have now. The same applies to inflation, energy efficiency, share of electric cars in the overall transportation system etc. Therefore, the entire set of n variables can be assumed instrumental to the optimization of one variable x from among them.

Mathematically, it starts with assuming a functional input f(x₁, x₂, …, x_n) which gets pitched against one specific outcome x_i. Subtraction comes as the most logical representation of that pitching, and thus we have the mathematical expression ‘x_i – f(x₁, x₂, …, x_n)’, which informs about how close the society observed has come to the desired outcome x_i. It is technically possible that people just nail it, and x_i = f(x₁, x₂, …, x_n), whence x_i – f(x₁, x₂, …, x_n) = 0. This is a perfect world, which, however, can be dangerously perfect. We know those societies of apparently perfectly happy people, who live in harmony with nature, even if that harmony means hosting most intestinal parasites of the local ecosystem. One day other people come, with big excavators, monetary systems, structured legal norms, and the bubble bursts, and it hurts.

Thus, on the whole, it might be better to hit x_i ≠ f(x₁, x₂, …, x_n), whence x_i – f(x₁, x₂, …, x_n) ≠ 0. It helps learning new stuff. The ‘≠ 0’ part means there is an error in adaptation. The functional input f(x₁, x₂, …, x_n) hits above or below the desired x_i. As we want to learn, that error in adaptation AKA e = x_i – f(x₁, x₂, …, x_n) ≠ 0, makes any practical sense when we utilize it in subsequent rounds of collective trial and error. Sequence means order, and a timeline. We have a sequence {t₀, t₁, t₂, …, t_m} of m moments in time. Local adaptation turns into ‘x_i(t) – f_t(x₁, x₂, …, x_n)’, and error of adaptation becomes the time-specific e_t = x_i(t) – f_t(x₁, x₂, …, x_n) ≠ 0. The clever trick consists in taking e(t₀) = x_i(t0) – f_t0(x₁, x₂, …, x_n) ≠ 0 and combining it somehow with the next functional input f_t1(x₁, x₂, …, x_n). Mathematically, if we want to combine two values, we can add them up or multiply them. We keep in mind that division is a special case of multiplication, namely x * (1/z). We I add up two values, I assume they are essentially of the same kind and sort of independent from each other. When, on the other hand, I multiply them, they become entwined so that each of them reproduces the other one. Multiplication ‘x * z’ means that x gets reproduced z times and vice versa. When I have the error of adaptation e_t0 from the last experimental round and I want to combine it with the functional input of adaptation f_t1(x₁, x₂, …, x_n) in the next experimental round, that whole reproduction business looks like a strong assumption, with a lot of weak spots on it. I settle for the weak assumption then, and I assume that f_t1(x₁, x₂, …, x_n) becomes f_t0(x₁, x₂, …, x_n) + e(t₀).

The expression f_t0(x₁, x₂, …, x_n) + e(t₀) makes any functional sense only when and after we have e(t₀) = x_i(t0) – f_t0(x₁, x₂, …, x_n) ≠ 0. Consequently, the next error of adaptation, namely e(t₁) = x_i(t₁) – f_t1(x₁, x₂, …, x_n) ≠ 0 can come into being only after its predecessor e_t0 has occurred. We have a chain of m states in the functional input of the society, i.e. {f_t0(x₁, x₂, …, x_n) => f_t1(x₁, x₂, …, x_n) => … => f_tm(x₁, x₂, …, x_n)}, associated with a chain of m desired outcomes {x_i(t₀) => x_i(t₁) => … => x_i(t_m)}, and with a chain of errors in adaptation {e(t₀) => e(t₁) => …=> e(t_m)}. That triad – chain of functional inputs, chain of desired outcomes, and the chain of errors in adaptation – makes for me the closest I can get now to the mathematical expression of the adaptation function. As errors get fed along the chain of states (as I see it, they are being fed forward, but in the algorithmic version, you can backpropagate them), those errors are some sort of dynamic memory in that society, the memory from learning to adapt.

Here we can see the epistemological difference between studying a society from outside, and explaining its workings with a regression function, on the one hand, and studying those mechanisms from inside, by simulation with an adaptation function, on the other hand. Adaptation function is the closest I can get, in mathematical form, to what I understand by collective intelligence. As I have been working with that general construct, I progressively zoomed in on another concept, namely that of intelligent structure, which I define as a structure which learns by experimenting with many alternative versions of itself whilst staying structurally coherent, i.e. by maintaining basic coupling between particular components.

I feel like comparing my approach to intelligent structures and their collective intelligence with the concept of complex systems, as discussed in the literature I have just referred to. I returned, therefore, to the book entitled ‘1177 B.C. The Year Civilisation Collapsed. Revised and Updated’, by Eric H. Cline, Princeton University Press, 2021. The theory of complex systems is brought forth in that otherwise very interesting piece in order to help formulating an answer to the following question: “Why did the great empires of the Late Bronze Age, such as Egypt, the Hittites, or the Myceneans, collapse all in approximately the same time, around 1200 – 1150 B.C.?”. The basic assertion which Eric Cline develops on and questions is that the entire patchwork of those empires in the Mediterranean, the Levant and the Middle East was one big complex system, which collapsed on the account of having overkilled it slightly in the complexity department.

I am trying to reconstruct the definition of systemic complexity such as Eric Cline uses it in his flow of logic. I start with the following quote: ‘Complexity science or theory is the study of a complex system or systems, with the goal of explaining the phenomena which emerge from a collection of interacting objects’. If we study a society as a complex system, we need to assume two things. There are many interacting objects in it, for one, and their mutual interaction leads to the emergence of some specific phenomena. Sounds cool. I move on, and a few pages later I find the following statement: ‘In one aspect of complexity theory, behavior of those objects is affected by their memories and “feedback” from what has happened in the past. They are able to adapt their strategies, partly on the basis of their knowledge of previous history’. Nice. We are getting closer. Entities inside a complex system accumulate memory, and they learn on that basis. This is sort of next door to the three sequential chains: states, desired outcomes, and errors in adaptation, which I coined up.

Further, I find an assertion that a complex social system is typically “alive”, which means that it evolves in a complicated, nontrivial way, whilst being open to influences from the environment. All that leads to the complex system to generate phenomena which can be considered as surprising and extreme. Good. This is the moment to move to the next book: ‘The Black Swan. The impact of the highly improbable’ by Nassim Nicolas Taleb , Penguin, 2010. Here comes a lengthy quote, which I bring here for the sheer pleasure of savouring one more time Nassim Taleb’s delicious style: “[…] say you attribute the success of the nineteenth-century novelist Honoré de Balzac to his superior “realism,” “insights,” “sensitivity,” “treatment of characters,” “ability to keep the reader riveted,” and so on. These may be deemed “superior” qualities that lead to superior performance if, and only if, those who lack what we call talent also lack these qualities. But what if there are dozens of comparable literary masterpieces that happened to perish? And, following my logic, if there are indeed many perished manuscripts with similar attributes, then, I regret to say, your idol Balzac was just the beneficiary of disproportionate luck compared to his peers. Furthermore, you may be committing an injustice to others by favouring him. My point, I will repeat, is not that Balzac is untalented, but that he is less uniquely talented than we think. Just consider the thousands of writers now completely vanished from consciousness: their record does not enter into analyses. We do not see the tons of rejected manuscripts because these writers have never been published. The New Yorker alone rejects close to a hundred manuscripts a day, so imagine the number of geniuses that we will never hear about. In a country like France, where more people write books while, sadly, fewer people read them, respectable literary publishers accept one in ten thousand manuscripts they receive from first-time authors”.

Many people write books, few people read them, and that creates something like a flow of highly risky experiments. That coincides with something like a bottleneck of success, with possibly great positive outcomes (fame, money, posthumous fame, posthumous money for other people etc.), and a low probability of occurrence. A few salient phenomena are produced – the Balzacs – whilst the whole build-up of other writing efforts, by less successful novelists, remains in the backstage of history. That, in turn, somehow rhymes with my intuition that intelligent structures need to produce big outliers, at least from time to time. On the one hand, those outliers can be viewed as big departures from the currently expected outcomes. They are big local errors. Big errors mean a lot of information to learn from. There is an even further-going, conceptual coincidence with the theory and practice of artificial neural networks. A network can be prone to overfitting, which means that it learns too fast, sort of by jumping prematurely to conclusions, before and without having worked through the required work through local errors in adaptation.

Seen from that angle, the function of adaptation I have come up with has a new shade. The sequential chain of errors appears as necessary for the intelligent structure to be any good. Good. Let’s jump to the third book I quoted with respect to the theory of complex systems: ‘Aware. The Science and Practice of Presence. The Ground-breaking Meditation Practice’, by Daniel J. Siegel, TarcherPerigee, 2018. I return to the idea of self-organisation in complex systems, and the choice between three different states: a) the optimal state of flexibility, adaptability, coherence, energy and stability b) non-optimal rigidity and c) non-optimal chaos.

That conceptual thread concurs interestingly with my draft paper: ‘Behavioral absorption of Black Swans: simulation with an artificial neural network’ . I found out that with the chain of functional input states {f_t0(x₁, x₂, …, x_n) => f_t1(x₁, x₂, …, x_n) => … => f_tm(x₁, x₂, …, x_n)} being organized in rigorously the same way, different types of desired outcomes lead to different patterns of learning, very similar to the triad which Daniel Siegel refers to. When my neural network does its best to optimize outcomes such as Gross Domestic Product, it quickly comes to rigidity. It makes some errors in the beginning of the learning process, but then it quickly drives the local error asymptotically to zero and is like ‘We nailed it. There is no need to experiment further’. There are other outcomes, such as the terms of trade (the residual fork between the average price of exports and that of imports), or the average number of hours worked per person per year, which yield a curve of local error in the form of a graceful sinusoid, cyclically oscillating between different magnitudes of error. This is the energetic, dynamic balance. Finally, some macroeconomic outcomes, such as the index of consumer prices, can make the same neural network go nuts, and generate an ever-growing curve of local error, as if the poor thing couldn’t learn anything sensible from looking at the prices of apparel and refrigerators. The (most) puzzling thing in all that differences in pursued outcomes are the source of discrepancy in the patterns of learning, not the way of learning as such. Some outcomes, when pursued, keep the neural network I made in a state of healthy adaptability, whilst other outcomes make it overfit or go haywire.

When I write about collective intelligence and complex system, it can come as a sensible idea to read (and quote) books which have those concepts explicitly named. Here comes ‘The Knowledge Illusion. Why we never think alone’ by Steven Sloman and Philip Fernbach, RIVERHEAD BOOKS (An imprint of Penguin Random House LLC, Ebook ISBN: 9780399184345, Kindle Edition). In the introduction, titled ‘Ignorance and the Community of Knowledge’, Sloman and Fernbach write: “The human mind is not like a desktop computer, designed to hold reams of information. The mind is a flexible problem solver that evolved to extract only the most useful information to guide decisions in new situations. As a consequence, individuals store very little detailed information about the world in their heads. In that sense, people are like bees and society a beehive: Our intelligence resides not in individual brains but in the collective mind. To function, individuals rely not only on knowledge stored within our skulls but also on knowledge stored elsewhere: in our bodies, in the environment, and especially in other people. When you put it all together, human thought is incredibly impressive. But it is a product of a community, not of any individual alone”. This is a strong statement, which I somehow distance myself from. I think that collective human intelligence can be really workable when individual humans are any good at being smart. Individuals need to have practical freedom of action, based on their capacity to figure s**t out in difficult situations, and the highly fluid ensemble of individual freedoms allows the society to make and experiment with many alternative versions of themselves.

Another book is more of a textbook. It is ‘What Is a Complex System?’ by James Landyman and Karoline Wiesner, published with Yale University Press (ISBN 978-0-300-25110-4, Kindle Edition). In the introduction (p.15), Landyman and Wiesner claim: “One of the most fundamental ideas in complexity science is that the interactions of large numbers of entities may give rise to qualitatively new kinds of behaviour different from that displayed by small numbers of them, as Philip Anderson says in his hugely influential paper, ‘more is different’ (1972). When whole systems spontaneously display behaviour that their parts do not, this is called emergence”. In my world, those ‘entities’ are essentially the chained functional input states {f_t0(x₁, x₂, …, x_n) => f_t1(x₁, x₂, …, x_n) => … => f_tm(x₁, x₂, …, x_n)}. My entities are phenomenological – they are cognitive structures which fault of a better word we call ‘empirical variables’. If the neural networks I make and use for my research are any good at representing complex systems, emergence is the property of data in the first place. Interactions between those entities are expressed through the function of adaptation, mostly through the chain {e(t₀) => e(t₁) => …=> e(t_m)} of local errors, concurrent with the chain of functional input states.

I think I know what the central point and thread of my book on collective intelligence is, should I (finally) write that book for good. Artificial neural networks can be used as simulators of collective social behaviour and social change. Still, they do not need to be super-performant network. My point is that with the right intellectual method, even the simplest neural networks, those possible to program into an Excel spreadsheet, can be reliable cognitive tools for social simulation.

[1] LCCN 2020024530 (print) | LCCN 2020024531 (ebook) | ISBN 9780691208015 (paperback) | ISBN 9780691208022 (ebook) ; Cline, Eric H.. 1177 B.C.: 6 (Turning Points in Ancient History, 1) . Princeton University Press. Kindle Edition.

[2] LCCN 2018016987 (print) | LCCN 2018027672 (ebook) | ISBN 9780143111788 | ISBN 9781101993040 (hardback) ; Siegel, Daniel J.. Aware (p. viii). Penguin Publishing Group. Kindle Edition.

[3] Keane, M. (1972). Strongly mixing measures. Inventiones mathematicae, 16(4), 309-324. DOI https://doi.org/10.1007/BF01425715

[4] Berghout, S., & Verbitskiy, E. (2021). On regularity of functions of Markov chains. Stochastic Processes and their Applications, Volume 134, April 2021, Pages 29-54, https://doi.org/10.1016/j.spa.2020.12.006

Plusieurs bouquins à la fois, comme d’habitude

Je suis en train de finir la première version, encore un peu rudimentaire, de mon article sur la faisabilité du « Projet Aqueduc » : un concept technologique en phase de naissance que j’essaie de développer et de promouvoir. Je pense que j’ai fait tous les calculs de base et j’ai l’intention d’en donner un compte rendu sommaire dans cette mise à jour. Je vais présenter ces résultats dans une structure logique qui est en train de faire sa percée dans le monde de la science : je commence par présenter l’idée de base et je l’associe avec du matériel empirique que je juge pertinent ainsi qu’avec la méthode d’analyse de ce matériel. Seulement après la description méthodologique je fais une revue de la littérature à propos des points saillants de la méthode et de l’idée de base. Ces trois composantes de base – introduction, matériel empirique et méthode d’analyse, revue de la littérature – forment la base de ce qui suit, donc de la présentation des calculs et leurs résultats ainsi que la discussion finale du tout. C’est une forme de composition qui est en train de remplacer une structure plus traditionnelle, qui était bâtie autour d’une transition rigoureuse de la théorie vers la partie empirique.

Je commence donc par reformuler et réaffirmer mon idée de base, donc l’essence même de « Projet Aqueduc ». Le travail de recherche que je viens de faire m’a fait changer les idées à ce propos. Initialement, je voyais le « Project Aqueduc » de la façon que vous pouvez voir décrite dans une mise à jour antérieure : « Ça semble expérimenter toujours ». Maintenant, je commence à apprécier la valeur cognitive et pratique de la méthode que j’ai mise au point pour conduire l’étude de faisabilité elle-même. La méthode en question est une application créative (enfin, j’espère) du rasoir d’Ockham : je divise mon concept entier en technologies composantes spécifiques et j’utilise la revue de littérature pour évaluer le degré d’incertitude attaché à chacune de parmi elles. Je concentre l’étude de faisabilité économique sur ce que peux dire de façon à peu près fiable à propos des technologies relativement le plus certaines et j’assume que ces technologies-là doivent générer un surplus de liquidité financière suffisant pour financer le développement de celles relativement plus incertaines.

Dans le cadre du « Projet Aqueduc », ce qui semble le mieux enraciné en termes de calcul ces coûts et d’investissement c’est la technologie de hydro-génération. Celle-ci est bien documentée et bien connue. Pas vraiment beaucoup d’innovation, par ailleurs. ça semble tourner tout seul. Les technologies de, respectivement, stockage d’énergie ainsi que chargement des voitures électriques viennent juste après en termes de prévisibilité : ça bouge, mais ça bouge de façon plutôt organisée. Il y a des innovations à espérer mais je pense que je suis capable de prédire plus ou moins de quelle direction elles vont venir.

Quoi qu’il en soit, j’ai simulé des installations hypothétiques de « Projet Aqueduc » dans les embouchures de 32 rivières de mon pays, la Pologne. J’ai pris les données officielles sur le débit par seconde, en mètres cubes, et j’ai simulé trois niveaux d’adsorption à partir de ce courant, à travers les béliers hydrauliques du « Projet Aqueduc » : 5%, 10% et 20%. En parallèle, j’ai simulé trois élévations possibles des réservoirs d’égalisation : 10 mètres, 20 mètres et 30 mètres. Avec les 654 millimètres de précipitations annuelles moyennes en Pologne, donc avec un ravitaillement hydrologique des précipitations avoisinant 201,8 milliards mètres cubes, ces 32 installations hypothétiques pourraient faire re-circuler entre 2,5% et 10% de ce total. Ceci fait un impact hydrologique substantiel pendant que l’impact sur le marché d’énergie n’est pas vraiment important. Avec l’adsorption d’eau au maximum, soit 20% du débit des rivières en question, ainsi qu’avec l’élévation des réservoirs d’égalisation fixée à 30 mètres (donc le maximum rationnellement possible vu la littérature du sujet), la puissance électrique totale de ces 32 installations hypothétiques serait de quelques 128,9 mégawatts, contre les 50 gigawatts déjà installés dans le système énergétique de la Pologne.

J’écrivais, dans mes mises à jour précédentes, que le « Projet Aqueduc » combine l’impact hydrologique avec celui sur le marché d’énergies renouvelables. Faut que je corrige. La production d’hydro-énergie est tout juste un moyen d’assurer la faisabilité économique du projet et puisque j’en suis là, encore quelques résultats de calculs. Vu les données d’Eurostat sur les prix d’énergie, le « Projet Aqueduc » semble faisable financièrement plutôt avec les prix moyens enregistrés en Europe qu’avec les prix minimum. Avec les prix moyens, l’exploitation des turbines hydroélectriques ainsi que celle d’installations de stockage d’énergie peut dégager quelques 90% de marge brute qui, à son tour, peut servir à financer les autres technologies du projet (pompage avec les béliers hydrauliques, infrastructure hydrologique etc.) et à créer un surplus net de trésorerie. En revanche, lorsque je simule les prix d’énergie à leur minimum empirique, ça donne un déficit brut de -18% après le coût d’énergie et de son stockage. Du coup, le « Projet Aqueduc » n’est pas vraiment du genre « énergies renouvelables pour tous et bon marché ». Le truc a des chances de marcher sans financement publique seulement lorsqu’il touche un marché de consommateurs prêts à payer plus que le minimum pour leur électricité.

En ce qui concerne la station de chargement de véhicules électriques, comme créneau marketing pour l’hydro-énergie produite, je répète tout simplement les conclusions que j’avais déjà exprimées dans la mise à jour intitulée « I have proven myself wrong » : ça n’a pas l’air de pouvoir marcher. A moins de créer une station de chargement hyper-demandée, avec des centaines de chargements par mois, il n’y aura tout simplement pas de trafic suffisant, au moins pas avec les proportions présentes entre la flotte de véhicules électriques en Europe et le réseau des stations de chargement. En revanche, il y a cette idée alternative de stations mobiles de chargement, développé de façon rigoureuse par Elmeligy et al. (2021[1]), par exemple. C’est un changement profond d’approche. Au lieu de construire une station puissante de chargement rapide, couplée avec un magasin d’énergie performant (et cher), on construit un système de batteries mobiles à puissance un peu moins élevée (200 kW dans la solution citée) et on les déplace à travers des parkings fréquentés dans un véhicule spécialement adapté à cette fin.

Maintenant, je change de sujet, mais alors complètement. Hier, j’ai reçu un courriel de la part d’une maison d’édition américaine, Nova Science Publishers, Inc., avec l’invitation à proposer un manuscrit de livre sur le sujet général d’intelligence collective. Apparemment, ils ont lu mon article dans le journal « Energy », intitulé « Energy efficiency as manifestation of collective intelligence in human societies ». Il est aussi possible que quelqu’un chez Nova suit mon blog et ce que je publie sur le phénomène d’intelligence collective. Écrire un livre est différent d’écrire un article. Ce dernier privilégie la concision et la brévité pendant que le premier exige un flot abondant d’idées tout comme un contexte riche et structuré.

En faisant un peu de lecture, ces dernières semaines, je me suis rendu compte que mon hypothèse générale d’intelligence collective des sociétés humaines – donc l’hypothèse d’apprentissage collectif à travers l’expérimentation avec plusieurs versions alternatives de la même structure sociale de base – se marie bien avec l’hypothèse des systèmes complexes. J’ai trouvé cette intersection intéressante comme je lisais le livre intitulé « 1177 B.C. The Year Civilisation Collapsed. Revised and Updated », publié par Eric H. Cline chez Princeton University Press en 2021[2]. En étudiant les mécanismes possibles de la décomposition des grands empires de l’âge de Bronze, Eric Cline cite la théorie des systèmes complexes. Si un ensemble est composé d’entités qui différent dans leur complexité – donc si nous observons entité dans entité et tout ça dans encore une autre entité – les connections fonctionnelles entre ces entités peuvent en quelque sorte stocker l’information et donc générer l’apprentissage spontané. De façon tout à fait surprenante, j’ai trouvé une référence scientifiquement sérieuse à la théorie des systèmes complexes dans un autre bouquin que je suis en train de lire (oui, j’ai l’habitude de lire plusieurs livres à la fois), donc dans « Aware. The Science and Practice of Presence. The Groundbreaking Meditation Practice », publié par Daniel J. Siegel chez TarcherPerigee en 2018[3]. Daniel J. Siegel developpe sur l’hypothèse que la conscience humaine est un système complexe et comme tel est capable d’auto-organisation. Je me permets de traduire ad hoc un court passage du début de ce livre : « L’une des caractéristiques émergentes fondamentales des systèmes complexes dans cette réalité qui est la nôtre est désignée comme auto-organisation. C’est un concept que vous pourriez croire être crée par quelqu’un en psychologie ou même dans les affaires, mais c’est un terme mathématique. La forme ou les contours du déploiement d’un système complexe sont déterminés par cette propriété émergente d’auto-organisation. Ce déploiement peut être optimisé ou bien il peut être contraint. Lorsqu’il ne s’optimise pas, il passe vers chaos ou vers la rigidité. Lorsqu’il s’optimise, il passe vers l’harmonie, en étant flexible, adaptable, cohérent, énergétique et stable ».

Intéressant : une étude systématique du développement et de la chute d’une civilisation peut trouver la même base théorique que l’étude scientifique de la méditation et cette base et la théorie des systèmes complexes. La façon do cette théorie se présente ressemble beaucoup à mes simulations de changement social et technologique où j’utilise des réseaux neuronaux comme représentation d’intelligence collective. Je suis en train de réfléchir sur la façon la plus générale possible d’exprimer et englober mon hypothèse d’intelligence collective. Je pense que le brouillon intitulé « Behavioral absorption of Black Swans: simulation with an artificial neural network », en combinaison avec la théorie des chaînes imparfaites de Markov (Berghout & Verbitskiy 2021[4]) sont peut-être le meilleur point de départ. J’assume donc que toute réalité sociale est une collection des phénomènes que nous ne percevons que de façon partielle et imparfaite et que nous estimons comme saillants lorsque leur probabilité d’occurrence dépasse un certain niveau critique.

Mathématiquement, la réalité sociale intelligible est donc un ensemble de probabilités. Je ne fais aucune assomption à priori quant à la dépendance mutuelle formelle de ces probabilités, mais je peux assumer que nous percevons tout changement de réalité sociale comme passage d’un ensemble des probabilités à un autre, donc comme une chaîne complexe d’états. Ici et maintenant, nous sommes dans une chaîne complexe A et à partir de là, tout n’est pas possible. Bien sûr, je ne veux pas dire que tout est impossible : j’assume tout simplement que la complexité d’ici et maintenant peut se transformer en d’autres complexités sous certaines conditions et contraintes. L’assomption la plus élémentaire à ce propos est que nous envisageons de bouger notre cul collectif seulement vers des états complexes qui nous rapprochent de ce que nous poursuivons ensemble et ceci quelles que soient les contraintes exogènes à notre choix. Je dirais même qu’en présence de contraintes sévères nous devenons particulièrement attentifs à l’état complexe prochain vers lequel nous transigeons. Une société constamment menacée par la pénurie de nourriture, par exemple, va être très tatillonne et en même temps très ingénieuse dans sa propre croissance démographique, en allant même jusqu’à la régulation culturelle du cycle menstruel des femmes.

Bon, ce sera tout dans cette mise à jour. Je m’en vais réfléchir et lire (plusieurs bouquins à la fois, comme d’habitude).

[1] Elmeligy, M. M., Shaaban, M. F., Azab, A., Azzouz, M. A., & Mokhtar, M. (2021). A Mobile Energy Storage Unit Serving Multiple EV Charging Stations. Energies, 14(10), 2969. https://doi.org/10.3390/en14102969

[2] LCCN 2020024530 (print) | LCCN 2020024531 (ebook) | ISBN 9780691208015 (paperback) | ISBN 9780691208022 (ebook) ; Cline, Eric H.. 1177 B.C.: 6 (Turning Points in Ancient History, 1) . Princeton University Press. Kindle Edition.

[3] LCCN 2018016987 (print) | LCCN 2018027672 (ebook) | ISBN 9780143111788 | ISBN 9781101993040 (hardback) ; Siegel, Daniel J.. Aware (p. viii). Penguin Publishing Group. Kindle Edition.

We haven’t nailed down all our equations yet

As I keep digging into the topic of collective intelligence, and my research thereon with the use of artificial neural networks, I am making a list of key empirical findings that pave my way down this particular rabbit hole. I am reinterpreting them with the new understandings I have from translating my mathematical model of artificial neural network into an algorithm. I am learning to program in Python, which comes sort of handy given I want to use AI. How could I have made and used artificial neural networks without programming, just using Excel? You see, that’s Laplace and his hypothesis that mathematics represent the structure of reality (https://discoversocialsciences.com/wp-content/uploads/2020/10/Laplace-A-Philosophical-Essay-on-Probabilities.pdf ).

An artificial neural network is a sequence of equations which interact, in a loop, with a domain of data. Just as any of us, humans, essentially. We just haven’t nailed down all of our own equations yet. What I can do and have done with Excel was to understand the structure of those equations and their order. This is a logical structure, and as long as I don’t give it any domain of data to feed on, is stays put.

When I feed data into that structure, it starts working. Now, with any set of empirical socio-economic variables I have worked with, so far, there is always 1 – 2 among them which are different from others as output. Generally, my neural network works differently according to the output variable I make it optimize. Yes, it is the output variable, supposedly being the desired outcome to optimize, and not the input variables treated as instrumental in that view, which makes the greatest difference in the results produced by the network.

That seems counterintuitive, and yet this is like the most fundamental common denominator of everything I have found out so far: the way that a simple neural network simulates the collective intelligence of human societies seems to be conditioned most of all by the variables pre-set as the output of the adaptation process, not by the input ones. Is it a sensible conclusion regarding collective intelligence in real life, or is it just a property of the data? In other words, is it social science or data science? This is precisely one of the questions which I want to answer by learning programming.

If it is a pattern of collective human intelligence, that would mean we are driven by the orientations pursued much more than by the actual perception of reality. What we are after would be more important a differentiating factor of your actions than what we perceive and experience as reality. Strangely congruent with the Interface Theory of Perception (Hoffman et al. 2015 [1], Fields et al. 2018 [2]).

As it is some kind of habit in me, in the second part of this update I put the account of my learning how to program and to Data Science in Python. This time, I wanted to work with hard cases of CSV import, like trouble files. I want to practice data cleansing. I have downloaded the ‘World Economic Outlook October 2020’ database from the website https://www.imf.org/en/Publications/WEO/weo-database/2020/October/download-entire-database . Already when downloading, I could notice that the announced format is ‘TAB delimited’, not ‘Comma Separated’. It downloads as Excel.

To start with, I used the https://anyconv.com/tab-to-csv-converter/ website to do the conversion. In parallel, I tested two other ways:

opening in Excel, and then saving as CSV
opening with Excel, converting to *.TXT, importing into Wizard for MacOS (statistical package), and then exporting as CSV.

What I can see like right off the bat are different sizes in the same data, technically saved in the same format. The AnyConv-generated CSV is 12,3 MB, the one converted through Excel is 9,6 MB, and the last one, filtered through Excel to TXT, then to Wizard and to CSV makes 10,1 MB. Intriguing.

I open JupyterLab online, and I create a Python 3-based Notebook titled ‘Practice 27_11_2020_part2’.

I prepare the Notebook by importing Numpy, Pandas, Matplotlib and OS. I do:

>> import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

import os

I upload the AnyConv version of the CSV. I make sure to have the name of the file right by doing:

>> os.listdir() 

…and I do:

>> WEO1=pd.DataFrame(pd.read_csv(‘AnyConv__WEOOct2020all.csv’))

Result:

/srv/conda/envs/notebook/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3072: DtypeWarning: Columns (83,85,87,89,91,93,95,98,99,102,103,106,107,110,111,114,115,118,119,122,123,126,127,130,131,134,135,138,139,142,143,146,147,150,151,154,155,158) have mixed types. Specify dtype option on import or set low_memory=False.

interactivity=interactivity, compiler=compiler, result=result)

As I have been told, I add the “low_memory=False” option to the command, and I retype:

>> WEO1=pd.DataFrame(pd.read_csv(‘AnyConv__WEOOct2020all.csv’, low_memory=False))

Result: the file is apparently imported successfully. I investigate the structure.

>> WEO1.describe()

Result: I know I have 8 rows (there should be much more, over 200), and 32 columns. Something is wrong.

I upload the Excel-converted CSV.

>> WEO2=pd.DataFrame(pd.read_csv(‘WEOOct2020all_Excel.csv’))

Result: Parser error

I retry, with parameter sep=‘;’ (usually works with Excel)

>> WEO2=pd.DataFrame(pd.read_csv(‘WEOOct2020all_Excel.csv’,sep=’;’))

Result: import successful. Let’s check the shape of the data

>> WEO2.describe()

Result: Pandas can see just the last column. I make sure.

>> WEO2.columns

Result:

Index([‘WEO Country Code’, ‘ISO’, ‘WEO Subject Code’, ‘Country’,

‘Subject Descriptor’, ‘Subject Notes’, ‘Units’, ‘Scale’,

‘Country/Series-specific Notes’, ‘1980’, ‘1981’, ‘1982’, ‘1983’, ‘1984’,

‘1985’, ‘1986’, ‘1987’, ‘1988’, ‘1989’, ‘1990’, ‘1991’, ‘1992’, ‘1993’,

‘1994’, ‘1995’, ‘1996’, ‘1997’, ‘1998’, ‘1999’, ‘2000’, ‘2001’, ‘2002’,

‘2003’, ‘2004’, ‘2005’, ‘2006’, ‘2007’, ‘2008’, ‘2009’, ‘2010’, ‘2011’,

‘2012’, ‘2013’, ‘2014’, ‘2015’, ‘2016’, ‘2017’, ‘2018’, ‘2019’, ‘2020’,

‘2021’, ‘2022’, ‘2023’, ‘2024’, ‘2025’, ‘Estimates Start After’],

dtype=’object’)

I will try to import the same file with a different ‘sep’ parameter, this time as sep=‘\t’

>> WEO3=pd.DataFrame(pd.read_csv(‘WEOOct2020all_Excel.csv’,sep=’\t’))

Result: import apparently successful. I check the shape of my data.

>> WEO3.describe()

Result: apparently, this time, no column is distinguished.

When I type:

>> WEO3.columns

…I get

Index([‘WEO Country Code;ISO;WEO Subject Code;Country;Subject Descriptor;Subject Notes;Units;Scale;Country/Series-specific Notes;1980;1981;1982;1983;1984;1985;1986;1987;1988;1989;1990;1991;1992;1993;1994;1995;1996;1997;1998;1999;2000;2001;2002;2003;2004;2005;2006;2007;2008;2009;2010;2011;2012;2013;2014;2015;2016;2017;2018;2019;2020;2021;2022;2023;2024;2025;Estimates Start After’], dtype=’object’)

Now, I test with the 3rd file, the one converted through Wizard.

>> WEO4=pd.DataFrame(pd.read_csv(‘WEOOct2020all_Wizard.csv’))

Result: import successful.

I check the shape.

>> WEO4.describe()

Result: still just 8 rows. Something is wrong.

I do another experiment. I take the original*.XLS from imf.org, and I save it as regular Excel *.XLSX, and then I save this one as CSV.

>> WEO5=pd.DataFrame(pd.read_csv(‘WEOOct2020all_XLSX.csv’))

Result: parser error

I will retry with two options as for the separator: sep=‘;’ and sep=‘\t’. Ledzeee…

>> WEO5=pd.DataFrame(pd.read_csv(‘WEOOct2020all_XLSX.csv’,sep=’;’))

Import successful. “WEO5.describe()” yields just one column.

>> WEO6=pd.DataFrame(pd.read_csv(‘WEOOct2020all_XLSX.csv’,sep=’\t’))

yields successful import, yet all the data is just one long row, without separation into columns.

I check WEO5 and WEO6 with “*.index”, and “*.shape”.

“WEO5.index” yields “RangeIndex(start=0, stop=8777, step=1)”

“WEO6.index” yields “RangeIndex(start=0, stop=8777, step=1)

“WEO5.shape” gives “(8777, 56)”

“WEO6.shape” gives “(8777, 1)”

Depending on the separator given as parameter in the “pd.read_csv” command, I get 56 columns or just 1 column, yet the “*.describe()” command cannot make sense of them.

I try the *.describe” command, thus more specific than the “*.describe()” one.

I can see that structures are clearly different.

I try another trick, namely to assume separator ‘;’ and TAB delimiter.

>> WEO7=pd.DataFrame(pd.read_csv(‘WEOOct2020all_XLSX.csv’,sep=’;’,delimiter=’\t’))

Result: WEO7.shape yields 8777 rows in just one column.

Maybe ‘header=0’? Same thing.

The provisional moral of the fairy tale is that ‘Data cleansing’ means very largely making sense of the exact shape and syntax of CSV files. Depending on the parametrisation of separators and delimiters, different Data Frames are obtained.

[1] Hoffman, D. D., Singh, M., & Prakash, C. (2015). The interface theory of perception. Psychonomic bulletin & review, 22(6), 1480-1506.

[2] Fields, C., Hoffman, D. D., Prakash, C., & Singh, M. (2018). Conscious agent networks: Formal analysis and application to cognition. Cognitive Systems Research, 47, 186-213. https://doi.org/10.1016/j.cogsys.2017.10.003

I re-run my executable script

I am thinking (again) about the phenomenon of collective intelligence, this time in terms of behavioural reinforcement that we give to each other, and the role that cities and intelligent digital clouds can play in delivering such reinforcement. As it is usually the case with science, there is a basic question to ask: ‘What’s the point of all the fuss with that nice theory of yours, Mr Wasniewski? Any good for anything?’.

Good question. My tentative answer is that studying human societies as collectively intelligent structures is a phenomenology, which allows some major methodological developments, which, I think, are missing from other methodologies in social sciences. First of all, it allows a completely clean slate at the starting point of research, as regards ethics and moral orientations, whilst it almost inevitably leads to defining ethical values through empirical research. This was my first big ‘Oh, f**k!’ with that method: I realized that ethical values can be reliably studied as objectively pursued outcomes at the collective level, and that study can be robustly backed with maths and empirics.

I have that thing with my science, and, as a matter of fact, with other people’s science too: I am an empiricist. I like prodding my assumptions and make them lose some fat, so as they become lighter. I like having as much of a clean slate at the starting point of my research as possible. I believe that one single assumption, namely that human social structures are collectively intelligent structures, almost automatically transforms all the other assumptions into hypotheses to investigate. Still, I need to go, very carefully, through that one single Mother Of All Assumptions, i.e. about us, humans as a society, being collectively intelligent a structure, in order to nail down, and possibly kick out any logical shortcut.

Intelligent structures learn by producing many alternative versions of themselves and testing those versions for fitness in coping with a vector of constraints. There are three claims hidden in this single claim: learning, production of different versions, and testing for fitness. Do human social structures learn, like at all? Well, we have that thing called culture, and culture changes. There is observable change in lifestyles, aesthetic tastes, fashions, institutions and technologies. This is learning. Cool. One down, two still standing.

Do human social structures produce many different versions of themselves? Here, we enter the subtleties of distinction between different versions of a structure, on the one hand, and different structures, on the other hand. A structure remains the same, and just makes different versions of itself, as long as it stays structurally coherent. When it loses structural coherence, it turns into a different structure. How can I know that a structure keeps its s**t together, i.e. it stays internally coherent? That’s a tough question, and I know by experience that in the presence of tough questions, it is essential to keep it simple. One of the simplest facts about any structure is that it is made of parts. As long as all the initial parts are still there, I can assume they hold together somehow. In other words, as long as whatever I observe about social reality can be represented as the same complex set, with the same components inside, I can assume this is one and the same structure just making copies of itself. Still, this question remains a tough one, especially that any intelligent structure should be smart enough to morph into another intelligent structure when the time is right.

The time is right when the old structure is no longer able to cope with the vector of constraints, and so I arrive to the third component question: how can I know there is adaptation to constraints? How can I know there are constraints for assessing fitness? In a very broad sense, I can see constraints when I see error, and correction thereof, in someone’s behaviour. In other words, when I can see someone sort of making two steps forward and one step back, correcting their course etc., this is a sign of adaptation to constraints. Unconstrained change is linear or exponential, whilst constrained change always shows signs of bumping against some kind of wall. Here comes a caveat as regards using artificial neural networks as simulators of collective human intelligence: they are any good only when they have constraints, and, consequently, when they make errors. An artificial neural network is no good at simulating unconstrained change. When I explore the possibility of simulating collective human intelligence with artificial neural networks, it has marks of a pleonasm. I can use AI as simulator only when the simulation involves constrained adaptation.

F**k! I have gone philosophical in those paragraphs. I can feel a part of my mind gently disconnecting from real life, and this is time to do something in order to stay close to said real life. Here is a topic, which I can treat as teaching material for my students, and, in the same time, make those general concepts bounce a bit around, inside my head, just to see what happens. I make the following claim: ‘Markets are manifestations of collective intelligence in human societies’. In science, this is a working hypothesis. It is called ‘working’ because it is not proven yet, and thus it has to earn its own living, so to say. This is why it has to work.

I pass in review the same bullet points: learning, for one, production of many alternative versions in a structure as opposed to creating new structures, for two, and the presence of constraints as the third component. Do markets manifest collective learning? Ledzzeee… Markets display fashions and trends. Markets adapt to lifestyles, and vice versa. Markets are very largely connected to technological change and facilitate the occurrence thereof. Yes, they learn.

How can I say whether a market stays the same structure and just experiments with many alternative versions thereof, or, conversely, whether it turns into another structure? It is time to go back to the fundamental concepts of microeconomics, and assess (once more), what makes a market structure. A market structure is the mechanism of setting transactional prices. When I don’t know s**t about said mechanism, I just observe prices and I can see two alternative pictures. Picture one is that of very similar prices, sort of clustered in the same, narrow interval. This is a market with equilibrium price, which translates into a local market equilibrium. Picture two shows noticeably disparate prices in what I initially perceived as the same category of goods. There is no equilibrium price in that case, and speaking more broadly, there is no local equilibrium in that market.

Markets with local equilibriums are assumed to be perfectly competitive or very close thereto. They are supposed to serve for transacting in goods so similar that customers perceive them as identical, and technologies used for producing those goods don’t differ sufficiently to create any kind of competitive advantage (homogeneity of supply), for one. Markets with local equilibriums require the customers to be so similar to each other in their tastes and purchasing patterns that, on the whole, they can be assumed identical (homogeneity of demand), for two. Customers are supposed to be perfectly informed about all the deals available in the market (perfect information). Oh, yes, the last one: no barriers to entry or exit. A perfectly competitive market is supposed to offer virtually no minimum investment required for suppliers to enter the game, and no sunk costs in the case of exit.

Here is that thing: many markets present the alignment of prices typical for a state of local equilibrium, and yet their institutional characteristics – such as technologies, the diversity of goods offered, capital requirements and whatnot – do not match the textbook description of a perfectly competitive market. In other words, many markets form local equilibriums, thus they display equilibrium prices, without having the required institutional characteristics for that, at least in theory. In still other words, they manifest the alignment of prices typical for one type of market structure, whilst all the other characteristics are typical for another type of market structure.

Therefore, the completely justified ‘What the hell…?’question arises. What is a market structure, at the end of the day? What is a structure, in general?

I go down another avenue now. Some time ago, I signalled on my blog that I am learning programming in Python, or, as I should rather say, I make one more attempt at nailing it down. Programming teaches me a lot about the basic logic of what I do, including that whole theory of collective intelligence. Anyway, I started to keep a programming log, and here below, I paste the current entry, from November 27^th, 2020.

Tasks to practice:

reading well structured CSV,
plotting
saving and retrieving a Jupyter Notebook in JupyterLab

I am practicing with Penn World Tables 9.1. I take the version without empty cells, and I transform it into CSV.

I create a new notebook on JupyterLab. I name it ‘Practice November 27th 2020’.

Shareable link :https://hub.gke2.mybinder.org/user/jupyterlab-jupyterlab-demo-zbo0hr9b/lab/tree/demo/Practice%20November%2027th%202020.ipynb

Path: demo/Practice November 27th 2020.ipynb

Download Path:https://hub.gke2.mybinder.org/user/jupyterlab-jupyterlab-demo-zbo0hr9b/files/demo/Practice%20November%2027th%202020.ipynb?_xsrf=2%7C2ce78815%7C547592bc83c83fd951870ab01113e7eb%7C1605464585

I upload the CSV version of Penn Tables 9.1 with no empty cells.

Shareable link: https://hub.gke2.mybinder.org/user/jupyterlab-jupyterlab-demo-zbo0hr9b/lab/tree/demo/PWT%209_1%20no%20empty%20cells.csv

Path: demo/PWT 9_1 no empty cells.csv

Download path: https://hub.gke2.mybinder.org/user/jupyterlab-jupyterlab-demo-zbo0hr9b/files/demo/PWT%209_1%20no%20empty%20cells.csv?_xsrf=2%7C2ce78815%7C547592bc83c83fd951870ab01113e7eb%7C1605464585

I code libraries:

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

import os

I check my directory:

>> os.getcwd()

result: ‘/home/jovyan/demo’

>> os.listdir()

result:

[‘jupyterlab.md’,

‘TCGA_Data’,

‘Lorenz.ipynb’,

‘lorenz.py’,

‘notebooks’,

‘data’,

‘jupyterlab-slides.pdf’,

‘markdown_python.md’,

‘big.csv’,

‘Practice November 27th 2020.ipynb’,

‘.ipynb_checkpoints’,

‘Untitled.ipynb’,

‘PWT 9_1 no empty cells.csv’]

>> PWT9_1=pd.DataFrame(pd.read_csv(‘PWT 9_1 no empty cells.csv’,header=0))

Result:

File “<ipython-input-5-32375ff59964>”, line 1

PWT9_1=pd.DataFrame(pd.read_csv(‘PWT 9_1 no empty cells.csv’,header=0))

SyntaxError: invalid character in identifier

>> I rename the file on Jupyter, into ‘PWT 9w1 no empty cells.csv’.

>> os.listdir()

Result:

[‘jupyterlab.md’,

‘TCGA_Data’,

‘Lorenz.ipynb’,

‘lorenz.py’,

‘notebooks’,

‘data’,

‘jupyterlab-slides.pdf’,

‘markdown_python.md’,

‘big.csv’,

‘Practice November 27th 2020.ipynb’,

‘.ipynb_checkpoints’,

‘Untitled.ipynb’,

‘PWT 9w1 no empty cells.csv’]

>> PWT9w1=pd.DataFrame(pd.read_csv(‘PWT 9w1 no empty cells.csv’,header=0))

Result: imported successfully

>> PWT9w1.describe()

Result: descriptive statistics

# I want to list columns (variables) in my file

>> PWT9w1.columns

Result:

Index([‘country’, ‘year’, ‘rgdpe’, ‘rgdpo’, ‘pop’, ’emp’, ’emp / pop’, ‘avh’,

‘hc’, ‘ccon’, ‘cda’, ‘cgdpe’, ‘cgdpo’, ‘cn’, ‘ck’, ‘ctfp’, ‘cwtfp’,

‘rgdpna’, ‘rconna’, ‘rdana’, ‘rnna’, ‘rkna’, ‘rtfpna’, ‘rwtfpna’,

‘labsh’, ‘irr’, ‘delta’, ‘xr’, ‘pl_con’, ‘pl_da’, ‘pl_gdpo’, ‘csh_c’,

‘csh_i’, ‘csh_g’, ‘csh_x’, ‘csh_m’, ‘csh_r’, ‘pl_c’, ‘pl_i’, ‘pl_g’,

‘pl_x’, ‘pl_m’, ‘pl_n’, ‘pl_k’],

dtype=’object’)

>> PWT9w1.columns()

Result:

TypeError Traceback (most recent call last)

<ipython-input-11-38dfd3da71de> in <module>

—-> 1 PWT9w1.columns()

TypeError: ‘Index’ object is not callable

# I try plotting

>> plt.plot(df.index, df[‘rnna’])

Result:

I get a long list of rows like: ‘<matplotlib.lines.Line2D at 0x7fc59d899c10>’, and a plot which is visibly not OK (looks like a fan).

# I want to separate one column from PWT9w1 as a separate series, and then plot it. Maybe it is going to work.

>> RNNA=pd.DataFrame(PWT9w1[‘rnna’])

Result: apparently successful.

# I try to plot RNNA

>> RNNA.plot()

Result:

<matplotlib.axes._subplots.AxesSubplot at 0x7fc55e7b9e10> + a basic graph. Good.

# I try to extract a few single series from PWT9w1 and to plot them. Let’s go for AVH, PL_I and CWTFP.

>> AVH=pd.DataFrame(PWT9w1[‘avh’])

>> PL_I=pd.DataFrame(PWT9w1[‘pl_i’])

>> CWTFP=pd.DataFrame(PWT9w1[‘cwtfp’])

>> AVH.plot()

>> PL_I.plot()

>> CWTFP.plot()

Result:

It worked. I have basic plots.

# It is 8:20 a.m. I go to make myself a coffee. I will quit JupyterLab for a moment. I saved my today’s notebook on server, and I will see how I can open it. Just in case, I make a PDF copy, and a Python copy on my disk.

I cannot do saving into PDF. An error occurs. I will have to sort it out. I made an *.ipynb copy on my disk.

demo/Practice November 27th 2020.ipynb

# It is 8:40 a.m. I am logging back into JupyterLab. I am trying to open my today’s notebook from path. Does not seem to work. I am uploading my *.ipynb copy. This worked. I know now: I upload the *.ipynb script from my own location and then just double click on it. I needed to re-upload my CSV file ‘PWT 9w1 no empty cells.csv’.

# I check if my re-uploaded CSV file is fully accessible. I discover that I need to re-create the whole algorithm. In other words: when I upload on JupyterLab a *.ipynb script from my disk, I need to re-run all the operations. My first idea is to re-run each executable cell in the uploaded script. That worked. Question: how to automatise it? Probably by making a Python script all in one piece, uploading my CSV data source first, and then run the whole script.

I like being a mad scientist

I like being a mad scientist. Am I a mad scientist? A tiny bit, yes, ‘cause I do research on things just because I feel like. Mind you, me being that mad scientist I like being happens to be practical. Those rabbit holes I dive into prove to have interesting outcomes in real life.

I feel like writing, and therefore thinking in an articulate way, about two things I do in parallel: science and investment. I have just realized these two realms of activity tend to merge and overlap in me. When I do science, I tend to think like an investor, or a gardener. I invest my personal energy in ideas which I think have potential for growth. On the other hand, I invest in the stock market with a strong dose of curiosity. Those companies, and the investment positions I can open therein, are like animals which I observe, try to figure out how not to get killed by them, or by predators that hunt them, and I try to domesticate those beasts.

The scientific thing I am working on is the application of artificial intelligence to studying collective intelligence in human societies. The thing I am working on sort of at the crest between science and investment is fundraising for scientific projects (my new job at the university).

The project aims at defining theoretical and empirical fundamentals for using intelligent digital clouds, i.e. large datasets combined with artificial neural networks, in the field of remote digital diagnostics and remote digital care, in medical sciences and medical engineering. That general purpose translates into science strictly speaking, and into the prospective development of medical technologies.

There is observable growth in the percentage of population using various forms of digital remote diagnostics and healthcare. Yet, that growth is very uneven across different social groups, which suggests an early, pre-popular stage of development in those technologies (Mahajan et al. 2020^[i]). Other research confirms that supposition, as judging by the very disparate results obtained with those technologies, in terms of diagnostic and therapeutic effectiveness (Cheng et al. 2020^[ii]; Wong et al. 2020^[iii]). There are known solutions where intelligent digital cloud allows transforming the patient’s place of stay (home, apartment) into the local substitute of a hospital bed, which opens interesting possibilities as regards medical care for patients with significantly reduced mobility, e.g. geriatric patients (Ben Hassen et al. 2020^[iv]). Already around 2015, creative applications of medical imagery appeared, where the camera of a person’s smartphone served for early detection of skin cancer (Bliznuks et al. 2017^[v]). The connection between distance diagnostics with the acquisition and processing of image comes as one of the most interesting and challenging innovations to make in the here-discussed field of technology (Marwan et al. 2018^[vi]). The experience of COVID-19 pandemic has already showed the potential of digital intelligent clouds in assisting national healthcare systems, especially in optimising and providing flexibility to the use of resources, both material and human (Alashhab et al. 2020^[vii]). Yet, the same pandemic experience has shown the depth of social disparities as regards real actual access to digital technologies supported by intelligent clouds (Whitelaw et al. 2020^[viii]). Intelligent digital clouds enter into learning-generative interactions with the professionals of healthcare. There is observable behavioural modification, for example, in students of healthcare who train with such technologies from the very beginning of their education (Brown Wilson et al. 2020^[ix]). That phenomenon of behavioural change requires rethinking from scratch, with the development of each individual technology, the ethical and legal issues relative to interactions between users, on the one hand, and system operators, on the other hand (Godding 2019^[x]).

Against that general background, the present project focuses on studying the phenomenon of tacit coordination among the users of digital technologies in remote medical diagnostics and remote medical care. Tacit coordination is essential as regards the well-founded application of intelligent digital cloud to support and enhance these technologies. Intelligent digital clouds are intelligent structures, i.e. they learn by producing many alternative versions of themselves and testing those versions for fitness in coping with a vector of external constraints. It is important to explore the extent and way that populations of users behave similarly, i.e. as collectively intelligent structures. The deep theoretical meaning of that exploration is the extent to which the intelligent structure of a digital cloud really maps and represents the collectively intelligent structure of the users’ population.

The scientific method used in the project explores the main working hypothesis that populations of actual and/or prospective patients, in their own health-related behaviour, and in their relations with the healthcare systems, are collectively intelligent structures, with tacit coordination. In practical terms, that hypothesis means that any intelligent digital cloud in the domain of remote medical care should assume collectively intelligent, thus more than just individual, behavioural change on the part of users. Collectively intelligent behavioural change in a population, marked by tacit coordination, is a long-term, evolutionary process of adaptive walk in rugged landscape (Kauffman & Levin 1987^[xi]; Nahum et al. 2015^[xii]). Therefore, it is something deeper and more durable that fashions and styles. It is the deep, underlying mechanism of social change accompanying the use of digital intelligent clouds in medical engineering.

The scientific method used in this project aims at exploring and checking the above-stated working hypothesis by creating a large and differentiated dataset of health-related data, and processing that dataset in an intelligent digital cloud, in two distinct phases. The first phase consists in processing a first sample of data with a relatively simple, artificial neural network, in order to discover its underlying orientations and its mechanisms of collective learning. The second phase allows an intelligent digital cloud to respond adaptively to users behaviour, i.e to produce intelligent interaction with them. The first phase serves to understand the process of adaptation observable in the second phase. Both phases are explained more in detail below.

The tests of, respectively, orientation and mode of learning, in the first phase of empirical research aim at defining the vector of collectively pursued social outcomes in the population studied. The initially collected empirical dataset is transformed, with the use of an artificial neural network, into as many representations as there are variables in the set, with each representation being oriented on a different variable as its output (with the remaining ones considered as instrumental input). Each such transformation of the initial set can be tested for its mathematical similarity therewith (e.g. for Euclidean distance between the vectors of expected mean values). Transformations displaying relatively the greatest similarity to the source dataset are assumed to be the most representative for the collectively intelligent structure in the population studied, and, consequently, their output variables can be assumed to represent collectively pursued social outcomes in that collective intelligence (see, for example: Wasniewski 2020^[xiii]). Modes of learning in that dataset can be discovered by creating a shadow vector of probabilities (representing, for example, a finite set of social roles endorsed with given probabilities by members of the population), and a shadow process that introduces random disturbance, akin to the theory of Black Swans (Taleb 2007^[xiv]; Taleb & Blyth 2011^[xv]). The so-created shadow structure is subsequently transformed with an artificial neural network in as many alternative versions as there are variables in the source empirical dataset, each version taking a different variable from the set as its pre-set output. Three different modes of learning can be observed, and assigned to particular variables: a) cyclical adjustment without clear end-state b) finite optimisation with defined end-state and c) structural disintegration with growing amplitude of oscillation around central states.

The above-summarised first phase of research involves the use of two basic digital tools, i.e. an online functionality to collect empirical data from and about patients, and an artificial neural network to process it. There comes an important aspect of that first phase in research, i.e. the actual collectability and capacity to process the corresponding data. It can be assumed that comprehensive medical care involves the collection of both strictly health-related data (e.g. blood pressure, blood sugar etc.), and peripheral data of various kinds (environmental, behavioural). The complexity of data collected in that phase can be additionally enhanced by including imagery such as pictures taken with smartphones (e.g. skin, facial symmetry etc.). In that respect, the first phase of research aims at testing the actual possibility and reliability of collection in various types of data. Phenomena such as outliers of fake data can be detected then.

Once the first phase is finished and expressed in the form of theoretical conclusions, the second phase of research is triggered. An intelligent digital cloud is created, with the capacity of intelligent adaptation to users’ behaviour. A very basic example of such adaptation are behavioural reinforcements. The cloud can generate simple messages of praise for health-functional behaviour (positive reinforcements), or, conversely, warning messages in the case of health-dysfunctional behaviour (negative reinforcements). More elaborate form of intelligent adaptation are possible to implement, e.g. a Twitter-like reinforcement to create trending information, or a Tik-Tok-like reinforcement to stay in the loop of communication in the cloud. This phase aims specifically at defining the actually workable scope and strength of possible behavioural reinforcements which a digital functionality in the domain of healthcare could use vis a vis its end users. Legal and ethical implications thereof are studied as one of the theoretical outcomes of that second phase.

I feel like generalizing a bit my last few updates, and to develop on the general hypothesis of collectively intelligent, human social structures. In order to consider any social structure as manifestation of collective intelligence, I need to place intelligence in a specific empirical context. I need an otherwise exogenous environment, which the social structure has to adapt to. Empirical study of collective intelligence, such as I have been doing it, and, as a matter of fact, the only one I know how to do, consists in studying adaptive effort in human social structures.

^[i] Shiwani Mahajan, Yuan Lu, Erica S. Spatz, Khurram Nasir, Harlan M. Krumholz, Trends and Predictors of Use of Digital Health Technology in the United States, The American Journal of Medicine, 2020, ISSN 0002-9343, https://doi.org/10.1016/j.amjmed.2020.06.033 (http://www.sciencedirect.com/science/article/pii/S0002934320306173 )

^[ii] Lei Cheng, Mingxia Duan, Xiaorong Mao, Youhong Ge, Yanqing Wang, Haiying Huang, The effect of digital health technologies on managing symptoms across pediatric cancer continuum: A systematic review, International Journal of Nursing Sciences, 2020, ISSN 2352-0132, https://doi.org/10.1016/j.ijnss.2020.10.002 , (http://www.sciencedirect.com/science/article/pii/S2352013220301630 )

^[iii] Charlene A. Wong, Farrah Madanay, Elizabeth M. Ozer, Sion K. Harris, Megan Moore, Samuel O. Master, Megan Moreno, Elissa R. Weitzman, Digital Health Technology to Enhance Adolescent and Young Adult Clinical Preventive Services: Affordances and Challenges, Journal of Adolescent Health, Volume 67, Issue 2, Supplement, 2020, Pages S24-S33, ISSN 1054-139X, https://doi.org/10.1016/j.jadohealth.2019.10.018 , (http://www.sciencedirect.com/science/article/pii/S1054139X19308675 )

^[iv] Hassen, H. B., Ayari, N., & Hamdi, B. (2020). A home hospitalization system based on the Internet of things, Fog computing and cloud computing. Informatics in Medicine Unlocked, 100368, https://doi.org/10.1016/j.imu.2020.100368

^[v] Bliznuks, D., Bolocko, K., Sisojevs, A., & Ayub, K. (2017). Towards the Scalable Cloud Platform for Non-Invasive Skin Cancer Diagnostics. Procedia Computer Science, 104, 468-476

^[vi] Marwan, M., Kartit, A., & Ouahmane, H. (2018). Security enhancement in healthcare cloud using machine learning. Procedia Computer Science, 127, 388-397.

^[vii] Alashhab, Z. R., Anbar, M., Singh, M. M., Leau, Y. B., Al-Sai, Z. A., & Alhayja’a, S. A. (2020). Impact of Coronavirus Pandemic Crisis on Technologies and Cloud Computing Applications. Journal of Electronic Science and Technology, 100059. https://doi.org/10.1016/j.jnlest.2020.100059

^[viii] Whitelaw, S., Mamas, M. A., Topol, E., & Van Spall, H. G. (2020). Applications of digital technology in COVID-19 pandemic planning and response. The Lancet Digital Health. https://doi.org/10.1016/S2589-7500(20)30142-4

^[ix] Christine Brown Wilson, Christine Slade, Wai Yee Amy Wong, Ann Peacock, Health care students experience of using digital technology in patient care: A scoping review of the literature, Nurse Education Today, Volume 95, 2020, 104580, ISSN 0260-6917, https://doi.org/10.1016/j.nedt.2020.104580 ,(http://www.sciencedirect.com/science/article/pii/S0260691720314301 )

^[x] Piers Gooding, Mapping the rise of digital mental health technologies: Emerging issues for law and society, International Journal of Law and Psychiatry, Volume 67, 2019, 101498, ISSN 0160-2527, https://doi.org/10.1016/j.ijlp.2019.101498 , (http://www.sciencedirect.com/science/article/pii/S0160252719300950 )

^[xi] Kauffman, S., & Levin, S. (1987). Towards a general theory of adaptive walks on rugged landscapes. Journal of theoretical Biology, 128(1), 11-45

^[xii] Nahum, J. R., Godfrey-Smith, P., Harding, B. N., Marcus, J. H., Carlson-Stevermer, J., & Kerr, B. (2015). A tortoise–hare pattern seen in adapting structured and unstructured populations suggests a rugged fitness landscape in bacteria. Proceedings of the National Academy of Sciences, 112(24), 7530-7535, www.pnas.org/cgi/doi/10.1073/pnas.1410631112

^[xiii] Wasniewski, K. (2020). Energy efficiency as manifestation of collective intelligence in human societies. Energy, 191, 116500. https://doi.org/10.1016/j.energy.2019.116500

^[xiv] Taleb, N. N. (2007). The black swan: The impact of the highly improbable (Vol. 2). Random house

^[xv] Taleb, N. N., & Blyth, M. (2011). The black swan of Cairo: How suppressing volatility makes the world less predictable and more dangerous. Foreign Affairs, 33-39

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this: