We haven’t nailed down all our equations yet

As I keep digging into the topic of collective intelligence, and my research thereon with the use of artificial neural networks, I am making a list of key empirical findings that pave my way down this particular rabbit hole. I am reinterpreting them with the new understandings I have from translating my mathematical model of artificial neural network into an algorithm. I am learning to program in Python, which comes sort of handy given I want to use AI. How could I have made and used artificial neural networks without programming, just using Excel? You see, that’s Laplace and his hypothesis that mathematics represent the structure of reality (https://discoversocialsciences.com/wp-content/uploads/2020/10/Laplace-A-Philosophical-Essay-on-Probabilities.pdf ).

An artificial neural network is a sequence of equations which interact, in a loop, with a domain of data. Just as any of us, humans, essentially. We just haven’t nailed down all of our own equations yet. What I can do and have done with Excel was to understand the structure of those equations and their order. This is a logical structure, and as long as I don’t give it any domain of data to feed on, is stays put.

When I feed data into that structure, it starts working. Now, with any set of empirical socio-economic variables I have worked with, so far, there is always 1 – 2 among them which are different from others as output. Generally, my neural network works differently according to the output variable I make it optimize. Yes, it is the output variable, supposedly being the desired outcome to optimize, and not the input variables treated as instrumental in that view, which makes the greatest difference in the results produced by the network.

That seems counterintuitive, and yet this is like the most fundamental common denominator of everything I have found out so far: the way that a simple neural network simulates the collective intelligence of human societies seems to be conditioned most of all by the variables pre-set as the output of the adaptation process, not by the input ones. Is it a sensible conclusion regarding collective intelligence in real life, or is it just a property of the data? In other words, is it social science or data science? This is precisely one of the questions which I want to answer by learning programming.

If it is a pattern of collective human intelligence, that would mean we are driven by the orientations pursued much more than by the actual perception of reality. What we are after would be more important a differentiating factor of your actions than what we perceive and experience as reality. Strangely congruent with the Interface Theory of Perception (Hoffman et al. 2015[1], Fields et al. 2018[2]). 

As it is some kind of habit in me, in the second part of this update I put the account of my learning how to program and to Data Science in Python. This time, I wanted to work with hard cases of CSV import, like trouble files. I want to practice data cleansing. I have downloaded the ‘World Economic Outlook October 2020’ database from the website https://www.imf.org/en/Publications/WEO/weo-database/2020/October/download-entire-database . Already when downloading, I could notice that the announced format is ‘TAB delimited’, not ‘Comma Separated’. It downloads as Excel.

To start with, I used the https://anyconv.com/tab-to-csv-converter/ website to do the conversion. In parallel, I tested two other ways:

  1. opening in Excel, and then saving as CSV
  2. opening with Excel, converting to *.TXT, importing into Wizard for MacOS (statistical package), and then exporting as CSV.

What I can see like right off the bat are different sizes in the same data, technically saved in the same format. The AnyConv-generated CSV is 12,3 MB, the one converted through Excel is 9,6 MB, and the last one, filtered through Excel to TXT, then to Wizard and to CSV makes 10,1 MB. Intriguing.

I open JupyterLab online, and I create a Python 3-based Notebook titled ‘Practice 27_11_2020_part2’.

I prepare the Notebook by importing Numpy, Pandas, Matplotlib and OS. I do:

>> import numpy as np

      import pandas as pd

      import matplotlib.pyplot as plt

      import os

I upload the AnyConv version of the CSV. I make sure to have the name of the file right by doing:

>> os.listdir()


…and I do:

>> WEO1=pd.DataFrame(pd.read_csv(‘AnyConv__WEOOct2020all.csv’))

Result:

/srv/conda/envs/notebook/lib/python3.7/site-packages/IPython/core/interactiveshell.py:3072: DtypeWarning: Columns (83,85,87,89,91,93,95,98,99,102,103,106,107,110,111,114,115,118,119,122,123,126,127,130,131,134,135,138,139,142,143,146,147,150,151,154,155,158) have mixed types. Specify dtype option on import or set low_memory=False.

  interactivity=interactivity, compiler=compiler, result=result)

As I have been told, I add the “low_memory=False” option to the command, and I retype:

>> WEO1=pd.DataFrame(pd.read_csv(‘AnyConv__WEOOct2020all.csv’, low_memory=False))

Result: the file is apparently imported successfully. I investigate the structure.

>> WEO1.describe()

Result: I know I have 8 rows (there should be much more, over 200), and 32 columns. Something is wrong.

I upload the Excel-converted CSV.

>> WEO2=pd.DataFrame(pd.read_csv(‘WEOOct2020all_Excel.csv’))

Result: Parser error

I retry, with parameter sep=‘;’ (usually works with Excel)

>> WEO2=pd.DataFrame(pd.read_csv(‘WEOOct2020all_Excel.csv’,sep=’;’))

Result: import successful. Let’s check the shape of the data

>> WEO2.describe()

Result: Pandas can see just the last column. I make sure.

>> WEO2.columns

Result:

Index([‘WEO Country Code’, ‘ISO’, ‘WEO Subject Code’, ‘Country’,

       ‘Subject Descriptor’, ‘Subject Notes’, ‘Units’, ‘Scale’,

       ‘Country/Series-specific Notes’, ‘1980’, ‘1981’, ‘1982’, ‘1983’, ‘1984’,

       ‘1985’, ‘1986’, ‘1987’, ‘1988’, ‘1989’, ‘1990’, ‘1991’, ‘1992’, ‘1993’,

       ‘1994’, ‘1995’, ‘1996’, ‘1997’, ‘1998’, ‘1999’, ‘2000’, ‘2001’, ‘2002’,

       ‘2003’, ‘2004’, ‘2005’, ‘2006’, ‘2007’, ‘2008’, ‘2009’, ‘2010’, ‘2011’,

       ‘2012’, ‘2013’, ‘2014’, ‘2015’, ‘2016’, ‘2017’, ‘2018’, ‘2019’, ‘2020’,

       ‘2021’, ‘2022’, ‘2023’, ‘2024’, ‘2025’, ‘Estimates Start After’],

      dtype=’object’)

I will try to import the same file with a different ‘sep’ parameter, this time as sep=‘\t’

>> WEO3=pd.DataFrame(pd.read_csv(‘WEOOct2020all_Excel.csv’,sep=’\t’))

Result: import apparently successful. I check the shape of my data.

>> WEO3.describe()

Result: apparently, this time, no column is distinguished.

When I type:

>> WEO3.columns

…I get

Index([‘WEO Country Code;ISO;WEO Subject Code;Country;Subject Descriptor;Subject Notes;Units;Scale;Country/Series-specific Notes;1980;1981;1982;1983;1984;1985;1986;1987;1988;1989;1990;1991;1992;1993;1994;1995;1996;1997;1998;1999;2000;2001;2002;2003;2004;2005;2006;2007;2008;2009;2010;2011;2012;2013;2014;2015;2016;2017;2018;2019;2020;2021;2022;2023;2024;2025;Estimates Start After’], dtype=’object’)

Now, I test with the 3rd file, the one converted through Wizard.

>> WEO4=pd.DataFrame(pd.read_csv(‘WEOOct2020all_Wizard.csv’))

Result: import successful.

I check the shape.

>> WEO4.describe()

Result: still just 8 rows. Something is wrong.

I do another experiment. I take the original*.XLS from imf.org, and I save it as regular Excel *.XLSX, and then I save this one as CSV.

>> WEO5=pd.DataFrame(pd.read_csv(‘WEOOct2020all_XLSX.csv’))

Result: parser error

I will retry with two options as for the separator: sep=‘;’ and sep=‘\t’. Ledzeee…

>> WEO5=pd.DataFrame(pd.read_csv(‘WEOOct2020all_XLSX.csv’,sep=’;’))

Import successful. “WEO5.describe()” yields just one column.

>> WEO6=pd.DataFrame(pd.read_csv(‘WEOOct2020all_XLSX.csv’,sep=’\t’))

yields successful import, yet all the data is just one long row, without separation into columns.

I check WEO5 and WEO6 with “*.index”, and “*.shape”. 

“WEO5.index” yields “RangeIndex(start=0, stop=8777, step=1)”

“WEO6.index” yields “RangeIndex(start=0, stop=8777, step=1)

“WEO5.shape” gives “(8777, 56)”

“WEO6.shape” gives “(8777, 1)”

Depending on the separator given as parameter in the “pd.read_csv” command, I get 56 columns or just 1 column, yet the “*.describe()” command cannot make sense of them.

I try the *.describe” command, thus more specific than the “*.describe()” one.

I can see that structures are clearly different.

I try another trick, namely to assume separator ‘;’ and TAB delimiter.

>> WEO7=pd.DataFrame(pd.read_csv(‘WEOOct2020all_XLSX.csv’,sep=’;’,delimiter=’\t’))

Result: WEO7.shape yields 8777 rows in just one column.

Maybe ‘header=0’? Same thing.

The provisional moral of the fairy tale is that ‘Data cleansing’ means very largely making sense of the exact shape and syntax of CSV files. Depending on the parametrisation of separators and delimiters, different Data Frames are obtained.


[1] Hoffman, D. D., Singh, M., & Prakash, C. (2015). The interface theory of perception. Psychonomic bulletin & review, 22(6), 1480-1506.

[2] Fields, C., Hoffman, D. D., Prakash, C., & Singh, M. (2018). Conscious agent networks: Formal analysis and application to cognition. Cognitive Systems Research, 47, 186-213. https://doi.org/10.1016/j.cogsys.2017.10.003

I re-run my executable script

I am thinking (again) about the phenomenon of collective intelligence, this time in terms of behavioural reinforcement that we give to each other, and the role that cities and intelligent digital clouds can play in delivering such reinforcement. As it is usually the case with science, there is a basic question to ask: ‘What’s the point of all the fuss with that nice theory of yours, Mr Wasniewski? Any good for anything?’.

Good question. My tentative answer is that studying human societies as collectively intelligent structures is a phenomenology, which allows some major methodological developments, which, I think, are missing from other methodologies in social sciences. First of all, it allows a completely clean slate at the starting point of research, as regards ethics and moral orientations, whilst it almost inevitably leads to defining ethical values through empirical research. This was my first big ‘Oh, f**k!’ with that method: I realized that ethical values can be reliably studied as objectively pursued outcomes at the collective level, and that study can be robustly backed with maths and empirics.

I have that thing with my science, and, as a matter of fact, with other people’s science too: I am an empiricist. I like prodding my assumptions and make them lose some fat, so as they become lighter. I like having as much of a clean slate at the starting point of my research as possible. I believe that one single assumption, namely that human social structures are collectively intelligent structures, almost automatically transforms all the other assumptions into hypotheses to investigate. Still, I need to go, very carefully, through that one single Mother Of All Assumptions, i.e. about us, humans as a society, being collectively intelligent a structure, in order to nail down, and possibly kick out any logical shortcut.

Intelligent structures learn by producing many alternative versions of themselves and testing those versions for fitness in coping with a vector of constraints. There are three claims hidden in this single claim: learning, production of different versions, and testing for fitness. Do human social structures learn, like at all? Well, we have that thing called culture, and culture changes. There is observable change in lifestyles, aesthetic tastes, fashions, institutions and technologies. This is learning. Cool. One down, two still standing.

Do human social structures produce many different versions of themselves? Here, we enter the subtleties of distinction between different versions of a structure, on the one hand, and different structures, on the other hand. A structure remains the same, and just makes different versions of itself, as long as it stays structurally coherent. When it loses structural coherence, it turns into a different structure. How can I know that a structure keeps its s**t together, i.e. it stays internally coherent? That’s a tough question, and I know by experience that in the presence of tough questions, it is essential to keep it simple. One of the simplest facts about any structure is that it is made of parts. As long as all the initial parts are still there, I can assume they hold together somehow. In other words, as long as whatever I observe about social reality can be represented as the same complex set, with the same components inside, I can assume this is one and the same structure just making copies of itself. Still, this question remains a tough one, especially that any intelligent structure should be smart enough to morph into another intelligent structure when the time is right.      

The time is right when the old structure is no longer able to cope with the vector of constraints, and so I arrive to the third component question: how can I know there is adaptation to constraints? How can I know there are constraints for assessing fitness? In a very broad sense, I can see constraints when I see error, and correction thereof, in someone’s behaviour. In other words, when I can see someone sort of making two steps forward and one step back, correcting their course etc., this is a sign of adaptation to constraints. Unconstrained change is linear or exponential, whilst constrained change always shows signs of bumping against some kind of wall. Here comes a caveat as regards using artificial neural networks as simulators of collective human intelligence: they are any good only when they have constraints, and, consequently, when they make errors. An artificial neural network is no good at simulating unconstrained change. When I explore the possibility of simulating collective human intelligence with artificial neural networks, it has marks of a pleonasm. I can use AI as simulator only when the simulation involves constrained adaptation.

F**k! I have gone philosophical in those paragraphs. I can feel a part of my mind gently disconnecting from real life, and this is time to do something in order to stay close to said real life. Here is a topic, which I can treat as teaching material for my students, and, in the same time, make those general concepts bounce a bit around, inside my head, just to see what happens. I make the following claim: ‘Markets are manifestations of collective intelligence in human societies’. In science, this is a working hypothesis. It is called ‘working’ because it is not proven yet, and thus it has to earn its own living, so to say. This is why it has to work.

I pass in review the same bullet points: learning, for one, production of many alternative versions in a structure as opposed to creating new structures, for two, and the presence of constraints as the third component. Do markets manifest collective learning? Ledzzeee… Markets display fashions and trends. Markets adapt to lifestyles, and vice versa. Markets are very largely connected to technological change and facilitate the occurrence thereof. Yes, they learn.

How can I say whether a market stays the same structure and just experiments with many alternative versions thereof, or, conversely, whether it turns into another structure? It is time to go back to the fundamental concepts of microeconomics, and assess (once more), what makes a market structure. A market structure is the mechanism of setting transactional prices. When I don’t know s**t about said mechanism, I just observe prices and I can see two alternative pictures. Picture one is that of very similar prices, sort of clustered in the same, narrow interval. This is a market with equilibrium price, which translates into a local market equilibrium. Picture two shows noticeably disparate prices in what I initially perceived as the same category of goods. There is no equilibrium price in that case, and speaking more broadly, there is no local equilibrium in that market.

Markets with local equilibriums are assumed to be perfectly competitive or very close thereto. They are supposed to serve for transacting in goods so similar that customers perceive them as identical, and technologies used for producing those goods don’t differ sufficiently to create any kind of competitive advantage (homogeneity of supply), for one. Markets with local equilibriums require the customers to be so similar to each other in their tastes and purchasing patterns that, on the whole, they can be assumed identical (homogeneity of demand), for two. Customers are supposed to be perfectly informed about all the deals available in the market (perfect information). Oh, yes, the last one: no barriers to entry or exit. A perfectly competitive market is supposed to offer virtually no minimum investment required for suppliers to enter the game, and no sunk costs in the case of exit.  

Here is that thing: many markets present the alignment of prices typical for a state of local equilibrium, and yet their institutional characteristics – such as technologies, the diversity of goods offered, capital requirements and whatnot – do not match the textbook description of a perfectly competitive market. In other words, many markets form local equilibriums, thus they display equilibrium prices, without having the required institutional characteristics for that, at least in theory. In still other words, they manifest the alignment of prices typical for one type of market structure, whilst all the other characteristics are typical for another type of market structure.

Therefore, the completely justified ‘What the hell…?’question arises. What is a market structure, at the end of the day? What is a structure, in general?

I go down another avenue now. Some time ago, I signalled on my blog that I am learning programming in Python, or, as I should rather say, I make one more attempt at nailing it down. Programming teaches me a lot about the basic logic of what I do, including that whole theory of collective intelligence. Anyway, I started to keep a programming log, and here below, I paste the current entry, from November 27th, 2020.

 Tasks to practice:

  1. reading well structured CSV,
  2. plotting
  3. saving and retrieving a Jupyter Notebook in JupyterLab

I am practicing with Penn World Tables 9.1. I take the version without empty cells, and I transform it into CSV.

I create a new notebook on JupyterLab. I name it ‘Practice November 27th 2020’.

  • Path: demo/Practice November 27th 2020.ipynb

I upload the CSV version of Penn Tables 9.1 with no empty cells.

Shareable link: https://hub.gke2.mybinder.org/user/jupyterlab-jupyterlab-demo-zbo0hr9b/lab/tree/demo/PWT%209_1%20no%20empty%20cells.csv

Path: demo/PWT 9_1 no empty cells.csv

Download path: https://hub.gke2.mybinder.org/user/jupyterlab-jupyterlab-demo-zbo0hr9b/files/demo/PWT%209_1%20no%20empty%20cells.csv?_xsrf=2%7C2ce78815%7C547592bc83c83fd951870ab01113e7eb%7C1605464585

I code libraries:

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

import os

I check my directory:

>> os.getcwd()

result: ‘/home/jovyan/demo’

>> os.listdir()

result:

[‘jupyterlab.md’,

 ‘TCGA_Data’,

 ‘Lorenz.ipynb’,

 ‘lorenz.py’,

 ‘notebooks’,

 ‘data’,

 ‘jupyterlab-slides.pdf’,

 ‘markdown_python.md’,

 ‘big.csv’,

 ‘Practice November 27th 2020.ipynb’,

 ‘.ipynb_checkpoints’,

 ‘Untitled.ipynb’,

 ‘PWT 9_1 no empty cells.csv’]

>> PWT9_1=pd.DataFrame(pd.read_csv(‘PWT 9_1 no empty cells.csv’,header=0))

Result:

  File “<ipython-input-5-32375ff59964>”, line 1

    PWT9_1=pd.DataFrame(pd.read_csv(‘PWT 9_1 no empty cells.csv’,header=0))

                                       ^

SyntaxError: invalid character in identifier

>> I rename the file on Jupyter, into ‘PWT 9w1 no empty cells.csv’.

>> os.listdir()

Result:

[‘jupyterlab.md’,

 ‘TCGA_Data’,

 ‘Lorenz.ipynb’,

 ‘lorenz.py’,

 ‘notebooks’,

 ‘data’,

 ‘jupyterlab-slides.pdf’,

 ‘markdown_python.md’,

 ‘big.csv’,

 ‘Practice November 27th 2020.ipynb’,

 ‘.ipynb_checkpoints’,

 ‘Untitled.ipynb’,

 ‘PWT 9w1 no empty cells.csv’]

>> PWT9w1=pd.DataFrame(pd.read_csv(‘PWT 9w1 no empty cells.csv’,header=0))

Result: imported successfully

>> PWT9w1.describe()

Result: descriptive statistics

# I want to list columns (variables) in my file

>> PWT9w1.columns

Result:

Index([‘country’, ‘year’, ‘rgdpe’, ‘rgdpo’, ‘pop’, ’emp’, ’emp / pop’, ‘avh’,

       ‘hc’, ‘ccon’, ‘cda’, ‘cgdpe’, ‘cgdpo’, ‘cn’, ‘ck’, ‘ctfp’, ‘cwtfp’,

       ‘rgdpna’, ‘rconna’, ‘rdana’, ‘rnna’, ‘rkna’, ‘rtfpna’, ‘rwtfpna’,

       ‘labsh’, ‘irr’, ‘delta’, ‘xr’, ‘pl_con’, ‘pl_da’, ‘pl_gdpo’, ‘csh_c’,

       ‘csh_i’, ‘csh_g’, ‘csh_x’, ‘csh_m’, ‘csh_r’, ‘pl_c’, ‘pl_i’, ‘pl_g’,

       ‘pl_x’, ‘pl_m’, ‘pl_n’, ‘pl_k’],

      dtype=’object’)

>> PWT9w1.columns()

Result:

TypeError                                 Traceback (most recent call last)

<ipython-input-11-38dfd3da71de> in <module>

—-> 1 PWT9w1.columns()

TypeError: ‘Index’ object is not callable

# I try plotting

>> plt.plot(df.index, df[‘rnna’])

Result:

I get a long list of rows like: ‘<matplotlib.lines.Line2D at 0x7fc59d899c10>’, and a plot which is visibly not OK (looks like a fan).

# I want to separate one column from PWT9w1 as a separate series, and then plot it. Maybe it is going to work.

>> RNNA=pd.DataFrame(PWT9w1[‘rnna’])

Result: apparently successful.

# I try to plot RNNA

>> RNNA.plot()

Result:

<matplotlib.axes._subplots.AxesSubplot at 0x7fc55e7b9e10> + a basic graph. Good.

# I try to extract a few single series from PWT9w1 and to plot them. Let’s go for AVH, PL_I and CWTFP.

>> AVH=pd.DataFrame(PWT9w1[‘avh’])

>> PL_I=pd.DataFrame(PWT9w1[‘pl_i’])

>> CWTFP=pd.DataFrame(PWT9w1[‘cwtfp’])

>> AVH.plot()

>> PL_I.plot()

>> CWTFP.plot()

Result:

It worked. I have basic plots.

# It is 8:20 a.m. I go to make myself a coffee. I will quit JupyterLab for a moment. I saved my today’s notebook on server, and I will see how I can open it. Just in case, I make a PDF copy, and a Python copy on my disk.

I cannot do saving into PDF. An error occurs. I will have to sort it out. I made an *.ipynb copy on my disk.

demo/Practice November 27th 2020.ipynb

# It is 8:40 a.m. I am logging back into JupyterLab. I am trying to open my today’s notebook from path. Does not seem to work. I am uploading my *.ipynb copy. This worked. I know now: I upload the *.ipynb script from my own location and then just double click on it. I needed to re-upload my CSV file ‘PWT 9w1 no empty cells.csv’.

# I check if my re-uploaded CSV file is fully accessible. I discover that I need to re-create the whole algorithm. In other words: when I upload on JupyterLab a *.ipynb script from my disk, I need to re-run all the operations. My first idea is to re-run each executable cell in the uploaded script. That worked. Question: how to automatise it? Probably by making a Python script all in one piece, uploading my CSV data source first, and then run the whole script.

I like being a mad scientist

I like being a mad scientist. Am I a mad scientist? A tiny bit, yes, ‘cause I do research on things just because I feel like. Mind you, me being that mad scientist I like being happens to be practical. Those rabbit holes I dive into prove to have interesting outcomes in real life.

I feel like writing, and therefore thinking in an articulate way, about two things I do in parallel: science and investment. I have just realized these two realms of activity tend to merge and overlap in me. When I do science, I tend to think like an investor, or a gardener. I invest my personal energy in ideas which I think have potential for growth. On the other hand, I invest in the stock market with a strong dose of curiosity. Those companies, and the investment positions I can open therein, are like animals which I observe, try to figure out how not to get killed by them, or by predators that hunt them, and I try to domesticate those beasts.

The scientific thing I am working on is the application of artificial intelligence to studying collective intelligence in human societies. The thing I am working on sort of at the crest between science and investment is fundraising for scientific projects (my new job at the university).

The project aims at defining theoretical and empirical fundamentals for using intelligent digital clouds, i.e. large datasets combined with artificial neural networks, in the field of remote digital diagnostics and remote digital care, in medical sciences and medical engineering. That general purpose translates into science strictly speaking, and into the prospective development of medical technologies.

There is observable growth in the percentage of population using various forms of digital remote diagnostics and healthcare. Yet, that growth is very uneven across different social groups, which suggests an early, pre-popular stage of development in those technologies (Mahajan et al. 2020[i]). Other research confirms that supposition, as judging by the very disparate results obtained with those technologies, in terms of diagnostic and therapeutic effectiveness (Cheng et al. 2020[ii]; Wong et al. 2020[iii]). There are known solutions where intelligent digital cloud allows transforming the patient’s place of stay (home, apartment) into the local substitute of a hospital bed, which opens interesting possibilities as regards medical care for patients with significantly reduced mobility, e.g. geriatric patients (Ben Hassen et al. 2020[iv]). Already around 2015, creative applications of medical imagery appeared, where the camera of a person’s smartphone served for early detection of skin cancer (Bliznuks et al. 2017[v]). The connection between distance diagnostics with the acquisition and processing of image comes as one of the most interesting and challenging innovations to make in the here-discussed field of technology (Marwan et al. 2018[vi]). The experience of COVID-19 pandemic has already showed the potential of digital intelligent clouds in assisting national healthcare systems, especially in optimising and providing flexibility to the use of resources, both material and human (Alashhab et al. 2020[vii]). Yet, the same pandemic experience has shown the depth of social disparities as regards real actual access to digital technologies supported by intelligent clouds (Whitelaw et al. 2020[viii]). Intelligent digital clouds enter into learning-generative interactions with the professionals of healthcare. There is observable behavioural modification, for example, in students of healthcare who train with such technologies from the very beginning of their education (Brown Wilson et al. 2020[ix]). That phenomenon of behavioural change requires rethinking from scratch, with the development of each individual technology, the ethical and legal issues relative to interactions between users, on the one hand, and system operators, on the other hand (Godding 2019[x]).

Against that general background, the present project focuses on studying the phenomenon of tacit coordination among the users of digital technologies in remote medical diagnostics and remote medical care. Tacit coordination is essential as regards the well-founded application of intelligent digital cloud to support and enhance these technologies. Intelligent digital clouds are intelligent structures, i.e. they learn by producing many alternative versions of themselves and testing those versions for fitness in coping with a vector of external constraints. It is important to explore the extent and way that populations of users behave similarly, i.e. as collectively intelligent structures. The deep theoretical meaning of that exploration is the extent to which the intelligent structure of a digital cloud really maps and represents the collectively intelligent structure of the users’ population.

The scientific method used in the project explores the main working hypothesis that populations of actual and/or prospective patients, in their own health-related behaviour, and in their relations with the healthcare systems, are collectively intelligent structures, with tacit coordination. In practical terms, that hypothesis means that any intelligent digital cloud in the domain of remote medical care should assume collectively intelligent, thus more than just individual, behavioural change on the part of users. Collectively intelligent behavioural change in a population, marked by tacit coordination, is a long-term, evolutionary process of adaptive walk in rugged landscape (Kauffman & Levin 1987[xi]; Nahum et al. 2015[xii]). Therefore, it is something deeper and more durable that fashions and styles. It is the deep, underlying mechanism of social change accompanying the use of digital intelligent clouds in medical engineering.

The scientific method used in this project aims at exploring and checking the above-stated working hypothesis by creating a large and differentiated dataset of health-related data, and processing that dataset in an intelligent digital cloud, in two distinct phases. The first phase consists in processing a first sample of data with a relatively simple, artificial neural network, in order to discover its underlying orientations and its mechanisms of collective learning. The second phase allows an intelligent digital cloud to respond adaptively to users behaviour, i.e to produce intelligent interaction with them. The first phase serves to understand the process of adaptation observable in the second phase. Both phases are explained more in detail below.

The tests of, respectively, orientation and mode of learning, in the first phase of empirical research aim at defining the vector of collectively pursued social outcomes in the population studied. The initially collected empirical dataset is transformed, with the use of an artificial neural network, into as many representations as there are variables in the set, with each representation being oriented on a different variable as its output (with the remaining ones considered as instrumental input). Each such transformation of the initial set can be tested for its mathematical similarity therewith (e.g. for Euclidean distance between the vectors of expected mean values). Transformations displaying relatively the greatest similarity to the source dataset are assumed to be the most representative for the collectively intelligent structure in the population studied, and, consequently, their output variables can be assumed to represent collectively pursued social outcomes in that collective intelligence (see, for example: Wasniewski 2020[xiii]). Modes of learning in that dataset can be discovered by creating a shadow vector of probabilities (representing, for example, a finite set of social roles endorsed with given probabilities by members of the population), and a shadow process that introduces random disturbance, akin to the theory of Black Swans (Taleb 2007[xiv]; Taleb & Blyth 2011[xv]). The so-created shadow structure is subsequently transformed with an artificial neural network in as many alternative versions as there are variables in the source empirical dataset, each version taking a different variable from the set as its pre-set output. Three different modes of learning can be observed, and assigned to particular variables: a) cyclical adjustment without clear end-state b) finite optimisation with defined end-state and c) structural disintegration with growing amplitude of oscillation around central states.

The above-summarised first phase of research involves the use of two basic digital tools, i.e. an online functionality to collect empirical data from and about patients, and an artificial neural network to process it. There comes an important aspect of that first phase in research, i.e. the actual collectability and capacity to process the corresponding data. It can be assumed that comprehensive medical care involves the collection of both strictly health-related data (e.g. blood pressure, blood sugar etc.), and peripheral data of various kinds (environmental, behavioural). The complexity of data collected in that phase can be additionally enhanced by including imagery such as pictures taken with smartphones (e.g. skin, facial symmetry etc.). In that respect, the first phase of research aims at testing the actual possibility and reliability of collection in various types of data. Phenomena such as outliers of fake data can be detected then.

Once the first phase is finished and expressed in the form of theoretical conclusions, the second phase of research is triggered. An intelligent digital cloud is created, with the capacity of intelligent adaptation to users’ behaviour. A very basic example of such adaptation are behavioural reinforcements. The cloud can generate simple messages of praise for health-functional behaviour (positive reinforcements), or, conversely, warning messages in the case of health-dysfunctional behaviour (negative reinforcements). More elaborate form of intelligent adaptation are possible to implement, e.g. a Twitter-like reinforcement to create trending information, or a Tik-Tok-like reinforcement to stay in the loop of communication in the cloud. This phase aims specifically at defining the actually workable scope and strength of possible behavioural reinforcements which a digital functionality in the domain of healthcare could use vis a vis its end users. Legal and ethical implications thereof are studied as one of the theoretical outcomes of that second phase.

I feel like generalizing a bit my last few updates, and to develop on the general hypothesis of collectively intelligent, human social structures. In order to consider any social structure as manifestation of collective intelligence, I need to place intelligence in a specific empirical context. I need an otherwise exogenous environment, which the social structure has to adapt to. Empirical study of collective intelligence, such as I have been doing it, and, as a matter of fact, the only one I know how to do, consists in studying adaptive effort in human social structures. 


[i] Shiwani Mahajan, Yuan Lu, Erica S. Spatz, Khurram Nasir, Harlan M. Krumholz, Trends and Predictors of Use of Digital Health Technology in the United States, The American Journal of Medicine, 2020, ISSN 0002-9343, https://doi.org/10.1016/j.amjmed.2020.06.033 (http://www.sciencedirect.com/science/article/pii/S0002934320306173  )

[ii] Lei Cheng, Mingxia Duan, Xiaorong Mao, Youhong Ge, Yanqing Wang, Haiying Huang, The effect of digital health technologies on managing symptoms across pediatric cancer continuum: A systematic review, International Journal of Nursing Sciences, 2020, ISSN 2352-0132, https://doi.org/10.1016/j.ijnss.2020.10.002 , (http://www.sciencedirect.com/science/article/pii/S2352013220301630 )

[iii] Charlene A. Wong, Farrah Madanay, Elizabeth M. Ozer, Sion K. Harris, Megan Moore, Samuel O. Master, Megan Moreno, Elissa R. Weitzman, Digital Health Technology to Enhance Adolescent and Young Adult Clinical Preventive Services: Affordances and Challenges, Journal of Adolescent Health, Volume 67, Issue 2, Supplement, 2020, Pages S24-S33, ISSN 1054-139X, https://doi.org/10.1016/j.jadohealth.2019.10.018 , (http://www.sciencedirect.com/science/article/pii/S1054139X19308675 )

[iv] Hassen, H. B., Ayari, N., & Hamdi, B. (2020). A home hospitalization system based on the Internet of things, Fog computing and cloud computing. Informatics in Medicine Unlocked, 100368, https://doi.org/10.1016/j.imu.2020.100368

[v] Bliznuks, D., Bolocko, K., Sisojevs, A., & Ayub, K. (2017). Towards the Scalable Cloud Platform for Non-Invasive Skin Cancer Diagnostics. Procedia Computer Science, 104, 468-476

[vi] Marwan, M., Kartit, A., & Ouahmane, H. (2018). Security enhancement in healthcare cloud using machine learning. Procedia Computer Science, 127, 388-397.

[vii] Alashhab, Z. R., Anbar, M., Singh, M. M., Leau, Y. B., Al-Sai, Z. A., & Alhayja’a, S. A. (2020). Impact of Coronavirus Pandemic Crisis on Technologies and Cloud Computing Applications. Journal of Electronic Science and Technology, 100059. https://doi.org/10.1016/j.jnlest.2020.100059

[viii] Whitelaw, S., Mamas, M. A., Topol, E., & Van Spall, H. G. (2020). Applications of digital technology in COVID-19 pandemic planning and response. The Lancet Digital Health. https://doi.org/10.1016/S2589-7500(20)30142-4

[ix] Christine Brown Wilson, Christine Slade, Wai Yee Amy Wong, Ann Peacock, Health care students experience of using digital technology in patient care: A scoping review of the literature, Nurse Education Today, Volume 95, 2020, 104580, ISSN 0260-6917, https://doi.org/10.1016/j.nedt.2020.104580 ,(http://www.sciencedirect.com/science/article/pii/S0260691720314301 )

[x] Piers Gooding, Mapping the rise of digital mental health technologies: Emerging issues for law and society, International Journal of Law and Psychiatry, Volume 67, 2019, 101498, ISSN 0160-2527, https://doi.org/10.1016/j.ijlp.2019.101498 , (http://www.sciencedirect.com/science/article/pii/S0160252719300950 )

[xi] Kauffman, S., & Levin, S. (1987). Towards a general theory of adaptive walks on rugged landscapes. Journal of theoretical Biology, 128(1), 11-45

[xii] Nahum, J. R., Godfrey-Smith, P., Harding, B. N., Marcus, J. H., Carlson-Stevermer, J., & Kerr, B. (2015). A tortoise–hare pattern seen in adapting structured and unstructured populations suggests a rugged fitness landscape in bacteria. Proceedings of the National Academy of Sciences, 112(24), 7530-7535, www.pnas.org/cgi/doi/10.1073/pnas.1410631112 

[xiii] Wasniewski, K. (2020). Energy efficiency as manifestation of collective intelligence in human societies. Energy, 191, 116500. https://doi.org/10.1016/j.energy.2019.116500

[xiv] Taleb, N. N. (2007). The black swan: The impact of the highly improbable (Vol. 2). Random house

[xv] Taleb, N. N., & Blyth, M. (2011). The black swan of Cairo: How suppressing volatility makes the world less predictable and more dangerous. Foreign Affairs, 33-39

Checkpoint for business

I am changing the path of my writing, ‘cause real life knocks at my door, and it goes ‘Hey, scientist, you economist, right? Good, ‘cause there is some good stuff, I mean, ideas for business. That’s economics, right? Just sort of real stuff, OK?’. Sure. I can go with real things, but first, I explain. At my university, I have recently taken on the job of coordinating research projects and finding some financing for them. One of the first things I did, right after November 1st, was to send around a reminder that we had 12 days left to apply, with the Ministry of Science and Higher Education, for relatively small grants, in a call titled ‘Students make innovation’. Honestly, I was expecting to have 1 – 2 applications max, in response. Yet, life can make surprises. There are 7 innovative ideas in terms of feedback, and 5 of them look like good material for business concepts and for serious development. I am taking on giving them a first prod, in terms of business planning. Interestingly, those ideas are all related to medical technologies, thus something I have been both investing a lot in, during 2020, and thinking a lot about, as a possible path of substantial technological change.

I am progressively wrapping my mind up around ideas and projects formulated by those students, and, walking down the same intellectual avenue, I am making sense of making money on and around science. I am fully appreciating the value of real-life experience. I have been doing research and writing about technological change for years. Until recently, I had that strange sort of complex logical oxymoron in my mind, where I had the impression of both understanding technological change, and missing a fundamental aspect of it. Now, I think I start to understand that missing part: it is the microeconomic mechanism of innovation.

I have collected those 5 ideas from ambitious students at Faculty of Medicine, in my university:

>> Idea 1: An AI-based app, with a chatbot, which facilitates early diagnosis of cardio-vascular diseases

>> Idea 2: Similar thing, i.e. a mobile app, but oriented on early diagnosis and monitoring of urinary incontinence in women.

>> Idea 3: Technology for early diagnosis of Parkinson’s disease, through the observation of speech and motor disturbance.

>> Idea 4: Intelligent cloud to store, study and possibly find something smart about two types of data: basic health data (blood-work etc.), and environmental factors (pollution, climate etc.).

>> Idea 5: Something similar to Idea 4, i.e. an intelligent cloud with medical edge, but oriented on storing and studying data from large cohorts of patients infected with Sars-Cov-2. 

As I look at those 5 ideas, surprisingly simple and basic association of ideas comes to my mind: hierarchy of interest and the role of overarching technologies. It is something I have never thought seriously about: when we face many alternative ideas for new technologies, almost intuitively we hierarchize them. Some of them seem more interesting, some others are less. I am trying to dig out of my own mind the criteria I use, and here they are: I hierarchize with the expected lifecycle of technology, and the breadth of the technological platform involved. In other words, I like big, solid, durable stuff. I am intuitively looking for innovations which offer a relatively long lifecycle in the corresponding technology, and the technology involved is sort of two-level, with a broad base and many specific applicational developments built upon that base.  

Why do I take this specific approach? One step further down into my mind, I discover the willingness to have some sort of broad base of business and scientific points of attachment when I start business planning. I want some kind of horizon to choose my exact target on. The common technological base among those 5 ideas is some kind of intelligent digital cloud, with artificial intelligence learns on the data that flows in. The common scientific base is the collection of health-related data, including behavioural aspects (e.g. sleep, diet, exercise, stress management).

The financial context which I am operating in is complex. It is made of public financial grants for strictly speaking scientific research, other public financing for projects more oriented on research and development in consortiums made of universities and business entities, still a different stream of financing for business entities alone, and finally private capital to look for once the technology is ripe enough for being marketed.

I am operating from an academic position. Intuitively, I guess that the more valuable science academic people bring to their common table with businesspeople and government people, the better position those academics will have in any future joint ventures. Hence, we should max out on useful, functional science to back those ideas. I am trying to understand what that science should consist in. An intelligent digital cloud can yield mind-blowing findings. I know that for a fact from my own research. Yet, what I know too is that I need very fundamental science, something at the frontier of logic, philosophy, mathematics, and of the phenomenology pertinent to the scientific research at hand, in order to understand and use meaningfully whatever the intelligent digital cloud spits back out, after being fed with data. I have already gone once through that process of understanding, as I have been working on the application of artificial neural networks to the simulation of collective intelligence in human societies. I had to coin up a theory of intelligent structure, applicable to the problem at hand. I believe that any application of intelligent digital cloud requires assuming that whatever we investigate with that cloud is an intelligent structure, i.e. a structure which learns by producing many alternative versions of itself, and testing them for their fitness to optimize a given desired outcome.  

With those medical ideas, I (we?) need to figure out what the intelligent structure in action is, how can it possibly produce many alternative versions of itself, and how those alternative thingies can be tested for fitness. What we have in a medically edged digital cloud is data about a population of people. The desired outcome we look for is health, quite simply. I said ‘simply’? No, it was a mistake. It is health, in all complexity. Those apps our students want to develop are supposed to pull someone out of the crowd, someone with early symptoms which they do not identify as relevant. In a next step, some kind of dialogue is proposed to such a person, sort of let’s dig a bit more into those symptoms, let’s try something simple to treat them etc. The vector of health in that population is made, roughly speaking, of three sub-vectors: preventive health (e.g. exercise, sleep, stop eating crap food), effectiveness of early medical intervention (e.g. c’mon men, if you are 30 and can’t have erection, you are bound to concoct some cardio-vascular s**t), and finally effectiveness of advanced medicine, applied when the former two haven’t worked.  

I can see at least one salient, scientific hurdle to jump over: that outcome vector of health. In my own research, I found out that artificial neural networks can give empirical evidence as for what outcomes we are really actually after, as collectively intelligent a structure. That’s my first big idea as regards those digital medical solutions: we collect medical and behavioural data in the cloud, we assume that data represents experimental learning of a collectively intelligent social structure, and we make the cloud discover the phenomena (variables) which the structure actually optimizes.

My own experience with that method is that societies which I studied optimize outcomes which look almost too simplistic in the fancy realm of social sciences, such as the average number of hours worked per person per year, the average amount of human capital per person, measured as years of education before entering the job market, or price index in exports, thus the average price which countries sell their exports at. In general, societies which I studied tend to optimize structural proportions, measurables as coefficients in the lines of ‘amount of thingy one divided by the amount of thingy two’.  

Checkpoint for business. Supposing that our research team, at the Andrzej Frycz – Modrzewski Krakow University, comes up with robust empirical results of that type, i.e. when we take a million of random humans and their broadly spoken health, and we assume they are collectively intelligent (I mean, beyond Facebook), then their collectively shared experimental learning of the stuff called ‘life’ makes them optimize health-related behavioural patterns A, B, and C. How can those findings be used in the form of marketable digital technologies? If I know the behavioural patterns someone tries to optimize, I can break those patterns down into small components and figure out a way to utilize the way to influence behaviour. It is a common technique in marketing. If I know someone’s lifestyle, and the values that come with it, I can artfully include into that pattern the technology I am marketing. In this specific case, it could be done ethically and for a good purpose, for a change.  In that context, my mind keeps returning to that barely marked trend of rising mortality in adult males in high-income countries, since 2016 (https://data.worldbank.org/indicator/SP.DYN.AMRT.MA). WTF? We’ll live, we’ll see.

The understanding of how collective human intelligence goes after health could be, therefore, the kind of scientific bacon our university could bring to the table when starting serious consortial projects with business partners, for the development of intelligent digital technologies in healthcare. Let’s move one step forward. As I have been using artificial neural network in my research on what I call, and maybe overstate as collective human intelligence, I have been running those experiments where I take a handful of behavioural patterns, I assign them probabilities of happening (sort of how many folks out of 10 000 will endorse those patterns), and I treat those probabilities as instrumental input in the optimization of pre-defined social outcomes. I was going to forget: I add random disturbance to that form of learning, in the lines of the Black Swan theory (Taleb 2007[1]; Taleb & Blyth 2011[2]).

I nailed down three patterns of collective learning in the presence of randomly happening s**t: recurrent, optimizing, and panic mode. The recurrent pattern of collective learning, which I tentatively expect to be the most powerful, is essentially a cycle with recurrent amplitude of error. We face a challenge, we go astray, we run around like headless chickens for a while, and then we figure s**t out, we progressively settle for solutions, and then the cycle repeats. It is like everlasting learning, without any clear endgame. The optimizing pattern is something I observed when making my collective intelligence optimize something like the headcount of population, or the GDP. There is a clear phase of ‘WTF!’(error in optimization goes haywire), which, passing through a somehow milder ‘WTH?’, ends up in a calm phase of ‘what works?’, with very little residual error.

The panic mode is different from the other two. There is no visible learning in the strict sense of the term, i.e. no visible narrowing down of error in what the network estimates as its desired outcome. On the contrary, that type of network consistently goes into the headless chicken mode, and it is becoming more and more headless with each consecutive hundred of experimental rounds, so to say. It happens when I make my network go after some very specific socio-economic outcomes, like price index in capital goods (i.e. fixed assets) or Total Factor Productivity.

Checkpoint for business, once again. That particular thing, about Black Swans randomly disturbing people in their endorsing of behavioural patterns, what business value does it have in a digital cloud? I suppose there are fields of applied medical sciences, for example epidemiology, or the management of healthcare systems, where it pays to know in advance which aspects of our health-related behaviour are the most prone to deep destabilization in the presence of exogenous stressors (e.g. epidemic, or the president of our country trending on Tik Tok). It could also pay off to know, which collectively pursued outcomes act as stabilizers. If another pandemic breaks out, for example, which social activities and social roles should keep going, at all price, on the one hand, and which ones can be safely shut down, as they will go haywire anyway?      


[1] Taleb, N. N. (2007). The black swan: The impact of the highly improbable (Vol. 2). Random house.

[2] Taleb, N. N., & Blyth, M. (2011). The black swan of Cairo: How suppressing volatility makes the world less predictable and more dangerous. Foreign Affairs, 33-39.

The collective archetype of striking good deals in exports

My editorial on You Tube

I keep philosophizing about the current situation, and I try to coin up a story in my mind, a story meaningful enough to carry me through the weeks and months to come. I try to figure out a strategy for future investment, and, in order to do that, I am doing that thing called ‘strategic assessment of the market’.

Now, seriously, I am profiting from that moment of forced reclusion (in Poland we have just had compulsory sheltering at home introduced, as law) to work a bit on my science, more specifically on the application of artificial neural networks to simulate collective intelligence in human societies. As I have been sending around draft papers on the topic, to various scientific journals (here you have a sample of what I wrote on the topic << click this link to retrieve a draft paper of mine), I have encountered something like a pretty uniform logic of constructive criticism. One of the main lines of reasoning in that logic goes like: ‘Man, it is interesting what you write. Yet, it would be equally interesting to explain what you mean exactly by collective intelligence. How does it or doesn’t it rhyme with individual intelligence? How does it connect with culture?’.

Good question, truly a good one. It is the question that I have been asking myself for months, since I discovered my fascination with the way that simple neural networks work. At the time, I observed intelligent behaviour in a set of four equations, put back to back in a looping sequence, and it was a ground-breaking experience for me. As I am trying to answer this question, my intuitive path is that of distinction between collective intelligence and the individual one. Once again (see The games we play with what has no brains at all ), I go back to William James’s ‘Essays in Radical Empiricism’, and to his take on the relation between reality and our mind. In Essay I, entitled ‘Does Consciousness Exist?’, he goes: “My thesis is that if we start with the supposition that there is only one primal stuff or material in the world, a stuff of which everything is composed, and if we call that stuff ‘pure experience,’ then knowing can easily be explained as a particular sort of relation towards one another into which portions of pure experience may enter. The relation itself is a part of pure experience; one of its ‘terms’ becomes the subject or bearer of the knowledge, the knower, the other becomes the object known. […] Just so, I maintain, does a given undivided portion of experience, taken in one context of associates, play the part of a knower, of a state of mind, of ‘consciousness’; while in a different context the same undivided bit of experience plays the part of a thing known, of an objective ‘content.’ In a word, in one group it figures as a thought, in another group as a thing. And, since it can figure in both groups simultaneously, we have every right to speak of it as subjective and objective both at once.”

Here it is, my distinction. Right, it is partly William James’s distinction. Anyway, individual intelligence is almost entirely mediated by conscious experience of reality, which is representation thereof, not reality as such. Individual intelligence is based on individual representation of reality. By opposition, my take on collective intelligence is based on the theory of adaptive walk in rugged landscape, a theory used both in evolutionary biology and in the programming of artificial intelligence. I define collective intelligence as the capacity to run constant experimentation across many social entities (persons, groups, cultures, technologies etc.), as regards the capacity of those entities to achieve a vector of desired social outcomes.

The expression ‘vector of desired social outcomes’ sounds as something invented by a philosopher and mathematician, together, after a strong intake of strong spirits. I am supposed to be simple in getting my ideas across, and thus I am translating that expression into something simpler. As individuals, we are after something. We have values that we pursue, and that pursuit helps us making it through each consecutive day. Now, there is a question: do we have collective values that we pursue as a society? Interesting question. Bernard Bosanquet, the British philosopher who wrote ‘The Philosophical Theory of The State[1], claimed very sharply that individual desires and values hardly translate into collective, state-wide values and goals to pursue. He claimed that entire societies are fundamentally unable to want anything, they can just be objectively after something. The collective being after something is essentially non-emotional and non-intentional. It is something like a collective archetype, occurring at the individual level somewhere below the level of consciousness, in the collective unconscious, which mediates between conscious individual intelligence and the external stuff of reality, to use William James’ expression.

How to figure out what outcomes are we after, as a society? This is precisely, for the time being, the central axis of my research involving neural networks. I take a set of empirical observations about a society, e.g. a set of country-year observation of 30 countries across 40 quantitative variables. Those empirical observations are the closest I can get to the stuff of reality. I make a simple neural network supposed to simulate the way a society works. The simpler this network is, the better. Each additional component of complexity requires making ever strengthening assumptions about the way societies works. I use that network as a simple robot. I tell the robot: ‘Take one variable from among those 40 in the source dataset. Make it your output variable, i.e. the desired outcome of collective existence. Treat the remaining 39 variables as input, instrumental to achieving that outcome’.  I make 40 such robots, and each of them produces a set of numbers, which is like a mutation of the original empirical dataset, and I can assess the similarity between each such mutation and the source empirical stuff. I do it by calculating the Euclidean distance between vectors of mean values, respectively in each such clone and the original data. Other methods can be used, e.g. kernel functions.

I worked that method through with various empirical datasets, and my preferred one, for now, is Penn Tables 9.1. (Feenstra et al. 2015[2]), which is a pretty comprehensive overview of macroeconomic variables across the planetary board. The detailed results of my research vary, depending on the exact set of variables I take into account, and on the set of observations I select, still there is a tentative conclusion that emerges: as a set of national societies, living in separate countries on that crazy piece of rock, speeding through cosmic space with no roof whatsoever, just with air condition on, we are mostly after terms of trade, and about the way we work, we prepare for work, and the way we remunerate work. Numerical robots which I program to optimize variables such as average price in exports, the share of labour compensation in Gross National Income, the average number of hours worked per year per person, or the number of years spent in education before starting professional activity: all these tend to win the race for similarity to the source empirical data. These seem to be the desired outcomes that our human collective intelligence seems to be after.

Is it of any help regarding the present tough s**t we are waist deep in? If my intuitions are true, whatever we will do regarding the COVID-19 pandemic, will be based on an evolutionary, adaptive choice. Path #1 consists in collectively optimizing those outcomes, whilst trying to deal with the pandemic, and dealing with the pandemic will be instrumental to, for example, the deals we strike in international trade, and to the average number of hours worked per person per year. An alternative Path #2 means to reshuffle our priorities completely and reorganize so as to pursue completely different goals. Which one are we going to take? Good question, very much about guessing rather than forecasting. Historical facts indicate that so far, as a civilization, we have been rather slow out of the gate. Change in collectively pursued values had occurred slowly, progressively, at the pace of generations rather than press conferences.  

In parallel to doing research on collective intelligence, I am working on a business plan for the project I named ‘Energy Ponds’ (see, for example: Bloody hard to make a strategy). I have done some market research down this specific avenue of my intellectual walk, and here below I am giving a raw account of progress therein.

The study of market environment for the Energy Ponds project is pegged on one central characteristic of the technology, which will be eventually developed: the amount of electricity possible to produce in the structure based on ram pumps and relatively small hydroelectric turbines. Will this amount be sufficient just to supply energy to a small neighbouring community or will it be enough to be sold in wholesale amounts via auctions and deals with grid operators. In other words, is Energy Ponds a viable concept just for the off-grid installations or is it scalable up to facility size?

There are examples of small hydropower installations, which connect to big power grids in order to exploit incidental price opportunities (Kusakana 2019[3]).

That basic question kept in mind, it is worth studying both the off-grid market for hydroelectricity, as well as the wholesale, on-grid market. Market research for Energy Ponds starts, in the first subsection below, with a general, global take on the geographical distribution of the main factors, both environmental and socio-economic. The next sections study characteristic types of markets

Overview of environmental and socio-economic factors 

Quantitative investigation starts with the identification of countries, where hydrological conditions are favourable to implementation of Energy Ponds, namely where significant water stress is accompanied by relatively abundant precipitations. More specifically, this stage of analysis comprises two steps. In the first place, countries with significant water stress are identified[4], and then each of them is checked as for the amount of precipitations[5], hence the amount of rainwater possible to collect.

Two remarks are worth formulating at this point. Firstly, in the case of big countries, such as China or United States, covering both swamps and deserts, the target locations for Energy Ponds would be rather regions than countries as a whole. Secondly, and maybe a bit counterintuitively, water stress is not a strict function of precipitations. When studied in 2014, with the above-referenced data from the World Bank, water stress is Pearson-correlated with precipitations just at r = -0,257817141.

Water stress and precipitations have very different distributions across the set of countries reported in the World Bank’s database. Water stress strongly varies across space, and displays a variability (i.e. quotient of its standard deviation divided by its mean value) of v = 3,36. Precipitations are distributed much more evenly, with a variability of v = 0,68. With that in mind, further categorization of countries as potential markets for the implementation of Energy Ponds has been conducted with the assumption that significant water stress is above the median value observed, thus above 14,306296%. As for precipitations, a cautious assumption, prone to subsequent revision, is that sufficient rainfall for sustaining a structure such as Energy Ponds is above the residual difference between mean rainfall observed and its standard deviation, thus above 366,38 mm per year.      

That first selection led to focusing further analysis on 40 countries, namely: Kenya, Haiti, Maldives, Mauritania, Portugal, Thailand, Greece, Denmark, Netherlands, Puerto Rico, Estonia, United States, France, Czech Republic, Mexico, Zimbabwe, Philippines, Mauritius, Turkey, Japan, China, Singapore, Lebanon, Sri Lanka, Cyprus, Poland, Bulgaria, Germany, South Africa, Dominican Republic, Kyrgyz Republic, Malta, India, Italy, Spain, Azerbaijan, Belgium, Korea, Rep., Armenia, Tajikistan.

Further investigation focused on describing those 40 countries from the standpoint of the essential benefits inherent to the concept of Energy Ponds: prevention of droughts and floods on the one hand, with the production of electricity being the other positive outcome. The variable published by the World Bank under the heading of ‘Droughts, floods, extreme temperatures (% of population, average 1990-2009)[6] has been taken individually, and interpolated with the headcount of population. In the first case, the relative importance of extreme weather phenomena for local populations is measured. When recalculated into the national headcount of people touched by extreme weather, this metric highlights the geographical distribution of the aggregate benefits, possibly derived from adaptive resilience vis a vis such events.

Below, both metrics, i.e. the percentage and the headcount of population, are shown as maps. The percentage of population touched by extreme weather conditions is much more evenly distributed than its absolute headcount. In general, Asian countries seem to absorb most of the adverse outcomes resulting from climate change. Outside Asia, and, of course, within the initially selected set of 40 countries, Kenya seems to be the most exposed.    


Another possible take on the socio-economic environment for developing Energy Ponds is the strictly business one. Prices of electricity, together with the sheer quantity of electricity consumed are the chief coordinates in this approach. Prices of electricity have been reported as retail prices for households, as Energy Ponds are very likely to be an off-grid local supplier. Sources of information used in this case are varied: EUROSTAT data has been used as regards prices in European countries[1] and they are generally relevant for 2019. For other countries sites such as STATISTA or www.globalpetrolprices.com have been used, and most of them are relevant for 2018. These prices are national averages across different types of contracts.

The size of electricity markets has been measured in two steps, starting with consumption of electricity per capita, as published by the World Bank[2], which has been multiplied by the headcount of population. Figures below give a graphical idea of the results. In general, there seems to be a trade-off between price and quantity, almost as in the classical demand function. The biggest markets of electricity, such as China or the United States, display relatively low prices. Markets with high prices are comparatively much smaller in terms of quantity. An interesting insight has been found, when prices of electricity have been compared with the percentage of population with access to electricity, as published by the World Bank[3]. Such a comparison, shown in Further below, we can see interesting outliers: Haiti, Kenya, India, and Zimbabwe. These are countries burdened with significant limitations as regards access to electricity. In these locations, projects such as Energy Ponds can possibly produce entirely new energy sources for local populations. 

The possible implementation of Energy Ponds can take place in very different socio-economic environments. It is worth studying those environments as idiosyncratic types. Further below, the following types and cases are studied more in detail:

  1. Type ‘Large cheap market with a lot of environmental outcomes’: China, India >> low price of electricity, locally access to electricity, prevention of droughts and floods,
  • Type ‘Small or medium-sized, developed European economy with high prices of electricity and relatively small a market’
  • Special case: United States ‘Large, moderately priced market, with moderate environmental outcomes’: United States >> moderate price of electricity, possibility to go off grid with Energy Ponds, prevention of droughts and floods 
  • Special case: Kenya > quite low access to electricity (63%) and moderately high retail price of electricity (0,22/ kWh), big population affected by droughts and floods, Energy Ponds can increase access to electricity

Table 1, further below, exemplifies the basic metrics of a hypothetical installation of Energy Ponds, in specific locations representative for the above-mentioned types and special cases. These metrics are:

  1. Discharge (of water) in m3 per second, in selected riverain locations. Each type among those above is illustrated with a few specific, actual geographical spots. The central assumption at this stage is that a local installation of Energy Ponds abstracts 20% of the flow per second in the river. Of course, should a given location be selected for more in-depth a study, specific hydrological conditions have to be taken into account, and the 20%-assumption might be verified upwards or downwards.
  • Electric power to expect with the given abstraction of water. That power has been calculated with the assumption that an average ram pump can create elevation, thus hydraulic head, of about 20 metres. There are more powerful ram pumps (see for example: https://www.allspeeds.co.uk/hydraulic-ram-pump/ ), yet 20 metres is a safely achievable head to assume without precise knowledge of environmental conditions in the given location. Given that 20-meter head, the basic equation to calculate electric power in watts is:
  • [Flow per second, in m3, calculated as 20% of abstraction from the local river]

x

20 [head in meters, by ram pumping]

x

9,81 [Newtonian acceleration]

x

75% [average efficiency of hydroelectric turbines]

  • Financial results to expect from the sales of electricity. Those results are calculated on the basis of two empirical variables: the retail price of electricity, referenced as mentioned earlier in this chapter, and the LCOE (Levelized Cost Of Energy). The latter is sourced from a report by the International Renewable Energy Agency (IRENA 2019[1]), and provisionally pegged at $0,05 per kWh. This is a global average and in this context it plays the role of simplifying assumption, which, in turn, allows direct comparison of various socio-economic contexts. Of course, each specific location for Energy Ponds bears a specific LCOE, in the phase of implementation. With those two source variables, two financial metrics are calculated:
    • Revenues from the sales of electricity, as: [Electric power in kilowatts] x [8760 hours in a year] x [Local retail price for households per 1 kWh]
    • Margin generated over the LCOE, equal to: [Electric power in kilowatts] x [8760 hours in a year] x {[Retail price for households per 1 kWh] – $0,05}

Table 1

Country Location (Flow per second, with 20% abstraction from the river)   Electric power generated with 20% of abstraction from the river (Energy for sale) Annual revenue (Annual margin over LCOE)  
China Near Xiamen,  Jiulong River (26 636,23 m3 /s)   783,9 kW (6 867 006,38 kWh a year)   $549 360,51 ($206 010,19)
China   Near Changde, Yangtze River (2400 m3/s)     353,16 kW (3 093 681,60 kWh a year)     $247 494,53 ($92 810,45
India   North of Rajahmundry, Godavari River (701 m3/s)   103,15 kW (903 612,83 kWh a year) $54 216,77 ($9 036,13) 
India   Ganges River near Patna (2400 m3/s)   353,16 kW (3 093 681,60 kWh a year) $185 620,90  ($30 936,82)
Portugal Near Lisbon, Tagus river (100 m3/s)   14,72 kW (128 903,40 kWh a year)   € 27 765,79 (€22 029,59)
Germany   Elbe river between Magdeburg and Dresden (174 m3/s)   25,6 kW (224 291,92 kWh a year) €68 252,03 (€58 271,04)
  Poland   Vistula between Krakow and Sandomierz (89,8 m3/s)     13,21 kW (115 755,25 kWh a year)   € 18 234,93 (€13 083,82)
France   Rhone river, south of Lyon (3400 m3/s)   500,31 kW   (4 382 715,60 kWh a year)  € 773 549,30  (€ 582 901,17)
United States, California   San Joaquin River (28,8 m3/s)   4,238 kW (37 124,18 kWh a year) $ 7 387,71 ($5 531,50)
United States, Texas   Colorado River, near Barton Creek (100 m3/s)   14,72 kW (128 903,40 kWh a year) $14 643,43 ($8 198,26)
United States, South Carolina   Tennessee River, near Florence (399 m3/s)   58,8 kW   (515 097,99 kWh a year)    $66 499,15  ($40 744,25)
Kenya   Nile River, by the Lake Victoria (400 m3/s)   58,86 kW (515 613,6 kWh a year)  $113 435  ($87 654,31)
Kenya Tana River, near Kibusu (81 m3/s)   11,92 kW (104 411,75 kWh a year)   $22 970,59 ($17 750)

China and India are grouped in the same category for two reasons. Firstly, because of the proportion between the size of markets for electricity, and the pricing thereof. These are huge markets in terms of quantity, yet very frugal in terms of price per 1 kWh. Secondly, these two countries seem to be representing the bulk of populations, globally observed as touched damage from droughts and floods. Should the implementation of Energy Ponds be successful in these countries, i.e. should water management significantly improve as a result, environmental benefits would play a significant socio-economic role.

With those similarities to keep in mind, China and India display significant differences as for both the environmental conditions, and the economic context. China hosts powerful rivers, with very high flow per second. This creates an opportunity, and a challenge. The amount of water possible to abstract from those rivers through ram pumping, and the corresponding electric power possible to generate are the opportunity. Yet, ram pumps, as they are manufactured now, are mostly small-scale equipment. Creating ram-pumping systems able to abstract significant amounts of water from Chinese rivers, in the Energy Ponds scheme, is a technological challenge in itself, which would require specific R&D work.

That said, China is already implementing a nation-wide programme of water management, called ‘Sponge Cities’, which shows some affinity to the Energy Ponds concept. Water management in relatively small, network-like structures, seems to have a favourable economic and political climate in China, and that climate translates into billions of dollars in investment capital.

India is different in these respects. Indian rivers, at least in floodplains, where Energy Ponds can be located, are relatively slow, in terms of flow per second, as compared to China. Whilst Energy Ponds are easier to implement technologically in such conditions, the corresponding amount of electricity is modest. India seems to be driven towards financing projects of water management as big dams, or as local preservation of wetlands. Nothing like the Chinese ‘Sponge Cities’ programme seems to be emerging, to the author’s best knowledge.

European countries form quite a homogenous class of possible locations for Energy Ponds. Retail prices of electricity for households are generally high, whilst the river system is dense and quite disparate in terms of flow per second. In the case of most European rivers, flow per second is low or moderate, still the biggest rivers, such as Rhine or Rhone, offer technological challenges similar to those in China, in terms of required volume in ram pumping.

As regards the Energy Ponds business concept, the United States seem to be a market on their own right. Local populations are exposed to moderate (although growing) an impact of droughts and floods, whilst they consume big amounts of electricity, both in aggregate, and per capita. Retail prices of electricity for households are noticeable disparate from state to state, although generally lower than those practiced in Europe[2]. Prices range from less than $0,1 per 1 kWh in Louisiana, Arkansas or Washington, up to $0,21 in Connecticut. It is to note that with respect to prices of electricity, the state of Hawaii stands out, with more than $0,3 per 1 kWh.

The United States offer quite a favourable environment for private investment in renewable sources of energy, still largely devoid of systematic public incentives. It is a market of multiple, different ecosystems, and all ranges of flow in local rivers.    


[1] IRENA (2019), Renewable Power Generation Costs in 2018, International Renewable Energy Agency, Abu Dhabi. ISBN 978-92-9260-126-3

[2] https://www.electricchoice.com/electricity-prices-by-state/ last access March 6th, 2020


[1] https://ec.europa.eu/eurostat/

[2] https://data.worldbank.org/indicator/EG.USE.ELEC.KH.PC

[3] https://data.worldbank.org/indicator/EG.ELC.ACCS.ZS


[1] Bosanquet, B. (1920). The philosophical theory of the state (Vol. 5). Macmillan and Company, limited.

[2] Feenstra, Robert C., Robert Inklaar and Marcel P. Timmer (2015), “The Next Generation of the Penn World Table” American Economic Review, 105(10), 3150-3182, available for download at http://www.ggdc.net/pwt

[3] Kusakana, K. (2019). Optimal electricity cost minimization of a grid-interactive Pumped Hydro Storage using ground water in a dynamic electricity pricing environment. Energy Reports, 5, 159-169.

[4] Level of water stress: freshwater withdrawal as a proportion of available freshwater resources >> https://data.worldbank.org/indicator/ER.H2O.FWST.ZS

[5] Average precipitation in depth (mm per year) >> https://data.worldbank.org/indicator/AG.LND.PRCP.MM

[6] https://data.worldbank.org/indicator/EN.CLC.MDAT.ZS

When is it the right moment to expose ourselves?

My editorial

And so I know what I want. I mean, I sort of know what I want regarding some aspects of my life. That business I want to put together in connection to smart cities is a centre of experimental research focused on studying human behaviour in the interaction with smart technologies. The business idea behind is to supply experimental knowledge, and the tools for discovering new experimental knowledge, to businesses which would like to accelerate their innovation and to smoothen their relations with big institutional customers. The metric of utility I have come up with is the possible reduction of payment lag in those customer relations. This is what I have come up with so far, and you can find it in my last update in French: Ça semble tenir le coup. Ça promet .

The business concept I am entertaining for this experimental centre is based on sort of an educated guess, which I have formulated, over years, as I have been studying different business cases: when something lasts longer than it technically should last, someone in that process is learning something by trial and error. When an IT company, like Asseco Polska, whose financials I highlighted in that last update in French, used to receive payment for its services after 80 days, on average, in 2011, and this time extended to 101 days in 2016, the process of interaction with the average customer must have changed. Of course, some of that extension is due to changes in the financial liquidity in customers themselves. Still, as I try to reconstruct the interaction between an IT provider and an institutional customer, there is that component of tailoring the solutions supplied to the exact needs and characteristics of the latter. The supplier learns by doing, and that learning frequently manifests itself as an additional lag in payment: the customer withholds a part or the total of the sum due to the supplier until the latter removes from the solution provided what the customer sees as flawed.

This phase of a contract is quite tricky. It is crucial to separate true necessity of improvement in the solutions provided, on the one hand, from tactical withholding of payment, on the other hand. Said necessity of improvement seems to follow a common pattern, well-known from the management literature: engineers think in terms of blueprint (coding in the case of programmers), whilst the end-users of a technology think in terms of the most basic utility. That basic utility has to include fool-proof safeguards: if clicking the wrong link on the screen blocks an entire accounting system, there is something wrong.

You would say ‘No, these things don’t happen anymore. Digital solutions are past that phase’. Well, recently, I attended a presentation, by a fellow scientist, about how he and his team had improved some utilities in the SAP system. For the mildly initiated: SAP Business One, marketed by SAP SE, Germany, is one of the most advanced ERP-class systems in the world, maybe even the most advanced one. So, those fellow scientists from Krakow (Poland) did a simple thing: they took some parts of the SAP interface and they submitted it to behavioural testing, where users were being observed with an eye-tracking device. After the tests, the interface has been modified so as to optimize its functionality for the eye-movements of the users. Result: a series of operations which took the users 6 minutes before the modification, dropped to 1,5 minutes. Yes, this is a fourfold rise in productivity, and it was just one tiny piece of the total functionality available in SAP Business One.

My home university, the Andrzej Frycz Modrzewski Krakow University (Krakow, Poland), prides itself to rely on 100% their own technology. Library, grading, syllabuses, e learning platform: all that is genuinely made for the university by an appointed team of IT engineers, without any Google-made prosthesis. Still, man, sometimes you wish it was Google-powered somehow. It takes years to optimize each piece of integrated software so as to make it really functional. A lot of this optimizing relies on communication, which, of course, is what communication usually is: imperfect. I imagine the same thing happens in hundreds of cases, when an integrated technology (it does not have to be exclusively digital, you know) is being implemented in a large social structure. It is only after starting the implementation that engineers face the real users’ behaviour, and this is when the real tailoring starts.

Summing it up, partly, here is my idea: providers of complex technologies, when they have a big institutional client, go through that tricky phase in the contract, when the technology is basically in place, the invoice has been issued on the buyer, but the latter is still no satisfied and withholds the payment, and his dissatisfaction is well-founded, as the true utility of the technology installed is only taking shape, and it brings some surprises. The experimental centre I want to create would run experiments useful in minimizing the trial-and-error component in such situations. I would like to create a body of experimental knowledge about human behaviour in interaction with smart technologies, and experimental tools for developing more of such knowledge, so as to shorten the payment lag that the suppliers of smart technologies experience in their relations with big institutional customers.

Now, I am advancing a behavioural hypothesis: any large social structure, when absorbing a new technology, behaves like a mean institutional customer, and it pays the full bill only after it is fully satisfied with said technology. For those of you who want solid theory, here comes a piece: Robertson, T. S. (1967). The process of innovation and the diffusion of innovation. The Journal of Marketing, 14-19. The idea is the following: when you are a supplier of advanced technologies and you are selling your solutions on kind of a retail basis, to many small customers, they all make sort of a community in the sense that their behaviour regarding your technology is coordinated through socially recurrent, partly ritualized patterns. In that community, you will find the early innovators, the early adopters, the sceptical late adopters, and the fence-sitters. You can fully cash your returns on investment only when all those customers have absorbed your technology. If you are launching your technology in a form adapted to the tastes of the early innovators, it is going to bump against the wall of different tastes observable in more conservative customers. You need to modify your technology so as to make it easier to swallow conservatively. Modifications take time, and you need much of that time just in order to figure out the right modifications to make.

I think I can start generalising those ideas in the form of a model, regarding the interaction between the suppliers of technologies and their customers. Thus, the mission of my experimental centre will be to reduce the cost of innovation, or CI, incurred by the suppliers of smart technologies for smart cities. The cost of innovation is made of two component costs, namely: the cost CIN of ongoing invention in the strictly spoken new technologies (i.e. those not marketed yet), and the cost CIM of improvement/modification in the already marketed technologies. Both CIN and CIM contain two further parts: the cost of engineering work strictly spoken (blueprinting the technology), which I am going to symbolize with ‘B’ in subscript, and the cost of discovering behavioural patterns in the end-users of the technology in question, and I symbolize this one with ‘D’ in subscript.

Right, I am an economist, and so I need writing some formulae, so here is what I am thinking in mathematical terms:

[CI = CIN + CIM = CIN;B + CIN;D + CIM;B + CIM;D] =>  [CI = CIB + CID]

The last part of that complex logical implication, or ‘=>  [CI = CIB + CID]’ means that with the assumptions given above, the total cost of innovation is, in other words, the sum of what we spend on the strictly spoken blueprinting (CIB), on the one hand, and the expense on patterning the users’ behaviour (CID).

The mission of my experimental centre will be to shave off the CID. Next question: how the hell do I want to achieve that? Next piece of theory coming, then. The process of innovation, i.e. the process of blueprinting a technology, and optimizing it as user-friendly, can be represented as a series of interactions involving a prototyped technology, and a user testing it. Each prototype is an educated guess about the optimal utility for the users. Each interaction of a prototype with the users who are testing it can be a success (correct guess) or a failure (go and figure it out again, anew). In the process of innovation, we have n interactions, with p successes and q = n – p failures. The total CID cost comprises both the expenses connected to successful trials, and to the failed ones. Now, I start sailing uncharted waters, at least uncharted for me, as I am asking myself: how can I reduce the total cost of that process and/or increase its efficiency?

In my previous update in English, the one entitled ‘And so I ventured myself into the realm of what people think they can do’, I started developing on that idea that p successes and q = n – p failures can happen as many distinct sequences, technically equiprobable, but actually very different in their outcomes. My most personal intuition, which comes from my academic teaching, my sport practice, and my experience with the Wim Hof’s method of breathing, all that tells me that the sequence which works the best for learning is the following: ‘serious ass-kicking, technically a failure >>> small success >>> small success … (a whole series of small successes) >>> another ass-kicking (another failure) >>> and I loop on a series of small successes’ etc. My nervous system learns new skills through a sequence of successful attempts, not through a sequence of failures. Failures just teach me what not to do, but I need those small successes in order learn what to do, or form the right pattern of doing things. Still, failures keep me sharp and alert, they prevent me from becoming too self-enclosed, or, I should rather say, failures can keep me in that blessed frame of mind under the condition of being properly taken in.

Applying this general wisdom to experimenting with new technologies implies that I can calibrate successes and failures into different sizes, sort of “big success <> small success”, “big failure <> small failure”.  I imagine an experimental sequence, where engineers start with confronting an advanced prototype with testing from the part of users, and I arrange this particular step so as to make failure as probable and as deep as possible. I select the testing users, and I design the conditions of testing so as to put every possible hurdle and trap on this path. I want the engineers to bounce against something they are very likely to perceive as a brick wall. This is the initial ass-kicking. The next step consists in studying that failure very minutely, so as to own it completely. This phase of the experimental sequence brings two major learnings. Firstly, it shows what happens to our technology in a really unfriendly environment. This is the equivalent of those IKEA armchairs being crushed by a mechanical press thousands of times a day. Secondly, it teaches the engineers how they behave, as human beings, in the presence of a major failure. Behavioural study of the engineers is as important at this stage as that of the users.

In Stage 2, engineers modify the prototype on the grounds of the ass-kicking received in Stage 1. This time, it is important to break down the sequence ‘modification >> testing’ in very small steps, so as to make successful guessing of the users’ needs as probable as possible. In this stage, engineers learn to interact with users on a current basis, and, hopefully, create a series of successful – i.e. fully functional from the users’ point of view – modifications.

Now, the big question is: how to alternate Stage 1 and Stage 2. In other words, when is it the right moment to expose ourselves to the functional outcomes of every possible flaw in our technology? I have to ruminate it a bit.

I am consistently delivering good, almost new science to my readers, and love doing it, and I am working on crowdfunding this activity of mine. You can consider going to my Patreon page and become my patron. If you decide so, I will be grateful for suggesting me two things that Patreon suggests me to suggest you. Firstly, what kind of reward would you expect in exchange of supporting me? Secondly, what kind of phases would you like to see in the development of my research, and of the corresponding educational tools?