Cultural classes

Some of my readers asked me to explain how to get in control of one’s own emotions when starting their adventure as small investors in the stock market. The purely psychological side of self-control is something I leave to people smarter than me in that respect. What I do to have more control is the Wim Hof method (https://www.wimhofmethod.com/ ) and it works. You are welcome to try. I described my experience in that matter in the update titled ‘Something even more basic’. Still, there is another thing, namely, to start with a strategy of investment clever enough to allow emotional self-control. The strongest emotion I have been experiencing on my otherwise quite successful path of investment is the fear of loss. Yes, there are occasional bubbles of greed, but they are more like childish expectations to get the biggest toy in the neighbourhood. They are bubbles, which burst quickly and inconsequentially. The fear of loss is there to stay, on the other hand.

This is what I advise to do. I mean this is what I didn’t do at the very beginning, and fault of doing it I made some big mistakes in my decisions. Only after some time (around 2 months), I figured out the mental framework I am going to present. Start by picking up a market. I started with a dual portfolio, like 50% in the Polish stock market, and 50% in the big foreign ones, such as US, Germany, France etc. Define the industries you want to invest in, like biotech, IT, renewable energies. Whatever: pick something. Study the stock prices in those industries. Pay particular attention to the observed losses, i.e., the observed magnitude of depreciation in those stocks. Figure out the average possible loss, and the maximum one. Now, you have an idea of how much you can lose in percentage. Quantitative techniques such as mean-reversion or extrapolation of the past changes can help. You can consult my update titled ‘What is my take on these four: Bitcoin, Ethereum, Steem, and Golem?’ to see the general drift.

The next step is to accept the occurrence of losses. You need to acknowledge very openly the following: you will lose money on some of your investment positions, inevitably. This is why you build a portfolio of many investment positions. All investors lose money on parts of their portfolio. The trick is to balance losses with even greater gains. You will be experimenting, and some of those experiments will be successful, whilst others will be failures. When you learn investment, you fail a lot. The losses you incur when learning, are the cost of your learning.

My price of learning was around €600, and then I bounced back and compensated it with a large surplus. If I take those €600 and compare it to the cost of taking an investment course online, e.g. with Coursera, I think I made a good deal.

Never invest all your money in the stock market. My method is to take some 30% of my monthly income and invest it, month after month, patiently and rhythmically, by instalments. For you, it can be 10% or 50%, which depends on what exactly your personal budget looks like. Invest just the amount you feel you can afford exposing to losses. Nail down this amount honestly. My experience is that big gains in the stock market are always the outcome of many consecutive steps, with experimentation and the cumulative learning derived therefrom.

General remark: you are much calmer when you know what you’re doing. Look at the fundamental trends and factors. Look beyond stock prices. Try to understand what is happening in the real business you are buying and selling the stock of. That gives perspective and allows more rational decisions.

That would be it, as regards investment. You are welcome to ask questions. Now, I shift my topic radically. I return to the painful and laborious process of writing my book about collective intelligence. I feel like shaking things off a bit. I feel I need a kick in the ass. The pandemic being around and little social contacts being around, I need to be the one who kicks my own ass.

I am running myself through a series of typical questions asked by a publisher. Those questions fall in two broad categories: interest for me, as compared to interest for readers. I start with the external point of view: why should anyone bother to read what I am going to write? I guess that I will have two groups of readers: social scientists on the one hand, and plain folks on the other hand. The latter might very well have a deeper insight than the former, only the former like being addressed with reverence. I know something about it: I am a scientist.

Now comes the harsh truth: I don’t know why other people should bother about my writing. Honestly. I don’t know. I have been sort of carried away and in the stream of my own blogging and research, and that question comes as alien to the line of logic I have been developing for months. I need to look at my own writing and thinking from outside, so as to adopt something like a fake observer’s perspective. I have to ask myself what is really interesting in my writing.

I think it is going to be a case of assembling a coherent whole out of sparse pieces. I guess I can enumerate, once again, the main points of interest I find in my research on collective intelligence and investigate whether at all and under what conditions the same points are likely to be interesting for other people.

Here I go. There are two, sort of primary and foundational points. For one, I started my whole research on collective intelligence when I experienced the neophyte’s fascination with Artificial Intelligence, i.e. when I discovered that some specific sequences of equations can really figure stuff out just by experimenting with themselves. I did both some review of literature, and some empirical testing of my own, and I discovered that artificial neural networks can be and are used as more advanced counterparts to classical quantitative models. In social sciences, quantitative models are about the things that human societies do. If an artificial form of intelligence can be representative for what happens in societies, I can hypothesise that said societies are forms of intelligence, too, just collective forms.

I am trying to remember what triggered in me that ‘Aha!’ moment, when I started seriously hypothesising about collective intelligence. I think it was when I was casually listening to an online lecture on AI, streamed from the Massachusetts Institute of Technology. It was about programming AI in robots, in order to make them able to learn. I remember one ‘Aha!’ sentence: ‘With a given set of empirical data supplied for training, robots become more proficient at completing some specific tasks rather than others’. At the time, I was working on an article for the journal ‘Energy’. I was struggling. I had an empirical dataset on energy efficiency in selected countries (i.e. on the average amount of real output per unit of energy consumption), combined with some other variables. After weeks and weeks of data mining, I had a gut feeling that some important meaning is hidden in that data, only I wasn’t able to put my finger precisely on it.

That MIT-coined sentence on robots triggered that crazy question in me. What if I return to the old and apparently obsolete claim of the utilitarian school in social sciences, and assume that all those societies I have empirical data about are something like one big organism, with different variables being just different measurable manifestations of its activity?

Why was that question crazy? Utilitarianism is always contentious, as it is frequently used to claim that small local injustice can be justified by bringing a greater common good for the whole society. Many scholars have advocated for that claim, and probably even more of them have advocated against. I am essentially against. Injustice is injustice, whatever greater good you bring about to justify it. Besides, being born and raised in a communist country, I am viscerally vigilant to people who wield the argument of ‘greater good’.

Yet, the fundamental assumptions of utilitarianism can be used under a different angle. Social systems are essentially collective, and energy systems in a society are just as collective. There is any point at all in talking about the energy efficiency of a society when we are talking about the entire intricate system of using energy. About 30% of the energy that we use is used in transport, and transport is from one person to another. Stands to reason, doesn’t it?

Studying my dataset as a complex manifestation of activity in a big complex organism begs for the basic question: what do organisms do, like in their daily life? They adapt, I thought. They constantly adjust to their environment. I mean, they do if they want to survive. If I settle for studying my dataset as informative about a complex social organism, what does this organism adapt to? It could be adapting to a gazillion of factors, including some invisible cosmic radiation (the visible one is called ‘sunlight’). Still, keeping in mind that sentence about robots, adaptation can be considered as actual optimization of some specific traits. In my dataset, I have a range of variables. Each variable can be hypothetically considered as informative about a task, which the collective social robot strives to excel at.

From there, it was relatively simple. At the time (some 16 months ago), I was already familiar with the logical structure of a perceptron, i.e. a very basic form of artificial neural network. I didn’t know – and I still don’t – how to program effectively the algorithm of a perceptron, but I knew how to make a perceptron in Excel. In a perceptron, I take one variable from my dataset as output, the remaining ones are instrumental as input, and I make my perceptron minimize the error on estimating the output. With that simple strategy in mind, I can make as many alternative perceptrons out of my dataset as I have variables in the latter, and it was exactly what I did with my data on energy efficiency. Out of sheer curiosity, I wanted to check how similar were the datasets transformed by the perceptron to the source empirical data. I computed Euclidean distances between the vectors of expected mean values, in all the datasets I had. I expected something foggy and pretty random, and once again, life went against my expectations. What I found was a clear pattern. The perceptron pegged on optimizing the coefficient of fixed capital assets per one domestic patent application was much more similar to the source dataset than any other transformation.

In other words, I created an intelligent computation, and I made it optimize different variables in my dataset, and it turned out that, when optimizing that specific variable, i.e. the coefficient of fixed capital assets per one domestic patent application, that computation was the most fidel representation of the real empirical data.

This is when I started wrapping my mind around the idea that artificial neural networks can be more than just tools for optimizing quantitative models; they can be simulators of social reality. If that intuition of mine is true, societies can be studied as forms of intelligence, and, as they are, precisely, societies, we are talking about collective intelligence.

Much to my surprise, I am discovering similar a perspective in Steven Pinker’s book ‘How The Mind Works’ (W. W. Norton & Company, New York London, Copyright 1997 by Steven Pinker, ISBN 0-393-04535-8). Professor Steven Pinker uses a perceptron as a representation of human mind, and it seems to be a bloody accurate representation.

That makes me come back to the interest that readers could have in my book about collective intelligence, and I cannot help referring to still another book of another author: Nassim Nicholas Taleb’s ‘The black swan. The impact of the highly improbable’ (2010, Penguin Books, ISBN 9780812973815). Speaking from an abundant experience of quantitative assessment of risk, Nassim Taleb criticizes most quantitative models used in finance and economics as pretty much useless in making reliable predictions. Those quantitative models are good solvers, and they are good at capturing correlations, but they suck are predicting things, based on those correlations, he says.

My experience of investment in the stock market tells me that those mid-term waves of stock prices, which I so much like riding, are the product of dissonance rather than correlation. When a specific industry or a specific company suddenly starts behaving in an unexpected way, e.g. in the context of the pandemic, investors really pay attention. Correlations are boring. In the stock market, you make good money when you spot a Black Swan, not another white one. Here comes a nuance. I think that black swans happen unexpectedly from the point of view of quantitative predictions, yet they don’t come out of nowhere. There is always a process that leads to the emergence of a Black Swan. The trick is to spot it in time.

F**k, I need to focus. The interest of my book for the readers. Right. I think I can use the concept of collective intelligence as a pretext to discuss the logic of using quantitative models in social sciences in general. More specifically, I want to study the relation between correlations and orientations. I am going to use an example in order to make my point a bit more explicit, hopefully. In my preceding update, titled ‘Cool discovery’, I did my best, using my neophytic and modest skills in programming, the method of negotiation proposed in Chris Voss’s book ‘Never Split the Difference’ into a Python algorithm. Surprisingly for myself, I found two alternative ways of doing it: as a loop, on the one hand, and as a class, on the other hand. They differ greatly.

Now, I simulate a situation when all social life is a collection of negotiations between people who try to settle, over and over again, contentious issues arising from us being human and together. I assume that we are a collective intelligence of people who learn by negotiated interactions, i.e. by civilized management of conflictual issues. We form social games, and each game involves negotiations. It can be represented as a lot of these >>

… and a lot of those >>

In other words, we collectively negotiate by creating cultural classes – logical structures connecting names to facts – and inside those classes we ritualise looping behaviours.

Cool discovery

Writing about me learning something helps me to control emotions involved into the very process of learning. It is like learning on the top of learning. I want to practice programming, in Python, the learning process of an intelligent structure on the basis of negotiation techniques presented in Chris Voss’s book ‘Never Split the Difference’. It could be hard to translate a book into an algorithm, I know. I like hard stuff, and I am having a go at something even harder: translating two different books into one algorithm. A summary, and an explanation, are due. Chris Voss develops, in the last chapter of his book, a strategy of negotiation based on the concept of Black Swan, as defined by Nassim Nicholas Taleb in his book ‘The black swan. The impact of the highly improbable’ (I am talking about the revised edition from 2010, published with Penguin Books, ISBN 9780812973815).

Generally, Chriss Voss takes a very practical drift in his method of negotiation. By ‘practical’, I mean that he presents techniques which he developed and tested in hostage negotiations with FBI, where he used to be the chief international hostage negotiator. He seems to attach particular importance to all the techniques which allow unearthing the non-obvious in negotiations: hidden emotions, ethical values, and contextual factors with strong impact on the actual negotiation. His method is an unusual mix of rigorous cognitive approach with a very emotion-targeting thread. His reference to Black Swans, thus to what we don’t know we don’t know, is an extreme version of that approach. It consists in using literally all our cognitive tools to uncover events and factors in the game which we even didn’t initially know were in the game.

Translating a book into an algorithm, especially for a newbie of programming such as I am, is hard. Still, in the case of ‘Never Split the Difference’, it is a bit easier because of the very game-theoretic nature of the method presented. Chriss Voss attaches a lot of importance to taking our time in negotiations, and to making our counterpart make a move rather than overwhelming them with our moves. All that is close to my own perspective and makes the method easier to translate into a functional sequence where each consecutive phase depends on the preceding phase.

Anyway, I assume that a negotiation is an intelligent structure, i.e. it is an otherwise coherent and relatively durable structure which learns by experimenting with many alternative versions of itself. That implies a lot. Firstly, it implies that the interaction between negotiating parties is far from being casual and accidental: it is a structure, it has coherence, and it is supposed to last by recurrence. Secondly, negotiations are supposed to be learning much more than bargaining and confrontation. Yes, it is a confrontation of interests and viewpoints, nevertheless the endgame is learning. Thirdly, an intelligent structure experiments with many alternative versions of itself and learns by assessing the fitness of those versions in coping with a vector of external stressors. Therefore, negotiating in an intelligent structure means that, consciously or unconsciously, we, mutual counterparts in negotiation, experiment together with many alternative ways of settling our differences, and we are essentially constructive in that process.

Do those assumptions hold? I guess I can somehow verify them by making first steps into programming a negotiation. I already know two ways of representing an intelligent structure as an algorithm: in the form of a loop (primitive, tried it, does not fully work, yet has some interesting basic properties), or in the form of a class, i.e. a complex logical structure which connects names to numbers.

When represented as a loop, a negotiation is a range of recurrent steps, where the same action is performed a given number of times. Looping means that a negotiation can be divided into a finite number of essentially identical steps, and the endgame is the cumulative output of those steps. With that in mind, I can see that a loop is not truly intelligent a structure. Intelligent learning requires more than just repetition: we need consistent assessment and dissemination of new knowledge. Mind you, many negotiations can play out as ritualized loops, and this is when they are the least productive. Under the condition of unearthing Black Swans hidden in the contentious context of the negotiation, the whole thing can play out as an intelligent structure. Still, many loop-like negotiations which recurrently happen in a social structure, can together form an intelligent structure. Looks like intelligent structures are fractal: there are intelligent structures inside intelligent structures etc. Intelligent social structures can contain chains of ritualized, looped negotiations, which are intelligent structures in themselves.

Whatever. I program. When I try to sift out the essential phenomenological categories out of the Chris Voss’s book ‘Never Split the Difference’, I get to the following list of techniques recommended by Chriss Voss:

>> Mirroring – I build emotional rapport by just repeating the last three words of each big claim phrased out by my counterpart.

>> Labelling – I further build emotional rapport by carefully and impersonally naming emotions and aspirations in my counterpart.

>> Open-ended questions – I clarify claims and disarm emotional bottlenecks by asking calibrated open questions such as ‘How can we do X,Y, Z?’ or ‘What do we mean by…?’ etc.

>> Claims – I state either what I want or what I want my counterpart to think I want

Those four techniques can be used in various shades and combinations to accomplish typical partial outcomes in negotiation, namely: a) opportunities for your counterpart to say openly ‘No’ b) agreement in principle c) guarantee of implementation d) Black Swans, i.e. unexpected attributes of the situation which turn the negotiation in a completely different, favourable direction.

I practice phrasing it out as a class in Python. Here is what I came up with and which my JupyterLab compiler swallows nicely without yielding any errors:

Mind you, I don’t know how exactly it works, algorithmically. I am a complete newbie to programming classes in Python, and my first goal is to have the grammar right, and thus not to have to deal with those annoying, salmon-pink-shaded messages of error.

Before I go further into programming negotiation as a class, I feel like I need to go back to my primitive skills, i.e. to programming loops, in order to understand the mechanics of the class I have just created. Each ‘self’ in the class is a category able to have many experimental versions of itself. I try the following structure:

As you can see, I received an error of non-definition. I have not defined the dataset which I want to use for appending my lists. Such a dataset would contain linguistic strings, essentially. Thus, the type of datasets I am operating with, here, are sets of linguistic strings, thus sets of objects. An intelligent structure representative for negotiation is an algorithm for processing natural language. Cool discovery.

I got it all wrong

I like doing research on and writing about collective intelligence in human societies. I am even learning to program in Python in order to know how to code collective intelligence in the form of an artificial neural network. I decided to take on my own intelligence as an interesting diversion from the main course. I hope I can assume I am an intelligent structure. Whilst staying essentially coherent, i.e. whilst remembering who I am, I can experiment a bit with many different versions of myself. Of course, a substantial part of the existential Me works like a shuttle train, going back and forth on the rails of my habits. Still, I can learn heuristically on my own experience. Heuristic learning means that as I learn something, I gain new knowledge about how much more I can learn about and along the lines of the same something.

I want to put into a Python code the experience of heuristic, existential learning which I exemplified in the update titled ‘Something even more basic’. It starts with experience which happens through intentional action from my part. I define a vector of action, i.e. a vector of behavioural patterns, associated with the percentage of my total time they take. That percentage can be understood, alternatively, as the probability that any given minute in the day is devoted to that specific action. Some of those patterns are active, and some are dormant, with the possibility of being triggered into existence. Anyway, it is something like A = {a₁, a₂, …, a_n}. Now, in terms of coding in Python, is that vector of action a NumPy array, or is it a Pandas data frame? In terms of pure algorithmic technique, it is a trade-off between computational speed, with a NumPy array, and programmatic versatility in the case of a Pandas data frame. Here are a few points of view expressed, as regards this specific matter, by people smarter than me:

>> https://www.geeksforgeeks.org/difference-between-pandas-vs-numpy/

>> https://towardsdatascience.com/performance-of-numpy-and-pandas-comparison-7b3e0bea69bb

>> https://vitalflux.com/pandas-dataframe-vs-numpy-array-what-to-use/

In terms of algorithmic theory, these are two different, cognitive phenomena. A NumPy array is a structured collection of numbers, whilst a Pandas data frame is a structure combining many types of data, e.g. string objects with numbers. How does it translate into my own experience? I think that, essentially, my action is a data frame. I take purposeful action to learn something when I have a logical frame to put it in, i.e. when I have words to label what I do. That leads me to starting at even more elementary a level, namely that of a dictionary as regards my actions.

Anyway, I create a notebook with JupyterLab, and I start like a hamster, with stuffing my cheeks with libraries:

>> import numpy as np

>> import pandas as pd

>> import os

>> import math

Then, I make a first dictionary:

>> Types_of_action=[‘Action 1′,’Action 2′,’Action 3′,’Action 4′,’Action 5’]

A part of my brain says, at this point: ‘Wait a minute, bro. Before you put labels on the things that you do, you need to be doing things. Humans label stuff that happens, essentially. Yes, of course, later on, me can make them metaphors and abstract concepts but, fundamentally, descriptive language comes after experience’. Well, dear part of my brain, this is a valid argument. Things unfold into a paradox, just as I like it. I need raw experience, primal to any logical structuring. How to express it in Python? I can go like:

>> Raw_experience=np.random.rand(np.random.randint(1)) #This is a NumPy array made of random decimal values, and the number of those values in the array is random as well.

I check. I type ‘Raw_experience’ and run it. Python answers:

>> array([], dtype=float64) # I have just made a paradox: a totally empty array of numbers, i.e. with no numbers in it, and yet those inexistent numbers have a type, namely that of ‘float64’.

I try something less raw and more cooked, like:

>> Raw_experience_50=np.random.rand(50) # I assume a priori there are 50 distinct units of raw experience

>> Raw_experience_50 # yields…

>> array([0.73209089, 0.94390333, 0.44267215, 0.92111994, 0.4098961 ,

0.22435079, 0.61447481, 0.21183481, 0.10223352, 0.04481922,

0.01418667, 0.65747087, 0.22180559, 0.6158434 , 0.82275393,

0.22446375, 0.31331992, 0.64459349, 0.90762324, 0.65626915,

0.41462473, 0.35278516, 0.13978946, 0.79563848, 0.41794509,

0.12931173, 0.37012284, 0.37117378, 0.30989358, 0.26912215,

0.7404481 , 0.61690128, 0.41023962, 0.9405769 , 0.86930885,

0.84279381, 0.91174751, 0.04715724, 0.35663278, 0.75116884,

0.78188546, 0.30712707, 0.00615981, 0.93404037, 0.82483854,

0.99342718, 0.74814767, 0.49888401, 0.93164796, 0.87413073])

This is a short lesson of empiricism. When I try to code raw, completely unstructured experience, I obtain an empty expanse. I return to that interesting conversation with a part of my brain. Dear part of my brain, you were right to point out that experience comes before language, and yet, without language, i.e. without any logical structuring of reality, I don’t know s**t about experience, and I cannot intelligibly describe it. I need to go for a compromise. I make that experience as raw as possible by making it happen at random, and, in the same time, I need to give it some frame, like the number of times those random things are supposed to happen to me.

I defined a dictionary with 5 types of action in it. Thus, I define a random path of happening as an array made of 5 categories (columns), and 50 rows of experience: Raw_experience_for_action=np.random.rand(50,5).

I acknowledge the cognitive loop I am in, made of raw experience and requiring some language to put order in all that stuff. I make a data frame:

>> Frame_of_action = pd.DataFrame (Raw_experience_for_action, columns = [Types_of_action]) # One remark is due, just in case. In the Python code, normally, there are no spaces. I put spaces, somehow in phase with interpunction, just to make some commands more readable.

I check with ‘Frame_of_action.info()’ and I get:

>> <class 'pandas.core.frame.DataFrame'>

RangeIndex: 50 entries, 0 to 49

Data columns (total 5 columns):

 #   Column       Non-Null Count  Dtype

---  ------       --------------  -----

 0   (Action 1,)  50 non-null     float64

 1   (Action 2,)  50 non-null     float64

 2   (Action 3,)  50 non-null     float64

 3   (Action 4,)  50 non-null     float64

 4   (Action 5,)  50 non-null     float64

dtypes: float64(5)

memory usage: 2.1 KB

Once I have that basic frame of action, what is my next step? I need to learn from that experience. The frame of action is supposed to give me knowledge. What is knowledge coming from action? That type of knowledge is called ‘outcomes’. My action brings an outcome, and I evaluate it. Now, in a baseline algorithm of artificial neural network, evaluation of outcomes happens by pitching them against a predefined benchmark, something like expected outcome. As I am doing my best to be an intelligent structure, there is that aspect too, of course. Yet, there is something else, which I want to deconstruct, understand, and reconstruct as Python code. There is discovery and exploration, thus something that I perceive as entirely new a piece of experience. I don’t have any benchmark I can consciously pitch that experience against.

I can perceive my fresh experiential knowledge in two different ways: as a new piece of data, or as an error, i.e. as deviation from the expected state of reality. Both mathematically, and algorithmically, it is a difference. Mathematically, any number, thus any piece of data, is the result of an operation. If I note down, in the algorithm of my heuristic learning, my new knowledge as literally new, anyway it needs to come from some kind of mathematical operation: addition, subtraction, multiplication, or division.

As I think about myself learning new stuff, there is a phase, in the beginning, when I have some outcomes, and yet I don’t have any precise idea what those outcomes are, exactly. This is something that happens in coincidence (I don’t even know, yet, if this is a functional correlation) with the actions I do.

As I think about all that stuff, I try to make a loop of learning between action and outcomes, and as I am doing it, I realize I got it all wrong. For the last few weeks, I have been assuming that an intelligent structure can and should be coded as a loop (see, for example, ‘Two loops, one inside the other’). Still, as I am trying to code the process of my own heuristic learning, I realize that an algorithmic loop has fundamental flaws in that respect. Essentially, each experimental round – where I pitch the outcomes of my actions against a pre-defined expected outcome – is a separate loop, as I have to feed forward the resulting error. With many experimental rounds, like thousands, making a separate loop for each of them is algorithmically clumsy. I know it even at my neophytic stage of advancement in Python.

When I don’t know what to do, I can ask around. I can ask people smarter than me. And so I ask:

>> https://hackernoon.com/building-a-feedforward-neural-network-from-scratch-in-python-d3526457156b

>> https://towardsdatascience.com/how-to-build-your-own-neural-network-from-scratch-in-python-68998a08e4f6

After rummaging a bit in the content available under those links, I realize that intelligent structures can be represented algorithmically as classes (https://docs.python.org/3/tutorial/classes.html ), and it is more functional a way than representing them as loops. From the second of the above-mentioned links, I took an example of algorithm, which I allow myself to reproduce below. Discussing this algorithm will help me wrapping my own mind around it and developing new understandings.

(Source: https://towardsdatascience.com/how-to-build-your-own-neural-network-from-scratch-in-python-68998a08e4f6)

A neural network is a class, i.e. a type of object, which allows creating many different instances of itself. Inside the class, types of instances are defined, using selves: ‘self.input’, ‘self.output’ etc. Selves are composed into distinct functions, introduced with the command ‘def’. Among the three functions defined inside the class ‘NeuralNetwork’, one is particularly interesting, namely the ‘_init_’. As I rummage through online resources, it turns out that ‘_init_’ serves to create objects inside a class, and then to make selves of those objects.

I am trying to dissect the use of ‘_init_’ in this specific algorithm. It is introduced with three attributes: self, x, and y. I don’t quite get the corresponding logic. I am trying with something simpler: an algorithm I found at https://www.tutorialspoint.com/What-is-difference-between-self-and-init-methods-in-python-Class :

I think I start to understand. Inside the ‘_init_’ function, I need to signal there are different instances – selves – of the class I create. Then, I add the variables I intend to use. In other words, each specific self of the class ‘Rectangle’ has three dimensions: length, breadth, and unit cost.

I am trying to apply this logic to my initial problem, i.e. my own heuristic learning, with the bold assumption that I am an intelligent structure. I go:

>> class MyLearning:

def _init_(self, action, outcomes, new_knowledge = 0):

self.action = action

self.outcomes = outcomes

self.new_knowledge = new_knowledge

def learning(self):

return action-outcomes

When I run this code, there is no error message from Python, which is encouraging for a newbie such as I am. Mind you, I have truly vague an idea of what I have just coded. I know it is grammatically correct in Python.

Boots on the ground

I continue the fundamental cleaning in my head, as the year 2020 touches to its end. What do I want? Firstly,I want to exploit and develop on my hypothesis of collective intelligence in human societies, and I want to develop my programming skills in Python. Secondly, I want to develop my skills and my position as a facilitator and manager of research projects at the frontier of the academic world and that of business. How will I know I have what I want? If I actually program a workable (and working) intelligent structure, able to uncover and reconstruct the collective intelligence of a social structure out of available empirical data – namely to uncover and reconstruct the chief collective outcomes that structure is after, and its patterns of reaction to random exogenous disturbances – that would be an almost tangible outcome for me, telling me I have made a significant step. When I see that I have repetitive, predictable patterns of facilitating the start of joint research projects in consortiums of scientific entities and business ones, then I know I have nailed down something in terms of project management. If I can start something like an investment fund for innovative technologies, then I definitely know I am on the right track.

As I want to program an intelligent structure, it is essentially an artificial neural network, possibly instrumented with additional functions, such as data collection, data cleansing etc. I know I want to understand very specifically what my neural network does. I want to understand every step in takes. To that purpose, I need to figure out a workable algorithm of my own, where I understand every line of code. It can be sub-optimally slow and limited in its real computational power, yet I need it. On the other hand, Internet is more and more equipped with platforms and libraries in the form of digital clouds, such as IBM Watson, or Tensorflow, which provide optimized processes to build complex pieces of AI. I already know that being truly proficient in Data Science entails skills pertinent to using those cloud-based tools. My bottom line is that if I want to program an intelligent structure communicable and appealing to other people, I need to program it at two levels: as my own prototypic code, and as a procedure of using cloud-based platforms to replicate it.

At the juncture of those two how-will-I-know pieces of evidence, an idea emerges, a crazy one. What if I can program an intelligent structure which uncovers and reconstructs one or more alternative business models out of the available empirical data? Interesting. The empirical data I work the most with, as regards business models, is the data provided in the annual reports of publicly listed companies. Secondarily, data about financial markets sort of connects. My own experience as small investor supplies me with existential basis to back this external data, and that experience suggests me to define a business model as a portfolio of assets combined with broadly spoken behavioural patterns both in people active inside the business model, thus running it and employed with it, and in people connected to that model from outside, as customers, suppliers, investors etc.

How will other people know I have what I want? The intelligent structure I will have programmed has to work across different individual environments, which is an elegant way of saying it should work on different computers. Logically, I can say I have clearly demonstrated to other people that I achieved what I wanted with that thing of collective intelligence when said other people will be willing to and successful at trying my algorithm. Here comes the point of willingness in other people. I think it is something like an existential thing across the board. When we want other people to try and do something, and they don’t, we are pissed. When other people want us to try and do something, and we don’t, we are pissed, and they are pissed. As regards my hypothesis of collective intelligence, I have already experienced that sort of intellectual barrier, when my articles get reviewed. Reviewers write that my hypothesis is interesting, yet not articulate and not grounded enough. Honestly, I can’t blame them. My feeling is that it is even hard to say that I have that hypothesis of collective intelligence. It is rather as if that hypothesis was having me as its voice and speech. Crazy, I know, only this is how I feel about the thing, and I know by experience that good science (and good things, in general) turn up when I am honest with myself.

My point is that I feel I need to write a book about that concept of collective intelligence, in order to give a full explanation of my hypothesis. My observations about cities and their role in the human civilization make, for the moment, one of the most tangible topics I can attach the theoretical hypothesis to. Writing that book about cities, together with programming an intelligent structure, takes a different shade, now. It becomes a complex account of how we can deconstruct something – our own collective intelligence – which we know is there and yet, as we are inside that thing, we have hard times to describe it.

That book about cities, abundantly referring to my hypothesis of collective intelligence, could be one of the ways to convince other people to at least try what I propose. Thus, once again, I restate what I understand by intelligent structure. It is a structure which learns new patterns by experimenting with many alternative versions of itself, whilst staying internally coherent. I return to my ‘DU_DG’ database about cities (see ‘It is important to re-assume the meaning’) and I am re-assuming the concept of alternative versions, in an intelligent structure.

I have a dataset structured into n variables and m empirical observations. In my DU_DG database, as in many other economic datasets, distinct observations are defined as the state of a country in a given year. As I look at the dataset (metaphorically, it has content and meaning, but it does not have any physical shape save for the one my software supplies it with), and as I look at my thoughts (metaphorically, once again), I realize I have been subconsciously distinguishing two levels of defining an intelligent structure in that dataset, and, correspondingly, two ways of defining alternative versions thereof. At the first level, the entire dataset is supposed to be an intelligent structure and alternative versions thereof consist in alternative dichotomies of the type ‘one variable as output, i.e. as the desired outcome to optimize, and the remaining ones as instrumental input’. At this level of structural intelligence – by which term I understand the way of being in an intelligent structure – alternative versions are alternative orientations, and there are as many of them as there are variables.

Distinction into variables is largely, although not entirely, epistemic, and not ontological. The headcount of urban population is not fundamentally different phenomenon from the surface of agricultural land. Yes, the units of measurement are different, i.e. people vs. square kilometres, but, ontologically, it is largely the same existential stuff, possible to describe as people living somewhere in large numbers and being successful at it. Historically, social scientists and governments alike have come to the conclusion, though, that these two metrics have different a meaning, and thus it comes handy to distinguish them as semantic vessels to collect and convey information. The distinction of alternative orientations in an intelligent structure, supposedly represented in a dataset, is arbitrary and cognitive more than ontological. It depends on the number of variables we have. If I add variables to the dataset, e.g. by computing coefficients between the incumbent variables, I can create new orientations for the intelligent structure, i.e. new alternative versions to experiment with.

The point which comes to my mind is that the complexity of an intelligent structure, at that first level, depends on the complexity of my observational apparatus. The more different variables I can distinguish, and measure as regards a given society, the more complexity I can give to the allegedly existing, collectively intelligent structure of that society.

Whichever combination ‘output variable vs. input variables’ I am experimenting with, there comes the second level of defining intelligent structures, i.e. that of defining them as separate countries. They are sort of local intelligent structures, and, at the same time, they are alternative experimental versions of the overarching intelligent structure to be found in the vector of variables. Each such local intelligent structure, with a flag, a national anthem, and a government, produces many alternative versions of itself in consecutive years covered by the window of observation I have in my dataset.

I can see a subtle distinction here. A country produces alternative versions of itself, in different years of its existence, sort of objectively and without giving a f**k about my epistemological distinctions. It just exists and tries to be good at it. Experimenting comes as natural in the flow of time. This is unguided learning. On the other hand, I produce different orientations of the entire dataset. This is guided learning. Now, I understand the importance of the degree of supervision in artificial neural networks.

I can see an important lesson for me, here. If I want to program intelligent structures ‘able to uncover and reconstruct the collective intelligence of a social structure out of available empirical data – namely to uncover and reconstruct the chief collective outcomes that structure is after, and its patterns of reaction to random exogenous disturbances’, I need to distinguish those two levels of learning in the first place, namely the unguided flow of existential states from the guided structuring into variables and orientations. When I have an empirical dataset and I want to program an intelligent structure able to deconstruct the collective intelligence represented in that dataset, I need to define accurately the basic ontological units, i.e. the fundamentally existing things, then I define alternative states of those things, and finally I define alternative orientations.

Now, I am contrasting. I pass from those abstract thoughts on intelligent structures to a quick review of my so-far learning to program those structures in Python. Below, I present that review as a quick list of separate files I created in JupyterLab, together with a quick characteristic of problems I am trying to solve in each of those files, as well as of the solutions found and not found.

>> Practice Dec 11 2020.iypnb.

In this file, I work with IMF database WEOOct2020 (https://www.imf.org/en/Publications/WEO/weo-database/2020/October ). I practiced reading complex datasets, with an artificially flattened structure. It is a table, in which index columns are used to add dimensions to an otherwise two-dimensional format. I practiced the ‘read_excel’ and ‘read_csv’ commands. On the whole, it seems that converting an Excel to CSV and then reading CSV in Python is a better method than reading excel. Problems solved: a) cleansing the dataset of not-a-number components and successful conversion of initially ‘object’ columns into the desired ‘float64’ format b) setting descriptive indexes to the data frame c) listing unique labels from a descriptive index d) inserting new columns into the data frame e) adding (compounding) the contents of two existing, descriptive index columns into a third index column. Failures: i) reading data from XML file ii) reading data from SDMX format iii) transposing my data frame so as to put index values of economic variables as column names and years as index values in a column.

>> Practice Dec 8 2020.iypnb.

In this file, I worked with a favourite dataset of mine, the Penn Tables 9.1. (https://www.rug.nl/ggdc/productivity/pwt/?lang=en ). I described my work with it in two earlier updates, namely ‘Two loops, one inside the other’, and ‘Mathematical distance’. I succeeded in creating an intelligent structure from that dataset. I failed at properly formatting the output of that structure and thus at comparing the cognitive value of different orientations I made it simulate.

>> Practice with Mortality.iypnb.

I created this file as a first practice before working with the above-mentioned WEOOct2020 database. I took one dataset from the website of the World Bank, namely that pertinent to the coefficient of adult male mortality (https://data.worldbank.org/indicator/SP.DYN.AMRT.MA ). I practiced reading data from CSV files, and I unsuccessfully tried to stack the dataset, i.e. to transform columns corresponding to different years of observation into rows indexed with labels corresponding to years.

>> Practice DU_DG.iypnb.

In this file, I am practicing with my own dataset pertinent to the density of urban population and its correlates. The dataset is already structured in Excel. I start practicing the coding of the same intelligent structure I made with Penn Tables, supposed to study orientation of the societies studied. Same problems and same failures as with Penn Tables 9.1.: for the moment, I cannot nail down the way to get output data in structures that allow full comparability. My columns tend to wander across the output data frames. In other words, the vectors of mean expected values produced by the code I made have slightly (just slightly, and sufficiently to be annoying) different a structure from the original dataset. I don’t know why, yet, and I don’t know how to fix it.

On the other hand, in that same file, I have been messing around a bit with algorithms based on the ‘scikit’ library for Python. Nice graphs, and functions which I still need to understand.

>> Practice SEC Financials.iypnb.

Here, I work with data published by the US Securities and Exchange Commission, regarding the financials of individual companies listed in the US stock market (https://www.sec.gov/dera/data/financial-statement-data-sets.html ). The challenge here consists in translating data originally supplied in *.TXT files into numerical data frames in Python. The problem with I managed to solve, so far (this is the most recent piece of my programming), is the most elementary translation of TXT data into a Pandas data frame, using the ‘open()’ command, and the ‘f.readlines()’ one. Another small victory here is to read data from a sub-directory inside the working directory of JupyterLab, i.e. inside the root directory of my user profile. I used two methods of reading TXT data. Both worked sort of. First, I used the following sequence:

>> with open(‘2020q3/num.txt’) as f:

numbers=f.readlines()

>> Numbers=pd.DataFrame(numbers)

… which, when checked with the ‘Numbers.info()’ command, yields:

<class ‘pandas.core.frame.DataFrame’>

RangeIndex: 2351641 entries, 0 to 2351640

Data columns (total 1 columns):

# Column Dtype

— —— —–

0 0 object

dtypes: object(1)

memory usage: 17.9+ MB

In other words, that sequence did not split the string of column names into separate columns, and the ‘Numbers’ data frame contains one column, in which every row is a long string structured with the ‘\’ separators. I tried to be smart with it. I did:

>> Numbers.to_csv(‘Num2’) # I converted the Pandas data frame into a CSV file

>> Num3=pd.DataFrame(pd.read_csv(‘Num2′,sep=’;’)) # …and I tried to read back from CSV, experimenting with different separators. None of it worked. With the ‘sep=’ argument in the command, I kept getting an error of parsing, in the lines of ‘ParserError: Error tokenizing data. C error: Expected 1 fields in line 3952, saw 10’. When I didn’t use the ‘sep=’ argument, the command did not yield error, yet it yielded the same long column of structured strings instead of many data columns.

Thus, I gave up a bit, and I used Excel to open the TXT file, and to save a copy of it in the CSV format. Then, I just created a data frame from the CSV dataset, through the ‘NUM_from_CSV=pd.DataFrame(pd.read_csv(‘SEC_NUM.csv’, sep=’;’))’ command, which, checked with the ‘NUM_from_CSV.info()’ command, yields:

<class ‘pandas.core.frame.DataFrame’>

RangeIndex: 1048575 entries, 0 to 1048574

Data columns (total 9 columns):

# Column Non-Null Count Dtype

— —— ————– —–

0 adsh 1048575 non-null object

1 tag 1048575 non-null object

2 version 1048575 non-null object

3 coreg 30131 non-null object

4 ddate 1048575 non-null int64

5 qtrs 1048575 non-null int64

6 uom 1048575 non-null object

7 value 1034174 non-null float64

8 footnote 1564 non-null object

dtypes: float64(1), int64(2), object(6)

memory usage: 72.0+ MB

The ‘tag’ column in this data frame contains the names of financial variables ascribed to companies identified with their ‘adsh’ codes. I experience the same challenge, and, so far, the same failure as with the WEOOct2020 database from IMF, namely translating different values in a descriptive index into a dictionary, and then, in the next step, to flip the database so as to make those different index categories into separate columns (variables).

As I have passed in review that programming of mine, I have become aware that reading and properly structuring different formats of data is the sensory apparatus of the intelligent structure I want to program. Operations of data cleansing and data formatting are the fundamental skills I need to develop in programming. Contrarily to what I expected a few weeks ago, when I was taking on programming in Python, elaborate mathematical constructs are simpler to code than I thought they would be. What might be harder, mind you, is to program them so as to optimize computational efficiency with large datasets. Still, the very basic, boots-on-the-ground structuring of data seems to be the name of the game for programming intelligent structures.

Once again, I don’t really know what I am doing, and I love the feeling

I am digressing a bit in my learning of programming in Python, and I come back to a task which I have kept failing at so far, namely at reading data out of the World Economic Outlook database, published by the International Monetary Fund (https://www.imf.org/en/Publications/WEO/weo-database/2020/October ). This is good training in data cleansing. When I click on that link, I can choose two different formats: TAB delimited values or SDMX. The former download as an Excel file, essentially. Frankly, I feel not up to treating the latter: it is a dynamic format, essentially based on an XML tree. Still to learn, for me. This is one of those cases when I prefer staying in the cave. Daylight can wait. I stick to Excel. I download it, I open it in Excel and I preliminarily cleanse the spreadsheet of the most salient stuff, such as e.g. title rows in the heading above the column labels.

Preliminary cleansing done, I copy the Excel workbook to the working directory of my Anaconda, which is, by default, the root directory of my user profile. I create a new notebook in JupyterLab, and I start by importing whatever I think can be useful:

>> import numpy as np

>> import pandas as pd

>> import os

>> import math

I check the presence of the Excel file with the ‘os.listdir()’ command, and I go:

>> WEO=pd.DataFrame(pd.read_excel(‘WEOOct2020all.xlsx’,sheet_name=’WEOOct2020all’,header=0))

Seems cool. The kernel has swallowed the command. Just in case, I check with ‘WEO.describe()’, and I get:

	Estimates Start After
count	7585.000000
mean	2015.186421
std	80.240679
min	0.000000
25%	2018.000000
50%	2019.000000
75%	2019.000000
max	2020.000000

WTF? ‘Estimates start after’ is the last column of a two-dimensional table in Excel, and this column gives the year up to which the database provides actual empirics, and after which it is just projections. Besides this one, the database contains numerical columns corresponding to years, starting from 1980. When I go ‘WEO.columns’, I get:

Index([ ‘Country’, ‘Subject Descriptor’,

‘Units’, ‘Scale’,

1980, 1981,

1982, 1983,

1984, 1985,

1986, 1987,

1988, 1989,

1990, 1991,

1992, 1993,

1994, 1995,

1996, 1997,

1998, 1999,

2000, 2001,

2002, 2003,

2004, 2005,

2006, 2007,

2008, 2009,

2010, 2011,

2012, 2013,

2014, 2015,

2016, 2017,

2018, 2019,

2020, 2021,

2022, 2023,

2024, 2025,

‘Estimates Start After’],

dtype=’object’)

Aha! These columns are there, only Python sees them as non-numerical and does not compute any stats from them. As we say in Poland, I am trying to get my man from another angle. I open the source XLSX file in Excel and I save a copy thereof in the CSV format, in the working directory of my Anaconda. I remember that when saved out of an XLSX file, CSVs tend to have the semi-column as separator, instead of the coma. To everyone their ways, mind you. Thus, I go:

>> WEO2=pd.DataFrame(pd.read_csv(‘WEOOct2020all.csv’,header=0,sep=’;’))

When I check with ‘WEO2.info()’, I get:

<class ‘pandas.core.frame.DataFrame’>

RangeIndex: 8775 entries, 0 to 8774

Data columns (total 51 columns):

# Column Non-Null Count Dtype

— —— ————– —–

0 Country 8775 non-null object

1 Subject Descriptor 8775 non-null object

2 Units 8775 non-null object

3 Scale 3900 non-null object

4 1980 3879 non-null object

5 1981 4008 non-null object

6 1982 4049 non-null object

7 1983 4091 non-null object

8 1984 4116 non-null object

9 1985 4192 non-null object

10 1986 4228 non-null object

11 1987 4249 non-null object

12 1988 4338 non-null object

13 1989 4399 non-null object

14 1990 4888 non-null object

15 1991 5045 non-null object

16 1992 5428 non-null object

17 1993 5621 non-null object

18 1994 5748 non-null object

19 1995 6104 non-null object

20 1996 6247 non-null object

21 1997 6412 non-null object

22 1998 6584 non-null object

23 1999 6662 non-null object

24 2000 7071 non-null object

25 2001 7193 non-null object

26 2002 7289 non-null object

27 2003 7323 non-null object

28 2004 7391 non-null object

29 2005 7428 non-null object

30 2006 7433 non-null object

31 2007 7441 non-null object

32 2008 7452 non-null object

33 2009 7472 non-null object

34 2010 7475 non-null object

35 2011 7477 non-null object

36 2012 7484 non-null object

37 2013 7493 non-null object

38 2014 7523 non-null object

39 2015 7545 non-null object

40 2016 7547 non-null object

41 2017 7551 non-null object

42 2018 7547 non-null object

43 2019 7539 non-null object

44 2020 7501 non-null object

45 2021 7449 non-null object

46 2022 7389 non-null object

47 2023 7371 non-null object

48 2024 7371 non-null object

49 2025 7371 non-null object

50 Estimates Start After 7585 non-null float64

dtypes: float64(1), object(50)

memory usage: 3.4+ MB

There is some progress, still it is not the entire progress I expected. I still don’t have numerical data, in ‘float64’ type, where I expect it to have. I dig a bit and I see the source of the problem. In the WEO database there is plenty of empty cells, especially before the year 2000. They correspond to missing data, quite simply. In the source XLSX file, they are either just empty, or filled with something that looks like a double hyphen: ‘- -‘. Python shows the contents of these cells as ‘NaN’, which stands for ‘Not a Number’. That double hyphen is the most annoying of the two, as Excel does not see it in the command ‘Replace’. I need to use Python. I do two phases of cleansing:

>> WEO3=WEO2.replace(np.nan,”, regex=True)

>> WEO4=WEO3.replace(‘–‘,”, regex=True)

I check with ‘WEO4.info()’ aaaaand… Bingo! Columns from ‘1980’ to ‘2025’ are of the type ‘float64’.

The WEO database is made of several variables stacked one underneath the other in consecutive rows. You have one country, and for that country you have variables such as GDP, fiscal balance and whatnot. Essentially, it is a database presented in the two-dimensional format with multiple indexes, embedded one inside the other. The complexity of indexes replaces the multitude of dimensions in the actual data. I start intuitively, with creating lists of column labels corresponding, respectively, to numerical data, and to index descriptors:

>> Numerical_Data=[‘1980’, ‘1981’,

‘1982’, ‘1983’, ‘1984’, ‘1985’, ‘1986’, ‘1987’, ‘1988’, ‘1989’, ‘1990’,

‘1991’, ‘1992’, ‘1993’, ‘1994’, ‘1995’, ‘1996’, ‘1997’, ‘1998’, ‘1999’,

‘2000’, ‘2001’, ‘2002’, ‘2003’, ‘2004’, ‘2005’, ‘2006’, ‘2007’, ‘2008’,

‘2009’, ‘2010’, ‘2011’, ‘2012’, ‘2013’, ‘2014’, ‘2015’, ‘2016’, ‘2017’,

‘2018’, ‘2019’, ‘2020’, ‘2021’, ‘2022’, ‘2023’, ‘2024’, ‘2025’]

>> Index_descriptors=[‘Country’, ‘Subject Descriptor’, ‘Units’, ‘Scale’,’Estimates Start After’]

Now, I mess around a bit with those dictionaries and with indexing that big dataset. In a moment, you will understand why I do so. I go:

>> Subject_Descriptors=pd.unique(WEO4[‘Subject Descriptor’]) # I made a data frame out of unique index labels in the column ‘Subject Descriptor’. I get:

>> array([‘Gross domestic product, constant prices’,

‘Gross domestic product, current prices’,

‘Gross domestic product, deflator’,

‘Gross domestic product per capita, constant prices’,

‘Gross domestic product per capita, current prices’,

‘Output gap in percent of potential GDP’,

‘Gross domestic product based on purchasing-power-parity (PPP) share of world total’,

‘Implied PPP conversion rate’, ‘Total investment’,

‘Gross national savings’, ‘Inflation, average consumer prices’,

‘Inflation, end of period consumer prices’,

‘Six-month London interbank offered rate (LIBOR)’,

‘Volume of imports of goods and services’,

‘Volume of Imports of goods’,

‘Volume of exports of goods and services’,

‘Volume of exports of goods’, ‘Unemployment rate’, ‘Employment’,

‘Population’, ‘General government revenue’,

‘General government total expenditure’,

‘General government net lending/borrowing’,

‘General government structural balance’,

‘General government primary net lending/borrowing’,

‘General government net debt’, ‘General government gross debt’,

‘Gross domestic product corresponding to fiscal year, current prices’,

‘Current account balance’], dtype=object)

In other words, each country is characterized in the WEOOct2020 database with the above characteristics. I need to group and extract data so as to have those variables separated. The kind of transformation which I want to nail down is to transpose those variables with years. In the source version of WEOOct2020, years are separate columns that cut across three basic indexes: countries, for one, the above presented subject descriptors, for two, and finally the indexes of units and scale. The latter is important to the extent that most macroeconomic aggregates are presented either as absolute amounts or as percentages of the country’s GDP. Probably you remember from math classes at school, and those of physics and chemistry too, actually, that confusing units of measurement is a cardinal sin in science. What I want to do is to flip the thing on its side. I want each country to be associated with a series of index labels corresponding to years, and variables associated with proper units of measurement being the columns of the dataset.

In other words, now, years are the main quantitative categories of the WEOOct202 data frame, and categorial variables are index labels, or phenomenological units of observation. I want these two to change places, as it essentially should be: categorial variables should become phenomenological categories, and years should gracefully step down to the status of observational units.

As I don’t know what to do, I reach to what I know how to do, i.e. to creating some sort of dictionaries out of index labels. What I did for subject descriptors, I do for units,

>> Units=pd.unique(WEO4[‘Units’])

…which yields:

array([‘National currency’, ‘Percent change’, ‘U.S. dollars’,

‘Purchasing power parity; international dollars’, ‘Index’,

‘Purchasing power parity; 2017 international dollar’,

‘Percent of potential GDP’, ‘Percent’,

‘National currency per current international dollar’,

‘Percent of GDP’, ‘Percent of total labor force’, ‘Persons’],

dtype=object)

and…

>> Countries=pd.unique(WEO4[‘Country’])

…which yields:

array([‘Afghanistan’, ‘Albania’, ‘Algeria’, ‘Angola’,

‘Antigua and Barbuda’, ‘Argentina’, ‘Armenia’, ‘Aruba’,

‘Australia’, ‘Austria’, ‘Azerbaijan’, ‘The Bahamas’, ‘Bahrain’,

‘Bangladesh’, ‘Barbados’, ‘Belarus’, ‘Belgium’, ‘Belize’, ‘Benin’,

‘Bhutan’, ‘Bolivia’, ‘Bosnia and Herzegovina’, ‘Botswana’,

‘Brazil’, ‘Brunei Darussalam’, ‘Bulgaria’, ‘Burkina Faso’,

‘Burundi’, ‘Cabo Verde’, ‘Cambodia’, ‘Cameroon’, ‘Canada’,

‘Central African Republic’, ‘Chad’, ‘Chile’, ‘China’, ‘Colombia’,

‘Comoros’, ‘Democratic Republic of the Congo’, ‘Republic of Congo’,

‘Costa Rica’, “CÙte d’Ivoire”, ‘Croatia’, ‘Cyprus’,

‘Czech Republic’, ‘Denmark’, ‘Djibouti’, ‘Dominica’,

‘Dominican Republic’, ‘Ecuador’, ‘Egypt’, ‘El Salvador’,

‘Equatorial Guinea’, ‘Eritrea’, ‘Estonia’, ‘Eswatini’, ‘Ethiopia’,

‘Fiji’, ‘Finland’, ‘France’, ‘Gabon’, ‘The Gambia’, ‘Georgia’,

‘Germany’, ‘Ghana’, ‘Greece’, ‘Grenada’, ‘Guatemala’, ‘Guinea’,

‘Guinea-Bissau’, ‘Guyana’, ‘Haiti’, ‘Honduras’, ‘Hong Kong SAR’,

‘Hungary’, ‘Iceland’, ‘India’, ‘Indonesia’,

‘Islamic Republic of Iran’, ‘Iraq’, ‘Ireland’, ‘Israel’, ‘Italy’,

‘Jamaica’, ‘Japan’, ‘Jordan’, ‘Kazakhstan’, ‘Kenya’, ‘Kiribati’,

‘Korea’, ‘Kosovo’, ‘Kuwait’, ‘Kyrgyz Republic’, ‘Lao P.D.R.’,

‘Latvia’, ‘Lebanon’, ‘Lesotho’, ‘Liberia’, ‘Libya’, ‘Lithuania’,

‘Luxembourg’, ‘Macao SAR’, ‘Madagascar’, ‘Malawi’, ‘Malaysia’,

‘Maldives’, ‘Mali’, ‘Malta’, ‘Marshall Islands’, ‘Mauritania’,

‘Mauritius’, ‘Mexico’, ‘Micronesia’, ‘Moldova’, ‘Mongolia’,

‘Montenegro’, ‘Morocco’, ‘Mozambique’, ‘Myanmar’, ‘Namibia’,

‘Nauru’, ‘Nepal’, ‘Netherlands’, ‘New Zealand’, ‘Nicaragua’,

‘Niger’, ‘Nigeria’, ‘North Macedonia’, ‘Norway’, ‘Oman’,

‘Pakistan’, ‘Palau’, ‘Panama’, ‘Papua New Guinea’, ‘Paraguay’,

‘Peru’, ‘Philippines’, ‘Poland’, ‘Portugal’, ‘Puerto Rico’,

‘Qatar’, ‘Romania’, ‘Russia’, ‘Rwanda’, ‘Samoa’, ‘San Marino’,

‘S„o TomÈ and PrÌncipe’, ‘Saudi Arabia’, ‘Senegal’, ‘Serbia’,

‘Seychelles’, ‘Sierra Leone’, ‘Singapore’, ‘Slovak Republic’,

‘Slovenia’, ‘Solomon Islands’, ‘Somalia’, ‘South Africa’,

‘South Sudan’, ‘Spain’, ‘Sri Lanka’, ‘St. Kitts and Nevis’,

‘St. Lucia’, ‘St. Vincent and the Grenadines’, ‘Sudan’, ‘Suriname’,

‘Sweden’, ‘Switzerland’, ‘Syria’, ‘Taiwan Province of China’,

‘Tajikistan’, ‘Tanzania’, ‘Thailand’, ‘Timor-Leste’, ‘Togo’,

‘Tonga’, ‘Trinidad and Tobago’, ‘Tunisia’, ‘Turkey’,

‘Turkmenistan’, ‘Tuvalu’, ‘Uganda’, ‘Ukraine’,

‘United Arab Emirates’, ‘United Kingdom’, ‘United States’,

‘Uruguay’, ‘Uzbekistan’, ‘Vanuatu’, ‘Venezuela’, ‘Vietnam’,

‘West Bank and Gaza’, ‘Yemen’, ‘Zambia’, ‘Zimbabwe’], dtype=object)

Once again, I don’t really know what I am doing. I just intuitively look for some sort of landmarks in that landscape of data. By the way, this is what we all do when we don’t know what to do: we look for reliable ways to partition observable reality into categories.

Now, I want to make sure that Python has the same views as me as for what index descriptors are in that dataset. I go:

>> pd.MultiIndex.from_frame(WEO4[Index_descriptors])

… and I get:

MultiIndex([(‘Afghanistan’, …),

(‘Afghanistan’, …),

…

( ‘Zimbabwe’, …),

( ‘Zimbabwe’, …)],

names=[‘Country’, ‘Subject Descriptor’, ‘Units’, ‘Scale’, ‘Estimates Start After’], length=8775)

Seems OK.

Now, I need to fuse somehow the index of Subject Descriptor with the Index of Units, so as to have categories ready for flipping. I keep sort of feeling my way forward, rather than seeing it clearly. Love it, actually. I create an empty data series to contain the merged indexes of ‘Subject Descriptor’ and ‘Units’:

>> Variable=pd.Series(‘object’) # The ‘object’ part means that I want to have words in that data series

Now, I append and I check:

>> WEO4.append(Variable,ignore_index=True)

Aaaand… it doesn’t work. When I check ‘WEO4.info()’, I get the list of columns I had before, without the ‘Variable’. In other words, Python acknowledged that I want to append that columns, and it sort of appended, but just sort of. There is that thing I have already learnt with Python: there is a huge difference between having sort of expected output, on the one hand, and having it 100%, on the other hand. The one hand is bloody frustrating.

I try another trick, the ‘df.insert’ command. I do:

>> WEO4.insert(1,’Variable’,’ ‘)

I check with ‘WEO4.info()’ aaaaand….this time, it worked sort of. I get the new column ‘Variable’, yes, and I have all my numerical columns, the one with ‘Year’ headers, turned back into the ‘object’ format. I f**king love programming. I do:

>> for i in range(0,len(WEO4.columns)):

WEO4.iloc[:,i]=pd.to_numeric(WEO4.iloc[:,i], errors=’ignore’)

… and I check with ‘WEO4.info()’ once again. Victory: numerical is back to numerical.

Now, I am looking for a method to sort of concatenate smartly the contents of two incumbent columns, namely ‘Subject Descriptor’ and ‘Units’, into the new vessel, i.e. the column ‘Variable’. I found the simplest possible method, which is straightforward addition:

>> WEO4[“Variable”]=WEO4[“Subject Descriptor”]+WEO4[“Units”]

I typed it, I executed, and, as strange as it seems, Python seems to be OK with that. Well, after all, Python is a language, and languages have that thing: they add words to each other. It is called ‘making sentences’. Cool. I check by creating an array of unique values in the index labels of ‘Variable:

>> Variables=pd.unique(WEO4[‘Variable’])

I check by just typing:

>> Variables

… and running it as a command. I get:

array([‘Gross domestic product, constant pricesNational currency’,

‘Gross domestic product, constant pricesPercent change’,

‘Gross domestic product, current pricesNational currency’,

‘Gross domestic product, current pricesU.S. dollars’,

‘Gross domestic product, current pricesPurchasing power parity; international dollars’,

‘Gross domestic product, deflatorIndex’,

‘Gross domestic product per capita, constant pricesNational currency’,

‘Gross domestic product per capita, constant pricesPurchasing power parity; 2017 international dollar’,

‘Gross domestic product per capita, current pricesNational currency’,

‘Gross domestic product per capita, current pricesU.S. dollars’,

‘Gross domestic product per capita, current pricesPurchasing power parity; international dollars’,

‘Output gap in percent of potential GDPPercent of potential GDP’,

‘Gross domestic product based on purchasing-power-parity (PPP) share of world totalPercent’,

‘Implied PPP conversion rateNational currency per current international dollar’,

‘Total investmentPercent of GDP’,

‘Gross national savingsPercent of GDP’,

‘Inflation, average consumer pricesIndex’,

‘Inflation, average consumer pricesPercent change’,

‘Inflation, end of period consumer pricesIndex’,

‘Inflation, end of period consumer pricesPercent change’,

‘Six-month London interbank offered rate (LIBOR)Percent’,

‘Volume of imports of goods and servicesPercent change’,

‘Volume of Imports of goodsPercent change’,

‘Volume of exports of goods and servicesPercent change’,

‘Volume of exports of goodsPercent change’,

‘Unemployment ratePercent of total labor force’,

‘EmploymentPersons’, ‘PopulationPersons’,

‘General government revenueNational currency’,

‘General government revenuePercent of GDP’,

‘General government total expenditureNational currency’,

‘General government total expenditurePercent of GDP’,

‘General government net lending/borrowingNational currency’,

‘General government net lending/borrowingPercent of GDP’,

‘General government structural balanceNational currency’,

‘General government structural balancePercent of potential GDP’,

‘General government primary net lending/borrowingNational currency’,

‘General government primary net lending/borrowingPercent of GDP’,

‘General government net debtNational currency’,

‘General government net debtPercent of GDP’,

‘General government gross debtNational currency’,

‘General government gross debtPercent of GDP’,

‘Gross domestic product corresponding to fiscal year, current pricesNational currency’,

‘Current account balanceU.S. dollars’,

‘Current account balancePercent of GDP’], dtype=object)

Cool. It seems to have worked.

Mathematical distance

I continue learning Python as regards data analysis. I have a few thoughts on what I have already learnt, and a new challenge, namely to repeat the same thing with another source of data, namely the World Economic Outlook database, published by the International Monetary Fund (https://www.imf.org/en/Publications/WEO/weo-database/2020/October ). My purpose is to use that data in the same way as I used that from Penn Tables 9.1 (see ‘Two loops, one inside the other’, for example), namely to run it through a digital intelligent structure consisting of a double algorithmic loop.

First things first, I need to do what I promised to do in Two loops, one inside the other, that is to test the cognitive value of the algorithm I presented there. By the way, as I keep working with algorithms known as ‘artificial intelligence’, I am more and more convinced that the term ‘artificial neural networks’ is not really appropriate. I think that talking about artificial intelligent structure is much closer to reality. Giving the name of ‘neurons’ to particular fragments of the algorithm reflects the properties of some of those neurons, I get it. Yet, the neurons of a digital technology are the micro-transistors in the CPU or in the GPU. Yes, micro-transistors do what neurons do in our brain: they fire conditionally and so they produce neural signals. Algorithms of AI can be run on any computer with proper software. AI is software, not hardware.

Yes, I know I’m ranting. This is how I am gathering intellectual speed for my writing. Learning to program in Python has led me to a few realizations about the digital intelligent structures I am working with, as simulators of collective intelligence in human societies. Algorithms are different from equations in the sense that algorithms do things, whilst equations represent things. When I want an algorithm to do the things represented with equations, I need functional coherence between commands. A command needs data to work on, and it is a good thing if I can utilize the data it puts out. A chain of commands is functional, when earlier commands give accurate input to later commands, and when the final output of the last command can be properly stored and utilized. On the other hand, equations don’t need data to work, because equations don’t work. They just are.

I can say my equations are fine when they make sense logically. On the other hand, I can be sure my algorithm works the way it is supposed to work, when I can empirically prove its functionality by testing it. Hence, I need a method of testing it and I need to be sure the method in itself is robust. Now, I understand why the hell in all the tutorials which I could find as regards programming in Python there is that ‘print(output)’ command at the end of each algorithm. Whatever the output is, printing it, i.e. displaying it on the screen, is the most elementary method of checking whether that output is what I expect it to be. By the way, I have made my own little discovery about the usefulness of the ‘print()’ command. In looping algorithms, which, by nature, are prone to looping forever if the range of iterations is not properly defined, I put ‘print(‘Finished’)’ at the very end of the code. When I see ‘Finished’ printed in the line below, I can be sure the thing has done the work it was supposed to do.

Good, I was supposed to write about the testing of my algorithm. How do I test? I start by taking small pieces of the algorithm and checking the kind of output they give. By doing that, I modified the algorithm from ‘Two loops, one inside the other’, into the form you can see below:

That’s the preliminary part: importing libraries and data for analysis >>

In [1]: import numpy as np

…: import pandas as pd

…: import os

…: import math

In [2]: PWT=pd.DataFrame(pd.read_csv(‘PWT 9_1 no empty cells.csv’,header=0)) # PWT 9_1 no empty cells.csv is a CSV version of the database I made with non-empty observations in the Penn Tables 9.1 database

Now, I extract the purely numerical columns, into another data frame, which I label ‘PWT_Numerical’

In [3]: Variables=[‘rgdpe’, ‘rgdpo’, ‘pop’, ’emp’, ’emp / pop’, ‘avh’,

…: ‘hc’, ‘ccon’, ‘cda’, ‘cgdpe’, ‘cgdpo’, ‘cn’, ‘ck’, ‘ctfp’, ‘cwtfp’,

…: ‘rgdpna’, ‘rconna’, ‘rdana’, ‘rnna’, ‘rkna’, ‘rtfpna’, ‘rwtfpna’,

…: ‘labsh’, ‘irr’, ‘delta’, ‘xr’, ‘pl_con’, ‘pl_da’, ‘pl_gdpo’, ‘csh_c’,

…: ‘csh_i’, ‘csh_g’, ‘csh_x’, ‘csh_m’, ‘csh_r’, ‘pl_c’, ‘pl_i’, ‘pl_g’,

…: ‘pl_x’, ‘pl_m’, ‘pl_n’, ‘pl_k’]

In [4]: PWT_Numerical=pd.DataFrame(PWT[Variables])

My next step is to practice with creating dictionaries out of column names in my data frame

In [5]: Names_Output_Data=[]

…: for i in range(42):

…: Name_Output_Data=PWT_Numerical.iloc[:,i].name

…: Names_Output_Data.append(Name_Output_Data)

I start coding the intelligent structure. I start by defining empty lists, to store data which the intelligent structure produces.

In [6]: ER=[]

…: Transformed=[]

…: MEANS=[]

…: EUC=[]

I define an important numerical array in NumPy: the vector of mean expected values in the variables of PWT_Numerical.

…: Source_means=np.array(PWT_Numerical.mean())

I open the big external loop of my intelligent structure. This loop is supposed to produce as many alternative intelligent structures as there are variables in my PWT_Numerical data frame.

…: for i in range(42):

…: Name_Output_Data=PWT_Numerical.iloc[:,i].name

…: Names_Output_Data.append(Name_Output_Data)

…: Output=pd.DataFrame(PWT_Numerical.iloc[:,i]) # I make an output data frame

…: Mean=Output.mean()

…: MEANS.append(Mean) # I store the expected mean of each output variable in a separate list.

…: Input=pd.DataFrame(PWT_Numerical.drop(Output,axis=1)) # I make an input data frame, coupled with output

…: Input_STD=pd.DataFrame(Input/Input.max(axis=0)) # I standardize input data over the respective maximum of each variable

…: Input_Means=pd.DataFrame(Input.mean()) # I prepare two data frames sort of for later: one with the vector of means…

…: Input_Max=pd.DataFrame(Input.max(axis=0)) #… and another one with the vector of maximums

Now, I put in motion the intelligent structure strictly speaking: a simple perceptron, which…

…: for j in range(10): # … is short, for testing purposes, just 10 rows in the source data

…: Input_STD_randomized=np.array(Input_STD.iloc[j])*np.random.rand(41) #… sprays the standardized input data with random weights

…: Input_STD_summed=Input_STD_randomized.sum(axis=0) # … and then sums up sort of ∑(input variable *random weight).

…: T=math.tanh(Input_STD_summed) # …computes the hyperbolic tangent of summed randomized input. This is neural activation.

…: D=1-(T**2) # …computes the local first derivative of that hyperbolic tangent

…: E=(Output.iloc[j]-T)*D # … computes the local error of estimating the value of output variable, with input data neural-activated with the function of hyperbolic tangent

…: E_vector=np.array(np.repeat(E,41)) # I spread the local error into a vector to feed forward

…: Next_row_with_error=Input_STD.iloc[j+1]+E_vector # I feed the error forward

…: Next_row_DESTD=Next_row_with_error*Input.max(axis=0) # I destandardize

…: ER.append(E) # I store local errors in the list ER

…: ERROR=pd.DataFrame(ER) # I make a data frame out of the list ER

…: Transformed.append(Next_row_with_error) # I store the input values transformed by the perceptron (through the forward feed of error), in the list Transformed

…: TR=pd.DataFrame(Transformed) # I turn the Transformed list into a data frame

…: MEAN_TR=pd.DataFrame(TR.mean()) # I compute the mean values of transformed input and store them in a data frame. They are still mean values of standardized data.

…: MEAN_TR_DESTD=pd.DataFrame(MEAN_TR*Input_Max) # I destandardise

…: MEANS_DF=pd.DataFrame(MEANS)

…: print(MEANS)

…: print(‘Finished’)

The general problem which I encounter with that algorithm is essentially that of reading correctly and utilizing the output, or, at least, this is how I understand that problem. First, I remind the general hypothesis which I want to test and explore with that algorithm presented above. Here it comes: for a given set of phenomena, informative about the state of a human social structure, and observable as a dataset of empirical values in numerical variables, there is a subset of variables which inform about the ethical and functional orientation of that human social structure; orientation manifests as relatively the least significant transformation, which the original dataset needs to undergo in order to minimize error in estimating the orientation-informative variable as output, when the remaining variables are used as input.

When the empirical dataset in question is being used as training set for an artificial neural network of the perceptron type, i.e. a network which tests for the optimal values in the input variables, for minimizing the error in estimating the output variable, such neural testing transforms the original dataset into a specific version thereof. I want to know how far away from the original empirical dataset does the specific transformation, oriented on a specific output, go. I measure that mathematical distance as the Euclidean distance between the vector of mean expected values in the transformed dataset, and the original one.

Therefore, I need two data frames in Pandas, or two arrays in NumPy, one containing the mean expected values of the original input data, the other storing mean expected values of the transformed dataset. Here is where my problems start, with the algorithm presented above. The ‘TR’ data frame has a different shape and structure than the ‘Input’ data frame, from which, technically, it is derived. The Input data frame has 41 columns, and the TR has 42 columns. Besides, one column from ‘Input’, the ‘rgdpe’, AKA real GDP on the final consumption side, moves from being the first column in ‘Input’ to being the last ‘column’ in ‘TR’. For the moment, I have no clue what’s going on at that level. I even checked the algorithm with a debugger, available with the integrated development environment called Spyder (https://www.spyder-ide.org ). Technically, as far as the grammar of Python is concerned, the algorithm is OK. Still, it produces different than expected vectors of mean expected values in transformed data. I don’t even know where to start looking for a solution.

There is one more thing I want to include in this algorithm, which I have already been doing in Excel. At each row of transformed data, thus at each ‘Next_row_with_error’, I want to add a mirroring row of mean Euclidean distance from each individual variable to all the remaining ones. It is a measure of internal coherence in the process of learning through trial and error, and I already know, having learnt it by trial and error, that including that specific metric, and feeding it forward together with the error, changes a lot in the way a perceptron learns.

Pardon my French, but the thing is really intelligent

My editorial on You Tube

And so I am meddling with neural networks. It had to come. It just had to. I started with me having many ideas to develop at once. Routine stuff with me. Then, the Editor-in-Chief of the ‘Energy Economics’ journal returned my manuscript of article on the energy-efficiency of national economies, which I had submitted with them, with a general remark that I should work both on the clarity of my hypotheses, and on the scientific spin of my empirical research. In short, Mr Wasniewski, linear models tested with Ordinary Least Squares is a bit oldie, if you catch my drift. Bloody right, Mr Editor-In-Chief. Basically, I agree with your remarks. I need to move out of my cavern, towards the light of progress, and get acquainted with the latest fashion. The latest fashion we are wearing this season is artificial intelligence, machine learning, and neural networks.

It comes handy, to the extent that I obsessively meddle with the issue of collective intelligence, and am dreaming about creating a model of human social structure acting as collective intelligence, sort of a beehive. Whilst the casting for a queen in that hive remains open, and is likely to stay this way for a while, I am digging into the very basics of neural networks. I am looking in the Python department, as I have already got a bit familiar with that environment. I found an article by James Loy, entitled “How to build your own Neural Network from scratch in Python”. The article looks a bit like sourcing from another one, available at the website of ProBytes Software, thus I use both to develop my understanding. I pasted the whole algorithm by James Loy into my Python Shell, made in run with an ‘enter’, and I am waiting for what it is going to produce. In the meantime, I am being verbal about my understanding.

The author declares he wants to do more or less the same thing that I, namely to understand neural networks. He constructs a simple algorithm for a neural network. It starts with defining the neural network as a class, i.e. as a callable object that acts as a factory for new instances of itself. In the neural network defined as a class, that algorithm starts with calling the constructor function ‘_init_’, which constructs an instance ‘self’ of that class. It goes like ‘def __init__(self, x, y):’. In other words, the class ‘Neural network’ generates instances ‘self’ of itself, and each instance is essentially made of two variables: input x, and output y. The ‘x’ is declared as input variable through the ‘self.input = x’ expression. Then, the output of the network is defined in two steps. Yes, the ‘y’ is generally the output, only in a neural network, we want the network to predict a value of ‘y’, thus some kind of y^. What we have to do is to define ‘self.y = y’, feed the real x-s and the real y-s into the network, and expect the latter to turn out some y^-s.

Logically, we need to prepare a vessel for holding the y^-s. The vessel is defined as ‘self.output = np.zeros(y.shape)’. The ‘shape’ function defines a tuple – a table, for those mildly fond of maths – with given dimensions. What are the dimensions of ‘y’ in that ‘y.shape’? They have been given earlier, as the weights of the network were being defined. It goes as follows. It starts, thus, right after the ‘self.input = x’ has been said, ‘self.weights1 = np.random.rand(self.input.shape[1],4)’ fires off, closely followed by ‘self.weights2 = np.random.rand(4,1)’. All in all, the entire class of ‘Neural network’ is defined in the following form:

class NeuralNetwork:

def __init__(self, x, y):

self.input = x

self.weights1 = np.random.rand(self.input.shape[1],4)

self.weights2 = np.random.rand(4,1)

self.y = y

self.output = np.zeros(self.y.shape)

The output of each instance in that neural network is a two-dimensional tuple (table) made of one row (I hope I got it correctly), and four columns. Initially, it is filled with zeros, so as to make room for something more meaningful. The predicted y^-s are supposed to jump into those empty sockets, held ready by the zeros. The ‘random.rand’ expression, associated with ‘weights’ means that the network is supposed to assign randomly different levels of importance to different x-s fed into it.

Anyway, the next step is to instruct my snake (i.e. Python) what to do next, with that class ‘Neural Network’. It is supposed to do two things: feed data forward, i.e. makes those neurons work on predicting the y^-s, and then check itself by an operation called backpropagation of errors. The latter consists in comparing the predicted y^-s with the real y-s, measuring the discrepancy as a loss of information, updating the initial random weights with conclusions from that measurement, and do it all again, and again, and again, until the error runs down to very low values. The weights applied by the network in order to generate that lowest possible error are the best the network can do in terms of learning.

The feeding forward of predicted y^-s goes on in two steps, or in two layers of neurons, one hidden, and one final. They are defined as:

def feedforward(self):

self.layer1 = sigmoid(np.dot(self.input, self.weights1))

self.output = sigmoid(np.dot(self.layer1, self.weights2))

The ‘sigmoid’ part means sigmoid function, AKA logistic function, expressed as y=1/(1+e^-x), where, at the end of the day, the y always falls somewhere between 0 and 1, and the ‘x’ is not really the empirical, real ‘x’, but the ‘x’ multiplied by a weight, ranging between 0 and 1 as well. The sigmoid function is good for testing the weights we apply to various types of input x-es. Whatever kind of data you take: populations measured in millions, or consumption of energy per capita, measured in kilograms of oil equivalent, the basic sigmoid function y=1/(1+e^-x), will always yield a value between 0 and 1. This function essentially normalizes any data.

Now, I want to take differentiated data, like population as headcount, energy consumption in them kilograms of whatever oil equals to, and the supply of money in standardized US dollars. Quite a mix of units and scales of measurement. I label those three as, respectively, x_a, x_b, and x_c. I assign them weights ranging between 0 and 1, so as the sum of weights never exceeds 1. In plain language it means that for every vector of observations made of x_a, x_b, and x_c I take a pinchful of x_a, then a zest of x_b, and a spoon of x_c. I make them into x = w_a*x_a + w_b*x_b + w_c*x_c, I give it a minus sign and put it as an exponent for the Euler’s constant.

That yields y=1/(1+e^-( ^{wa*xa + wb*xb + wc*xc)}). Long, but meaningful to the extent that now, my y is always to find somewhere between 0 and 1, and I can experiment with various weights for my various shades of x, and look what it gives in terms of y.

In the algorithm above, the ‘np.dot’ function conveys the idea of weighing our x-s. With two dimensions, like the input signal ‘x’ and its weight ‘w’, the ‘np.dot’ function yields a multiplication of those two one-dimensional matrices, exactly in the x = w_a*x_a + w_b*x_b + w_c*x_c drift.

Thus, the first really smart layer of the network, the hidden one, takes the empirical x-s, weighs them with random weights, and makes a sigmoid of that. The next layer, the output one, takes the sigmoid-calculated values from the hidden layer, and applies the same operation to them.

One more remark about the sigmoid. You can put something else instead of 1, in the nominator. Then, the sigmoid will yield your data normalized over that something. If you have a process that tends towards a level of saturation, e.g. number of psilocybin parties per month, you can put that level in the nominator. On the top of that, you can add parameters to the denominator. In other words, you can replace the 1+e^-x with ‘b + e^-k*x’, where b and k can be whatever seems to make sense for you. With that specific spin, the sigmoid is good for simulating anything that tends towards saturation over time. Depending on the parameters in denominator, the shape of the corresponding curve will change. Usually, ‘b’ works well when taken as a fraction of the nominator (the saturation level), and the ‘k’ seems to be behaving meaningfully when comprised between 0 and 1.

I return to the algorithm. Now, as the network has generated a set of predicted y^-s, it is time to compare them to the actual y-s, and to evaluate how much is there to learn yet. We can use any measure of error, still, most frequently, them algorithms go after the simplest one, namely the Mean Square Error MSE = [(y₁ – y^₁)² + (y₂ – y^₂)² + … + (y_n – y^_n)²]^0,5. Yes, it is Euclidean distance between the set of actual y-s and that of predicted y^-s. Yes, it is also the standard deviation of predicted y^-s from the actual distribution of empirical y-s.

In this precise algorithm, the author goes down another avenue: he takes the actual differences between observed y-s and predicted y^-s, and then multiplies it by the sigmoid derivative of predicted y^-s. Then he takes the transpose of a uni-dimensional matrix of those (y – y^)*(y^)’ with (y^)’ standing for derivative. It goes like:

def backprop(self):

# application of the chain rule to find derivative of the loss function with respect to weights2 and weights1

d_weights2 = np.dot(self.layer1.T, (2*(self.y – self.output) * sigmoid_derivative(self.output)))

d_weights1 = np.dot(self.input.T, (np.dot(2*(self.y – self.output) * sigmoid_derivative(self.output), self.weights2.T) * sigmoid_derivative(self.layer1)))

# update the weights with the derivative (slope) of the loss function

self.weights1 += d_weights1

self.weights2 += d_weights2

def sigmoid(x):

return 1.0/(1+ np.exp(-x))

def sigmoid_derivative(x):

return x * (1.0 – x)

I am still trying to wrap my mind around the reasons for taking this specific approach to the backpropagation of errors. The derivative of a sigmoid y=1/(1+e^-x) is y’ = [1/(1+e^-x)]*{1 – [1/(1+e^-x)]} and, as any derivative, it measures the slope of change in y. When I do (y₁ – y^₁)*(y^₁)’ + (y₂ – y^₂)*(y^₂)’ + … + (y_n – y^_n)*(y^_n)’ it is as if I were taking some kind of weighted average. That weighted average can be understood in two alternative ways. Either it is standard deviation of y^ from y, weighted with the local slopes, or it is a general slope weighted with local deviations. Now I take the transpose of a matrix like {(y₁ – y^₁)*(y^₁)’ ; (y₂ – y^₂)*(y^₂)’ ; … (y_n – y^_n)*(y^_n)’}, it is a bit as if I made a matrix of inverted terms, i.e. 1/[(y_n – y^_n)*(y^_n)’]. Now, I make a ‘.dot’ product of those inverted terms, so I multiply them by each other. Then, I feed the ‘.dot’ product into the neural network with the ‘+=’ operator. The latter means that in the next round of calculations, the network can do whatever it wants with those terms. Hmmweeellyyeess, makes some sense. I don’t know what exact sense is that, but it has some mathematical charm.

Now, I try to apply the same logic to the data I am working with in my research. Just to give you an idea, I show some data for just one country: Australia. Why Australia? Honestly, I don’t see why it shouldn’t be. Quite a respectable place. Anyway, here is that table. GDP per unit of energy consumed can be considered as the target output variable y, and the rest are those x-s.

Table 1 – Selected data regarding Australia

Year	GDP per unit of energy use (constant 2011 PPP $ per kg of oil equivalent)	Share of aggregate amortization in the GDP	Supply of broad money, % of GDP	Energy use (tons of oil equivalent per capita)	Urban population as % of total population	GDP per capita, ‘000 USD
	y	X₁	X₂	X₃	X₄	X₅
1990	5,662020744	14,46	54,146	5,062	85,4	26,768
1991	5,719765048	14,806	53,369	4,928	85,4	26,496
1992	5,639817305	14,865	56,208	4,959	85,566	27,234
1993	5,597913126	15,277	56,61	5,148	85,748	28,082
1994	5,824685357	15,62	59,227	5,09	85,928	29,295
1995	5,929177604	15,895	60,519	5,129	86,106	30,489
1996	5,780817973	15,431	62,734	5,394	86,283	31,566
1997	5,860645225	15,259	63,981	5,47	86,504	32,709
1998	5,973528571	15,352	65,591	5,554	86,727	33,789
1999	6,139349354	15,086	69,539	5,61	86,947	35,139
2000	6,268129418	14,5	67,72	5,644	87,165	35,35
2001	6,531818805	14,041	70,382	5,447	87,378	36,297
2002	6,563073754	13,609	70,518	5,57	87,541	37,047
2003	6,677186947	13,398	74,818	5,569	87,695	38,302
2004	6,82834791	13,582	77,495	5,598	87,849	39,134
2005	6,99630318	13,737	78,556	5,564	88	39,914
2006	6,908872246	14,116	83,538	5,709	88,15	41,032
2007	6,932137612	14,025	90,679	5,868	88,298	42,022
2008	6,929395465	13,449	97,866	5,965	88,445	42,222
2009	7,039061961	13,698	94,542	5,863	88,59	41,616
2010	7,157467568	12,647	101,042	5,649	88,733	43,155
2011	7,291989544	12,489	100,349	5,638	88,875	43,716
2012	7,671605162	13,071	101,852	5,559	89,015	43,151
2013	7,891026044	13,455	106,347	5,586	89,153	43,238
2014	8,172929207	13,793	109,502	5,485	89,289	43,071

In his article, James Loy reports the cumulative error over 1500 iterations of training, with just four series of x-s, made of four observations. I do something else. I am interested in how the network works, step by step. I do step-by-step calculations with data from that table, following that algorithm I have just discussed. I do it in Excel, and I observe the way that the network behaves. I can see that the hidden layer is really hidden, to the extent that it does not produce much in terms of meaningful information. What really spins is the output layer, thus, in fact, the connection between the hidden layer and the output. In the hidden layer, all the predicted sigmoid y^ are equal to 1, and their derivatives are automatically 0. Still, in the output layer, when the second random distribution of weights overlaps with the first one from the hidden layer. Then, for some years, those output sigmoids demonstrate tiny differences from 1, and their derivatives become very small positive numbers. As a result, tiny, local (y_i – y^_i)*(y^_i)’ expressions are being generated in the output layer, and they modify the initial weights in the next round of training.

I observe the cumulative error (loss) in the first four iterations. In the first one it is 0,003138796, the second round brings 0,000100228, the third round displays 0,0000143, and the fourth one 0,005997739. Looks like an initial reduction of cumulative error, by one order of magnitude at each iteration, and then, in the fourth round, it jumps up to the highest cumulative error of the four. I extend the number to those hand-driven iterations from four to six, and I keep feeding the network with random weights, again and again. A pattern emerges. The cumulative error oscillates. Sometimes the network drives it down, sometimes it swings it up.

F**k! Pardon my French, but just six iterations of that algorithm show me that the thing is really intelligent. It generates an error, it drives it down to a lower value, and then, as if it was somehow dysfunctional to jump to conclusions that quickly, it generates a greater error in consecutive steps, as if it was considering more alternative options. I know that data scientists, should they read this, can slap their thighs at that elderly uncle (i.e. me), fascinated with how a neural network behaves. Still, for me, it is science. I take my data, I feed it into a machine that I see for the first time in my life, and I observe intelligent behaviour in something written on less than one page. It experiments with weights attributed to the stimuli I feed into it, and it evaluates its own error.

Now, I understand why that scientist from MIT, Lex Fridman, says that building artificial intelligence brings insights into how the human brain works.

I am consistently delivering good, almost new science to my readers, and love doing it, and I am working on crowdfunding this activity of mine. As we talk business plans, I remind you that you can download, from the library of my blog, the business plan I prepared for my semi-scientific project Befund (and you can access the French version as well). You can also get a free e-copy of my book ‘Capitalism and Political Power’ You can support my research by donating directly, any amount you consider appropriate, to my PayPal account. You can also consider going to my Patreon page and become my patron. If you decide so, I will be grateful for suggesting me two things that Patreon suggests me to suggest you. Firstly, what kind of reward would you expect in exchange of supporting me? Secondly, what kind of phases would you like to see in the development of my research, and of the corresponding educational tools?

Experimenting with new structures

My editorial

I return to doing my review of literature in order to find accurate distinctions as for historical changes. Besides those articles on the history of technology as such, I found an interesting paper by Bert J.M. de Vries, , Detlef P. van Vuuren, and Monique M. Hoogwijk (de Vries et al. 2007[1]). It is interesting because the authors attempted to assess the potential for producing renewable energies for the first half of the 21^st century, and they did it in 2007, so exactly at the moment when, according to my calculations, something really changed in the sector of renewable energies and it started developing significantly faster than before. It is interesting to see how did other researchers see the future at a moment, which is already the past, and quite a significant bit of past. As usually, I start from the end. The kitchen door I can poke my head through, regarding this paper, is Appendix B, presenting in a concise, tabular form, the so-called four scenarios developed by the authors. So I rummage through those scenarios, and one thing instantaneously jumps to my eye: the absence of the solar-thermal technology. Today, the solar-thermal is probably the most powerful technology for producing renewable energy, yet, when this paper was being written, so not later than in 2007, the solar-thermal technology was still in its infancy. It was precisely in 2007 that the World Bank declassified and published first reports about the big Ouarzazate project in Morocco, and only in 2011 it became more or less clear how to finance those projects. As I am engaged in research on renewable energies, I will probably mention that solar-thermal technology more than once, and for now I am content with noticing that solar-thermal changed a lot in the way we perceive the future of renewable energies.

Returning to this paper by Bert J.M. de Vries, , Detlef P. van Vuuren, and Monique M. Hoogwijk, and to their scenarios, I see four basic technologies being taken into account: wind, solar photovoltaic, biomass electric, and liquid biomass. As a matter of fact, we have another technology, besides the solar-thermal, which strikes by its absence at this precise party: the hydro, in its many shades. Right, probably the authors did not have access to the relevant data. I continue studying this table in appendix B, and what strikes me as odd is that each technology is described with a different set of quantitative variables. It is easy to see that the authors had hard times to drive different technologies to common denominators. As I stop clinging to that Appendix B and browse through the whole paper, I can see a lot of maths but very few numbers. It is hard to say, what was exactly the empirical basis for the scenarios presented. On the whole, this paper by de Vries et al. is interesting as for some methodological insights, but a little disappointing on the whole. I can see nothing that looks like real discovery.

And so I am drifting towards other sources, and I come by an interesting author, David Edgerton and his book ‘Shock of the old: Technology and global history since 1900’ (Edgerton 2011[2]). I am fishing the most interesting bits out of this study, and I find some insights worth stopping by and think. First of all, David Edgerton points out that we commonly live in an illusion of constant technological progress, i.e. of a process, which consistently brings improvement in the conditions of living, as technology changes. Edgerton shows, quite convincingly, that technological change is not necessarily to put at equality with technological progress. According to his findings, there were just a few periods of real technological progress since 1900: between 1900 and 1913, followed by another between 1950 and 1973. Save for those short windows in time, the link between technological change and the conditions of living is really ambiguous.

In Section 4 of his book, entitled ‘Maintenance’, David Edgerton brings forth that intuition that I very much share: that the way we deal with both the physical wear and tear of our technology, and with its moral obsolescence, is deeply significant to our overall well-being due to technology. There is that interesting thing I noticed in Chinese cities, this summer: Chinese people do hardly any renovation in buildings. Save for those occasions when a store is being replaced by another and the facade needs some obvious patching, buildings in China seem to live a life similar to that of clothes: when they are used up, they are being demolished and something new is being built in the same location (they do put a lot of effort into maintaining infrastructure, mind you). In Europe, we are almost obsessively attached to maintaining our buildings in good condition, and we spend a lot on renovation. Two cultures, two completely different roles of maintenance in the construction business. At this point, the research presented by David Edgerton gets strong support in earlier work, by the eminent French historian Fernand Braudel (Braudel 1981[3], 1983[4]) : real technological revolutions happened, in many fields of human ingenuity, when the technology in place allowed providing for more than simple maintenance. Both authors (Edgerton and Braudel) point on a turning point in the history of agriculture, probably by the end of the 19^th century, when the technology of farming allowed to spend relatively less effort on rebuilding the fertile power of the soil.

Another great insight to find with David Edgerton is the importance of killing in technological change. Warfare, hunting, whaling, combatting pathogens, eradicating parasitic species from our agricultural structures – it all means killing, and it apparently has had a tremendous impact on technological change in the 20^th century. This dramatic and deep reference to one of our fundamental functions as organisms (yes, my dear vegans, we all kill someone or something, even without knowing we do) makes me think about this elaborate metaphor of struggling civilisations, to find in Arnold Toynbee’s ‘Study of History’[5]. Development of societies is not an easy game: it is struggle, risk, sometimes stagnation. We, in our civilisation, as it is now, at the beginning of the 21^stcentury, have no grounds to assume we are different than those who struggled before us. Arnold Toynbee coined this metaphor of struggle in an even broader context: how can we distinguish between civilisations? His point was that criteria like race, skin colour or even religious beliefs are misleading in classifying cultures and societies. To him, the real distinction pertained to the state of the struggle, so to say: is there a visible breakthrough in the past of the given social group, a kind of dramatic threshold which made those people take a completely different turn? If the answer is ‘yes’, then we could very well have two different civilisations, before and after that pivotal point in time. If no, we have, in fact, the same cultural paradigm, just dressed differently according to the current fashion.

All the review of literature about the history of technology, and about renewable energies, makes me ask once more that fundamental question: what exactly happened in 2007 – 2008, when the market of renewable energies suddenly moved its lazy ass and started growing much faster than before? Hypothesis #1: it was mostly technological a change, possibly connected to the banalisation of photovoltaic modules, or to the concurrent emergence of the solar-thermal technology. Hypothesis #2, just a shade away from #1 is that we are talking about technological change in the immediate vicinity of renewable sources of energy. Maybe something about electronics, nanotechnology, transport? Here, although the hypothesis is really interesting, I am very much at a loss. I will have to do some book-worming about it. Hypothesis #3: what happened in 2007 – 2008 was mostly social a change. Something shifted in our patterns of being together in society, which made people turn towards renewable energies more than before. That something could pertain to urbanisation, density of population, food deficit, velocity of money and whatnot. I have already done a lot of empirical testing in this respect.

Now, I am shifting my focus slightly, onto my personal experience in innovating, namely in learning the Python. As I have been wrestling with the Python, those last four days, I became aware of a pattern in my behaviour: I start with trying to make the new technology do exactly the same things that the old technology used to do for me, then I progressively become aware that I need to learn new skills in order to get those results (so I have that strange connection: new technology, old expectations regarding the outcomes, new skills to learn), and I also become aware that I need to reshuffle my resources, namely information, so as to squeeze it into new structures. Now, I am advancing one step further and I try to go the other way round: I had a look at what the logical structures of the Python look like and what I am trying to do now is to discover what can possibly the Python do for me. In other words, I am progressively passing from using Python as just a harder-to-tame version of the Excel towards discovering what it can do for me.

I am playing with one of the first things I found in the ‘Tutorial’ textbook, available at docs.python.org, namely with the structure ‘for n in range(x, y):’ and then you specify what do you want the Python to do while n stays in a given range. Here below is a simple try from my part:

>>> for x in range(1, 10):

structure={‘x’:(x**2, x**0,5)}

list(structure)

[‘x’]

# Honestly, I don’t even know why I formed this precise function. It just came like that. I imagined a structure when a cardinal order is established, from 1 to 10, and each point in the order has two coordinates derived from its position: square power and square root. It is nice and it didn’t exactly work as I planned. I got a ten-time repeated [‘x’]. As I put that ‘x’ in single quotes, it has been considered as a word, or a piece of content. Now, I modify slightly my structure: I took off those single quotes. Here is the result:

>>> for x in range(1, 10):

structure={x:{x**2, x**0,5}}

list(structure)

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

#Better a small step forward than no steps at all, as they say. After having removed those single quotes from around the x, I got a cardinal order in response. Still, the function did not yield the coordinates I expected to have out of it. Now, I replace the command ‘list()’ by the command ‘print()’. So far, I did not quite grasp the distinction between those two. I decided to experiment in order to acquire some knowledge, and here comes what I got:

>>> for x in range(1, 10):

structure={x:{x**2, x**0,5}}

print(structure)

{1: {1, 5}}

{2: {1, 4, 5}}

{3: {9, 5, 1}}

{4: {16, 1, 5}}

{5: {25, 5, 1}}

{6: {1, 36, 5}}

{7: {49, 5, 1}}

{8: {64, 1, 5}}

{9: {81, 5, 1}}

#Now, this is most interesting. As I replaced ‘list()’ with ‘print()’, the function finally yielded the coordinates. There is just one tiny little problem: instead of having just two coordinates, in 8 cases on 9, starting from the cardinal 1, I have three coordinates instead of two. Why? Oh, I see. I put ‘0,5’ instead of ‘0.5’. For the Python, decimal values are accepted as such if the decimal point is literally the point, not the coma. The expression ‘x**0,5’, which I meant to be x power 0,5, has been read by the Python as x power 0, followed by a 5. OK, I understand. My mistake. I replace the ‘0,5’ by ‘0.5’, by the book, and I launch it again.

>>> for x in range(1, 10):

structure={x:{x**2, x**0.5}}

print(structure)

{1: {1}}

{2: {1.4142135623730951, 4}}

{3: {9, 1.7320508075688772}}

{4: {16, 2.0}}

{5: {25, 2.23606797749979}}

{6: {2.449489742783178, 36}}

{7: {49, 2.6457513110645907}}

{8: {64, 2.8284271247461903}}

{9: {81, 3.0}}

#Finally, it worked. Now, I push my experiment further and I introduce a third coordinate, the natural logarithm of the cardinal position.

>>> import math #I turn on the module with mathematical functions

>>> for x in range(1, 10):

structure={x:{x**2, x**0.5, math.log(x)}}

print(structure)

{1: {0.0, 1}}

{2: {0.6931471805599453, 1.4142135623730951, 4}}

{3: {9, 1.0986122886681098, 1.7320508075688772}}

{4: {16, 1.3862943611198906, 2.0}}

{5: {25, 2.23606797749979, 1.6094379124341003}}

{6: {1.791759469228055, 2.449489742783178, 36}}

{7: {49, 2.6457513110645907, 1.9459101490553132}}

{8: {64, 2.8284271247461903, 2.0794415416798357}}

{9: {81, 2.1972245773362196, 3.0}}

Good. It seems to work. What I have just succeeded to do is to learn a typical, very simple structure in Python and this structure does something slightly different than the Excel: it generates a sequence of specific structures out of a general, logical structure I specified. I am wrapping up my learning from my learning: it took me four days to start looking actively for new possibilities offered by the Python. If I extrapolate to the scale of collective behaviour, we have those four patterns of innovating: a) trying to obtain old outcomes with the new technology b) making mistakes, seeing my efficiency plummeting, and learning new skills c) rearranging my resources for the new technology d) experimenting and exploring the new possibilities, which come with the new technology. As I refer this generalized account of my individual experience to the literature I quoted a few paragraphs earlier. How does a breakthrough occur in these specific patterns of behaviour? I can assume there is a critical amount of learning and adaptation, required in the presence of a new technology, which can possibly, referring once more to Arnold Toynbee’s metaphor of struggling civilisations, make a technological transition risky, impossible or null in its balanced outcomes.

[1] de Vries, Bert J.M., van Vuuren, Detlef P., Hoogwijk Monique M., 2007, Renewable energy sources: Their global potential for the first-half of the 21st century at a global level: An integrated approach, Energy Policy, vol. 35 (2007), pp. 2590–2610

[2] Edgerton, D. (2011). Shock of the old: Technology and global history since 1900. Profile books

[3] Braudel, F., 1981, Civilization and Capitalism, Vol. I: The Structures of Everyday Life, rev.ed., English Translation, William Collins Sons & Co London and Harper & Row New York, ISBN 00216303 9

[4] Braudel, F., 1983, Civilisation and Capitalism. Part II: The Wheels of Commerce, trans. Sian Reynolds, Book Club Associates, William Collins Sons & Co,

[5] Toynbee, J. Arnold. Study of history. University press, 1946, pp. 69

Je réorganise mes ressources pour le Python

Mon éditorial

Me voilà qui continue cette expérience étrange et rafraîchissante de faire de la recherche sur le changement technologique en même temps que je fais du changement technologique en moi-même, c’est-à-dire en apprenant le Python, une langue de programmation à la mode dans les endroits fréquentés par des types comme moi. Ces endroits sont des universités, des conférences, des bibliothèques etc. Vous voyez le genre. Durant les deux derniers jours j’ai déjà découvert qu’en appréhendant cette technologie nouvelle (nouvelle pour moi, je veux dire), j’avais commencé, intuitivement, par utiliser le Python pour qu’il fasse exactement la même chose que fait Excel ou mon logiciel d’analyse statistique, Wizard for MacOS. Je m’étais donc appliqué à bâtir une base de données comme je la vois, donc comme une table. En Python, les données sont organisées comme structures logiques et non pas graphiques, donc mes efforts avaient été vains dans une large mesure. Je commettais beaucoup d’erreurs, parfois stupides. J’avais besoin d’un nombre surprenant d’essais pour acquérir une capacité acceptablement intuitive de distinguer entre les structures logiques validées et celles qui ne le sont pas. Ça m’avait aussi pris un temps déconcertement long pour faire la distinction entre des structures logiques avec des mots et des symboles dedans – ces soi-disant « strings » – et les structures qui contiennent des nombres. Pour les non-initiés, une série de noms en Excel, ça se présente comme une colonne ou bien un vers, donc comme :

Mot#1

Mot#2

Mot#3

…ou bien comme

Mot#1

Mot#2

Mot#3

…tandis qu’en Python ce serait plutôt :

>>> Mots=[‘Mot#1’, ‘Mot#2’, ‘Mot#3’]

… donc chaque mot entre des apostrophes (qui sont des marques de citation de base en anglais), les mots séparés par des virgules et tout ça compris entre des parenthèses carrées. C’est précisement ce qu’on appelle un « string ».

Si je veux organiser des valeurs numériques, la façon que j’ai déjà apprise pour les organiser est une série entre des parenthèses rondes :

>>> Nombres=(23, 34, 45, 56)

Comme vous pouvez le constater, le Python requiert un ordre logique : valeur logique ou numérique en position no. 1, ensuite celle en position no. 2 et ainsi de suite.

Bref, en un premier temps, j’avais mis beaucoup d’effort pour insérer de force cette nouvelle technologie dans une vieille structure de mes habitudes, et ce n’est qu’ensuite que j’ai commencé à apprendre des nouvelles habitudes. Tout ça se reflétait dans une efficacité tout ce qu’il y a de plus lamentable. J’avançais à un pas d’escargot. Oui, je sais, un escargot, ça n’a pas de pattes, donc ça ne peut pas faire de pas, mais « pas d’escargot » ça sonne bien.

Hier soir, tout en m’entraînant à développer une compréhension aussi intuitive que possible de ces structures logiques de Python, j’avais commencé à faire autre chose : engranger des données organisées façon Python. Les tables que je télécharge du site de la Banque Mondiale sont organisées en pays et années d’observation, pays en colonne, années dans un vers en haut. Je sens intuitivement qu’une fois que je crée un string des pays et je le sauve dans un fichier Python à part, il me servira plusieurs fois pour apprendre des nouveaux trucs ou tout simplement pour faire des analyses plus tard, lorsque j’aurai appris comment utiliser se string des pays. Avec deux tests rapides pour la grammaire et avec deux petits réajustements, ça m’a pris 26 minutes de créer le fichier « countries_WB.py », qui contient un string nommé « countries », qui, à son tour, contient la même liste des pays et régions, en anglais, que vous pouvez trouver dans les tables Excel de la Banque Mondiale.

Ce que je viens de faire consiste à réorganiser mes ressources de façon à les rendre plus accessibles à la nouvelle technologie que j’apprends. Pour le moment, je l’avais fait manuellement, en copiant la table des pays en Excel dans mon éditeur Word, ensuite en la convertissant en texte, copiant ce texte dans le compilateur Python et organisant ces noms des pays en un string « countries=[‘Pays#1’, ‘Pays#2’, etc.]. Je sais, je sais, je pourrais télécharger directement, du site de la Banque Mondiale, un fichier en format CSV et le déballer avec les commandes Python du module « csv ». Oui, j’aurai pu le faire si je savais exactement comment le faire et ça, c’est encore de l’avenir pour moi. Je faisais donc face à un choix : organiser mes ressources pour la nouvelle technologie en une manière bien grossière et improductive ou bien attendre que j’apprenne une façon plus rapide et élégante d’accomplir la même tâche. J’avais choisi la première option.

Voilà donc que je peux généraliser mes observations au sujet de mon propre comportement dans cette expérience sur moi-même. En présence d’une technologie nouvelle, j’ai développé comme trois modèles de comportement :

Modèle #1 : j’essaie d’utiliser la nouvelle technologie (Python) pour accomplir les mêmes résultats que j’accomplissais avec la technologie précédente (Excel, Wizard). Mon choix consiste surtout à décider combien de temps et d’énergie je vais consacrer à essayer d’appliquer la logique ancienne à la technologie nouvelle.

Modèle #2 : Je me suis rendu compte que les vieilles habitudes ne marchent pas et je commence à en apprendre des nouvelles. Mon choix se fait à deux niveaux. Premièrement, je fais des petites décisions instantanées à propos du prochain pas d’apprentissage à faire. Sur un deuxième niveau, je décide combien de temps et d’énergie consacrer à l’apprentissage des compétences requises par la technologie nouvelle. C’est un choix fait en vue de bénéfices futurs incertains.

Modèle #3 : Je sais que l’une des compétences de base dans l’univers de Python consiste à organiser mes ressources de base – l’information – en des structures spécifiques. Je décide de faire un premier pas dans cette direction et je consacre environ une demi-heure à créer un string des pays et régions, exactement conforme à l’ordre utilisé dans les données de la Banque Mondiale. Je me souviens d’avoir fait deux choix distincts : est-ce que je m’y prends du tout et quel type d’information organiser en premier. J’avais pris ces deux décision en sachant que j’utilise une technique qui est loin d’être optimale dans l’univers de cette nouvelle technologie, ainsi que je savais que je ne sais pas du tout comment je vais utiliser ce string des pays dans l’avenir (je n’y suis pas encore arrivé dans mon apprentissage de Python).

A propos, juste pour que vous ayez du solide après avoir lu cette mise à jour : en-dessous du texte, j’ai copié ce string des pays exactement comme je l’avais écrit manuellement. Comme ça se présente maintenant, ça devrait être directement utilisable en Python. Vous pouvez le copier et mettre dans un programme. Côté syntaxe, c’est correct : je l’avais testé avec la commande « list(countries) » et Python avait rendu une liste à partir du string sans gueuler « erreur ! ». A cette occasion, j’ai appris que le dépositoire des fichiers sur mon site https://discoversocialsciences.com , dans l’environnement Word Press, n’accepte pas des fichiers en Python. Je voulais y télécharger et stocker le fichier « countries_WB.py » et le site a rendu erreur. Une petite leçon à propos de la compatibilité des technologies.

Maintenant, j’applique ces nouvelles connaissances, tout ce qu’il y a de plus empirique, pour faire une généralisation théorique. Si nous considérons deux technologies, une ancienne TC₀ et une nouvelle TC₁, la transition de TC₀ à TC₁ est liée, entre autres, à trois phénomènes : une substitution fonctionnelle complexe, un coût d’apprentissage CA(TC₀ ; TC₁) et un coût de réorganisation de ressources CR(TC₀ ; TC₁). Le premier phénomène, celui de substitution complexe, peut être représenté comme une relation entre deux ensembles. Il y a un ensemble F(TC₀) = {f₁, f₂, …, f_i} des fonctions remplies par la technologie TC₀, et par analogie, je définis un ensemble F(TC₁) = {g₁, g₂, …, g_k} des fonctions remplies par la technologie TC₁. Maintenant, avant que je passe plus loin, un petit mon d’explication à propos de la présentation : j’écris ce contenu-ci en sachant que je vais le copier dans mes deux blogs, celui en Word Press à https://discoversocialsciences.com , ainsi que celui dans l’environnement Blogger, à l’addresse https://researchsocialsci.blogspot.com . Tous les deux ne sont pas vraiment ami-ami avec l’éditeur d’équation de MS Word, donc j’écris les équations avec les symboles du clavier. Je n’ai pu trouver aucun clavier qui permet de taper directement les opérateurs mathématiques, y compris les opérateurs d’ensembles. J’adopte donc une convention simplifiée où les symboles +, -, * et / correspondent, respectivement, à la somme, différence, produit et quotient des ensembles.

La relation de substitution complexe entre technologies veut dire, qu’avec un niveau donné de compétences de la part d’utilisateurs, les ensembles F(TC₀) = {f₁, f₂, …, f_i} et F(TC₁) = {g₁, g₂, …, g_k} ont une partie commune, ou F(TC₀)*F(TC₁), qui correspond aux fonctions remplies par les deux technologies. Je définis le degré de substitution entre les deux technologies comme le quotient complexe : SU(TC₀ ; TC₁) = [F(TC₀)*F(TC₁)] / [ F(TC₀) + F(TC₁)]. Je pose formellement l’hypothèse que le coût d’apprentissage CA(TC₀ ; TC₁) ainsi que le un coût de réorganisation de ressources CR(TC₀ ; TC₁) sont tous les deux inversement proportionnels à la valeur du quotient complexe SU(TC₀ ; TC₁) ou :

CA(TC₀ ; TC₁) = a1* SU(TC₀ ; TC₁)

CR(TC₀ ; TC₁) = a2* SU(TC₀ ; TC₁)

a1 > 0 ; a2 > 0

Je me dis, quand je regarde cette structure logique, qu’elle risque d’être un peu lourde avec beaucoup de technologies qui se substituent d’une façon complexe et avec beaucoup d’attributs fonctionnels. Je vois donc une façon alternative de représenter la même proposition, avec l’aide de la distance Euclidienne. Vous savez, ce truc basé sur le théorème de Pythagore : si on a deux points A et B, chacun défini par deux coordonnées x et y, on peut calculer la distance entre ces deux points comme d = ((x(A) – x(B))² + (y(A) – y(B))²)^0,5 . Maintenant, je remplace les points A et B par mes technologies TC0 et TC1, TC_{ce que vous voulez, par ailleurs}, et je dote chacune avec deux attributs mesurables x et y. Je peux alors calculer la distance Euclidienne « d » dans une paire donnée de technologies. Comme je suis toujours conscient que je devrais apprendre le Python, voilà, ci-dessous, je présente le fruit de trois jours d’apprentissage : un petit calculateur de distance Euclidienne en Python 3.6.2 :

# je commence par définir les coordonnées des trois technologies TC0, TC1 et TC2

>>> TC0=(12, 45)

>>> TC1=(34, 15)

>>> TC2=(17, 30)

>>> import math #j’importe le module des fonctions mathématiques, juste pour me faciliter la tâche

# je définis mon calculateur pour la première paire de technologies

>>> for n in range(0, len(TC0)):

for m in range(0, len(TC1)):

print(math.sqrt((math.pow(34-12, 2)+math.pow(15-45, 2))))

37.20215047547655

# je n’ai pas la moindre idée pourquoi le compilateur a affiché le même résultat quatre fois ; voilà un défi potentiel

#je répète avec l’autre paire de technologies

>>> for n in range(0, len(TC0)):

for m in range(0, len(TC2)):

print(math.sqrt((math.pow(17 – 12, 2) + math.pow(30 – 15, 2))))

15.811388300841896

#voilà encore une fois le même résultat quatre fois ; amusant

Bon, je commence à ressentir de la fatigue intellectuelle. Temps de terminer pour aujourd’hui. A bientôt.

Comme promis, voilà le string des pays pour Python, conforme à la structure utilisée par la Banque Mondiale :

countries=[‘Aruba’,‘Afghanistan’,‘Angola’, ‘Albania’, ‘Andorra’, ‘Arab World’, ‘United Arab Emirates’, ‘Argentina’, ‘Armenia’, ‘American Samoa’, ‘Antigua and Barbuda’, ‘Australia’, ‘Austria’, ‘Azerbaijan’, ‘Burundi’, ‘Belgium’, ‘Benin’, ‘Burkina Faso’, ‘Bangladesh’, ‘Bulgaria’, ‘Bahrain’, ‘Bahamas The’, ‘Bosnia and Herzegovina’, ‘Belarus’, ‘Belize’, ‘Bermuda’, ‘Bolivia’, ‘Brazil’, ‘Barbados’, ‘Brunei Darussalam’, ‘Bhutan’, ‘Botswana’, ‘Central African Republic’, ‘Canada’, ‘Central Europe and the Baltics’, ‘Switzerland’, ‘Channel Islands’, ‘Chile’, ‘China’, ‘Cote d_Ivoire’, ‘Cameroon’, ‘Congo, Dem. Rep.’, ‘Congo, Rep.’, ‘Colombia’, ‘Comoros’, ‘Cabo Verde’, ‘Costa Rica’, ‘Caribbean small states’, ‘Cuba’,’Curacao’, ‘Cayman Islands’, ‘Cyprus’, ‘Czech Republic’, ‘Germany’, ‘Djibouti’, ‘Dominica’, ‘Denmark’, ‘Dominican Republic’, ‘Algeria’, ‘East Asia & Pacific (excluding high income)’, ‘Early-demographic dividend’, ‘East Asia & Pacific’, ‘Europe & Central Asia (excluding high income)’, ‘Europe & Central Asia’, ‘Ecuador’, ‘Egypt, Arab Rep.’, ‘Euro area’, ‘Eritrea’, ‘Spain’, ‘Estonia’, ‘Ethiopia’, ‘European Union’, ‘Fragile and conflict affected situations’, ‘Finland’, ‘Fiji’, ‘France’, ‘Faroe Islands’, ‘Micronesia, Fed. Sts.’, ‘Gabon’, ‘United Kingdom’, ‘Georgia’, ‘Ghana’, ‘Gibraltar’, ‘Guinea’, ‘Gambia, The’, ‘Guinea-Bissau’, ‘Equatorial Guinea’, ‘Greece’, ‘Grenada’, ‘Greenland’, ‘Guatemala’, ‘Guam’, ‘Guyana’, ‘High income’, ‘Hong Kong SAR, China’, ‘Honduras’, ‘Heavily indebted poor countries (HIPC)’, ‘Croatia’, ‘Haiti’, ‘Hungary’, ‘IBRD only’, ‘IDA & IBRD total’, ‘IDA total’, ‘IDA blend’, ‘Indonesia’, ‘IDA only’, ‘Isle of Man’, ‘India’, ‘Not classified’, ‘Ireland’, ‘Iran, Islamic Rep.’, ‘Iraq’, ‘Iceland’, ‘Israel’, ‘Italy’, ‘Jamaica’, ‘Jordan’, ‘Japan’, ‘Kazakhstan’, ‘Kenya’, ‘Kyrgyz Republic’, ‘Cambodia’, ‘Kiribati’, ‘St. Kitts and Nevis’, ‘Korea, Rep.’, ‘Kuwait’, ‘Latin America & Caribbean (excluding high income)’, ‘Lao PDR’, ‘Lebanon’, ‘Liberia’, ‘Libya’, ‘St. Lucia’, ‘Latin America & Caribbean’, ‘Least developed countries: UN classification’, ‘Low income’, ‘Liechtenstein’, ‘Sri Lanka’, ‘Lower middle income’, ‘Low & middle income’, ‘Lesotho’, ‘Late-demographic dividend’, ‘Lithuania’, ‘Luxembourg’, ‘Latvia’, ‘Macao SAR, China’, ‘St. Martin (French part)’, ‘Morocco’, ‘Monaco’, ‘Moldova’, ‘Madagascar’, ‘Maldives’, ‘Middle East & North Africa’, ‘Mexico’, ‘Marshall Islands’, ‘Middle income’, ‘Macedonia, FYR’, ‘Mali’, ‘Malta’, ‘Myanmar’, ‘Middle East & North Africa (excluding high income)’, ‘Montenegro’, ‘Mongolia’, ‘Northern Mariana Islands’, ‘Mozambique’, ‘Mauritania’, ‘Mauritius’, ‘Malawi’, ‘Malaysia’, ‘North America’, ‘Namibia’, ‘New Caledonia’, ‘Niger’, ‘Nigeria’, ‘Nicaragua’, ‘Netherlands’, ‘Norway’, ‘Nepal’, ‘Nauru’, ‘New Zealand’, ‘OECD members’, ‘Oman’, ‘Other small states’, ‘Pakistan’, ‘Panama’, ‘Peru’, ‘Philippines’, ‘Palau’, ‘Papua New Guinea’, ‘Poland’, ‘Pre-demographic dividend’, ‘Puerto Rico’, ‘Korea, Dem. People’s Rep.’, ‘Portugal’, ‘Paraguay’, ‘West Bank and Gaza’, ‘Pacific island small states’, ‘Post-demographic dividend’, ‘French Polynesia’, ‘Qatar’, ‘Romania’, ‘Russian Federation’, ‘Rwanda’, ‘South Asia’, ‘Saudi Arabia’, ‘Sudan’, ‘Senegal’, ‘Singapore’, ‘Solomon Islands’, ‘Sierra Leone’, ‘El Salvador’, ‘San Marino’, ‘Somalia’, ‘Serbia’, ‘Sub-Saharan Africa (excluding high income)’, ‘South Sudan’, ‘Sub-Saharan Africa’, ‘Small states’, ‘Sao Tome and Principe’, ‘Suriname’, ‘Slovak Republic’, ‘Slovenia’, ‘Sweden’, ‘Swaziland’, ‘Sint Maarten (Dutch part)’, ‘Seychelles’, ‘Syrian Arab Republic’, ‘Turks and Caicos Islands’, ‘Chad’, ‘East Asia & Pacific (IDA & IBRD countries)’, ‘Europe & Central Asia (IDA & IBRD countries)’, ‘Togo’, ‘Thailand’, ‘Tajikistan’, ‘Turkmenistan’, ‘Latin America & the Caribbean (IDA & IBRD countries)’, ‘Timor-Leste’, ‘Middle East & North Africa (IDA & IBRD countries)’, ‘Tonga’, ‘South Asia (IDA & IBRD)’, ‘Sub-Saharan Africa (IDA & IBRD countries)’, ‘Trinidad and Tobago’, ‘Tunisia’, ‘Turkey’, ‘Tuvalu’, ‘Tanzania’, ‘Uganda’, ‘Ukraine’, ‘Upper middle income’, ‘Uruguay’, ‘United States’, ‘Uzbekistan’, ‘St. Vincent and the Grenadines’, ‘Venezuela, RB’, ‘British Virgin Islands’, ‘Virgin Islands (U.S.)’, ‘Vietnam’, ‘Vanuatu’, ‘World’, ‘Samoa’, ‘Kosovo’, ‘Yemen, Rep.’, ‘South Africa’, ‘Zambia’, ‘Zimbabwe’]

Theorizing and learning by doing, whilst wrestling with Python

My editorial

And so I am experimenting with my thinking. I am writing about choices we make regarding new technologies whilst placing myself in a situation of absorbing a new technology: The Python. I am talking about that programming language, reputed for being particularly useful in studying quantitative data. I thought it would be interesting to write about people’s choices whilst making a choice about new technology. Just to be precise: the variety of Python I am enduring my pains of neophyte with is Python 3.6.2, and besides using the materials provided at the standard library of the Python I am using a book by Toby Segaran, entitled ‘Programming Collective Intelligence’, Published by O’Reilly Media, Inc. (ISBN-10: 0-596-52932-5; ISBN-13: 978-0-596-52932-1). I want to discover a few things. Firstly, I want to rediscover the process of discovering something new. It has been a long time since I really had to absorb some brand new set of skills and understandings. I recently became aware of that: I am writing about innovation and technological change, about whole communities switching to renewable energies but with all that, I am not absorbing any true novelty. Secondly, data science becomes kind of adjacent a discipline for social sciences, and I wanted to discover what can I get more out of Python that my statistical software (i.e. Wizard for MacOS) does not provide. So I want to discover new possibilities, and I want to rediscover what is it to discover new possibilities.

I really started this experiment yesterday. As I am recollecting what I have been doing, yesterday and today, a few patterns emerge. As I mentioned it, I started under a twofold impulse: an idea connected to my research and writing, for one, and sort of a vague peer pressure, for two. Being my own guinea pig, I assume I am somehow representative, and so I peg down those two motivations: curiosity awoken by current activity, and peer pressure, also connected to the present activity. As a matter of fact, I started with implementing Python in my current activity, i.e. in research. I tried to use the basic notations for expressing deterministic choice (see ‘Qu’est-ce que ça fait d’être mon propre étudiant, avec Python’). Then, I used Python to meddle with quantitative data, the one in the CSV format. I played with building strings and sets of values out of that format. I am progressively getting acquainted with the distinction between names, defined in single quotes (i.e. ‘x’), and values without quotation marks. Anyway, I took something I already do with the tools I already have, and I am trying to do it with Python. It is awkward, it is slow, it involves a lot of trial and error. I have to solve new problems. I am confronted with something apparently smart (the Python), which returns dull, preformatted answers in the case of failed attempts from my part. Tons of fun, in other words. Here, I have a second pattern, important for understanding innovation: every time we absorb a new technology, we kind of insert it into our current life and we keenly observe what changes under the impulse. That change involves an initial drop in efficiency. What I am trying to do with Python now, I do much faster and more artfully with my Excel and my Wizard combined.

The drop in efficiency I am experiencing is very much connected to the logic I have to use, or, in other words, to the structure of information. In Excel, I have tabularized structures. In Wizard, I almost don’t pay attention to the raw data, previously imported from Excel, and I focus on analysis, which involves picking up, by clicking, from pre-formatted lists. Anyway, both in Excel and in Wizard, I have a two-dimensional structure, where I move mostly graphically, with a mouse or a touchpad. In Python, I have logical structures akin to typical writing. I have to type all those structures by myself, or copy from Excel and format into the written way. I am becoming aware, I mean really aware that what I used to perceive as adjacent columns or rows of data are, in fact, separate logical structures linked by some kind of function. I encounter two types of challenges here. The most elementary one is the shift from reversible to irreversible. In Python, in order to have any logical structure usable in subsequent operations, like a string of names or a set of values, I have to validate it with an ‘Enter’. Still, after I have validated it, I cannot alter it. If I made a mistake, I have to define a new structure, almost identical to the previous one, save for those corrected mistakes, and to replace it under the same name I gave to that previous one. Defining functions that link different logical structures is much harder for me, at least for the moment.

As I am wrestling with the Python, I recall my early experience with software like DBASE or Lotus, back in the early 1990ies. Those pieces of software were something between the Python and the modern software base on graphical interfacing. I am reconstructing the path of learning I have covered and I am forming another pattern regarding innovation. From DBASE or Lotus, the path has been towards simplification and greater a comfort on the user’s side, whilst creating an increasing complexity on the programming side. What I am doing now, as I am learning Python, is to deconstruct that comfort and simplicity and see what I can get if I rethink and relearn the technology of comfort and simplicity. Just to give you an idea of what I am working with right now (the initiated to Python, I implore magnanimity from your part), I give the example of what I have written kind of on the side, whilst writing what I am writing for the blog. Here it is:

# I want to define a set of people, a set of technologies, and to illustrate the fact of preferring some technologies more than others. So I define my set of ‘people’, as a string:

>>> people=[‘Geogre’, ‘Chris’, ‘Eleonore’, ‘Kathy’, ‘Miron’, ‘Eva’]

# S***, I made a ‘Geogre’ out of ‘George’; I have to define people again

>>> people=[‘George’, ‘Chris’, ‘Eleonore’, ‘Kathy’, ‘Miron’, ‘Eva’]

>>> technologies=[‘photovoltaic on the roof’, ‘own water turbine’, ‘connection to smart grid’, ‘cycling more driving less’]

>>> scores=(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

# Now, what I want is to connect somehow the scores to the people and to the technologies. I want to figure out some kind of function, which would put those three categories together. Here below you can see a failed attempt:

>>> for n in people and for m in technologies return(scores [3])

SyntaxError: invalid syntax << # What did I tell you?

#Now, I am less ambitious. I am trying to create a logical structure from just two categories: people and technologies. I just want to put them in pairs, and label that matching ‘experiments with’. Here is what I did pretty intuitively:

>> for n in range(0, len(people)):

for m in range(0, len(technologies)):

print(people [n], ‘experiments with’, technologies [n])

# It did not yield ‘SyntaxError’, which is already a plus. After an additional ‘Enter’, I had this:

George experiments with photovoltaic on the roof

Chris experiments with own water turbine

Eleonore experiments with connection to smart grid

Kathy experiments with cycling more driving less

# Oouff, it worked, for a change.

Simple lesson: coming up with structures that work takes experimentation. Experimentation takes time and energy, and still, in the same time, yields learning. Experimenting with new technologies is a trade-off between the time and energy devoted to experimentation, on the one hand, and the learning we have from it, on the other hand. The next step I am trying to achieve, unsuccessfully as for now, is to connect people and technologies to those numerical values from ‘scores’. Here below, you can find a short account of my failures:

>>> list(scores)

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

>>> scores=[list(scores)]

>>> scores [3]

Traceback (most recent call last):

File “<pyshell#12>”, line 1, in <module>

scores [3]

IndexError: list index out of range

>>> scores[2]

Traceback (most recent call last):

File “<pyshell#13>”, line 1, in <module>

scores[2]

IndexError: list index out of range

What did I do wrong? Where is the loophole in my thinking? I think I know: what I have been trying to do was to kind of pick up arbitrarily a value from the set of ‘scores’ and connect it somehow to people and technologies. I have been trying to do what I commonly do with Excel: I just sort of tried to match it, as if I were making a table. I was trying to work with the new technology (i.e. the Python) just as I am used to work with the old technology, namely Excel. It does not work. New technology requires new skills and habits. There is that thing about logical structures: the numerical ones are different from the strictly spoken logical ones. Numbers are of different nature than words and letters. If I want to express the preferences of my ‘people’ for particular ‘technologies’, I have to construct a function of preference, i.e. a function which yields a numerical result through a mathematically valid process. At that point, I am becoming aware that I have two alternative logics of constructing such a function: through positioning or through inference.

The first technique consists in ascribing, to my ‘people’ and my ‘technologies’, some kind of coordinates in a virtual space. It is very similar to the Bayesian rectangle: I represent phenomena as a geometrical structure. This structure has a frontier delimiting its expanse, and, possibly, some additional secants, which serve to fine-tune the position of a given point in this space. Positioning looks like the right way to construct a function of preference when position matters. Position is distance of the given point (person, technology etc.) from (to?) some peg points fixed in the given space. In that book by Toby Segaran, I found an interesting reminder about the application of the Pythagorean theorem when it comes to distance in an imaginary space: a² + b² = c². If you have two points, and you ascribe two coordinates to each (actually, this would be a plan, not a space, but who cares?), you can measure the distance between those two points on each coordinate separately, as a difference, then square up those differences, add them up, and take the square root of the so-calculated sum. Summing up, the technique of positioning allows ascribing to people preferences, which result from their relative distance from your peg points or peg axes: conservatism, openness, liberalism etc. The technique of inference assumes that if I have some characteristics X, Y, Z, my preferences are the most likely to be A, B, C and usually those As, Bs, and Cs are the expected values in a statistical distribution. If I am Polish and my friend is French, and France has greater a share of renewable energies in their final consumption of energy than Poland, my Friend is supposed to have stronger a preference for putting photovoltaic panels on his roof, or something in those lines.

I am wrapping up this account of my personal experience with innovation. I use the new technology, in the first place, to reach the same outcomes I commonly achieve with the old technology. I basically squeeze that new technology into the social role I have now. When a lot of people do the same, it means that the new technology, in a first step of collective learning, adapts to the existing social structure. Then, I intuitively use old habits and skills with the new technology, which causes problems, and diminishes my efficiency, and still it is lots of fun, and it makes me learn something new. Once again, as I translate this personal experience into collective patterns of behaviour, the greater the relative distance between the new technology and the old one, the more trial and error, the more learning required, thus the deeper is the trough in efficiency, and the greater is the leap caused by learning for the new technology. Relating it to the old concept of technological determinism by Karl Marx, I can see the weakness of his thinking. We assumed that the greater the economic power of a technology, so the greater possible technical improvements it can bring, the greater power it has to bend the social structure around it. Still, my little experiment brings a new variable: the greater the transformative power of a new technology, the more learning it requires and the less we can predict about how the hell it will change the social structure around it.

Tag: Python

I got it all wrong

Like this:

Boots on the ground

Like this:

Mathematical distance

Like this:

Pardon my French, but the thing is really intelligent

My editorial on You Tube

Like this:

Experimenting with new structures

My editorial

Like this:

Je réorganise mes ressources pour le Python

Mon éditorial

Like this:

Theorizing and learning by doing, whilst wrestling with Python

My editorial

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this: