Philosophical Foundations of AI, Part II: Semantics and Pragmatics, (Social) Reality and Representation

Logic, meaning, and the social and ethical side of things

In Part II of this series (see Part I here), we move from the biblical foundations of language and logic to more of the human, social side of things. You might say we are moving from the realm of syntax to semantics and pragmatics.

Syntax defines what ‘counts’ as an expression in that language — its grammar, in other words. By semantics, I mean the way in which the expressions we deemed as ‘grammatical’ in our language ‘map’ to the world ‘out there’ (see the distinction between sense and reference in Part I if needed). Finally, pragmatics studies how speakers of a given language relate to and use the signs in that language. Where do these signs come from? How or why did we choose this sign to represent that object, for instance?

This post will also explore other functions of language, besides its use a medium for representation. I argue, on the basis of ideas from Soviet literary theorist Mikhail Bakhtin and French philosopher Emmanuel Levinas, that language can shape our ethical commitments towards others. Further, we’ll look at how language, as a system of signs, relates it to other forms of signification used in the natural sciences.

Towards the end I present a philosophical view that claims representation is merely denotation, and briefly critique the way in which we typically interpret the performance of machine learning models. Lastly, I touch on some ethical issues related to behavioral big data (BBD). I try to explain how referring to persons using numbers can lead us down a slippery slope towards barbarism.

I call you ‘baby Travis.” Photo by Alex Hockett on Unsplash

Persons and Names, Semantics and Pragmatics

Human beings have names, which they are called. Many languages with Indo-European origins make explicit use of this in their grammatical structures: me llamo Travis, je m’appelle Travis, ich heisse Travis, and so on. We are called so that we may be addressed (as, say, du or Sie; tu or usted, etc.) and respond toother humans in communication. Note, too, how languages often reserve a distinction between knowing persons and knowing general facts (roughly: knowledge by acquaintance vs. description). For example, conocer/saber in Spanish, kennen/wissen in German, 認識/知道 in Chinese and รู้จัก/รู้ in Thai. So this is not strictly a “Western” social phenomenon.

The idea is that ethical commitments derive from dialogical and social ones. It’s not a coincidence that in Chinese and Thai, for instance, honorific pronouns are family terms such as “uncle,” “aunt,” “big brother,” etc., implying that filial piety/responsibility goes along with such titles. We don’t just respond to those who call out for us, we have a responsibility to them.

The notion of ethical responsibility is derived from one’s ability to communicate dialogically with others — to be called on, and respond to, others. This suggests there are communicative norms guiding our dialogical behavior (see the work of German philosopher Jürgen Habermas for details).

At this level, we have gone beyond semantics to the level of pragmatics. We have moved from an analysis of how words relate to things in the world, to now an analysis of how humans use such linguistic expressions in their dealings with one another as they create social and cultural structures. What is fascinating and impressive is the way in which these philosophers have used this metaphysical analysis of self and other, identity and difference, to ground an ethics of dialogue.

Wittgenstein might argue AI will never capture meaning because it has no access to our dynamic, shared forms of life which ground the meaning of our behaviors and words we use in language games. Photo by Daniel Lonn on Unsplash

From Language to Ethics

Here’s how we go from language to ethics. Recognition of identity is a presupposition of ethical responsibility. You cannot respond to what you cannot first identify. You cannot re-cognize what you cannot first cognize. We can now understand why the recognition of identity is so crucial to modern political discourse, especially as relates to “identity politics.”

The failure of some modern computer vision systems to recognize the faces of minorities is a symptom of a lack of responsibility, a lack of recognition of identity, towards people that look different from those who have designed facial recognition systems. In our modern, democratic society, marginalized groups and individuals implicitly call out for recognition of similarity — as ultimately belonging to one and the same society of equals — but the engineers of such systems do not respond. Unable to engage in dialogue, to share in perspective and understanding, the engineers behind facial recognition systems assume no responsibility towards those viewed as different. The result is a false-positive rate up to 100X higher for Asian and Black people, as the article linked above describes.

What responsibility do you have towards that which is un-identifiable and thus un-namable? What would you call or how would you address that which has no name? I am reminded of the Dalit or “untouchables” in the Indian caste system. Ask yourself,

Have you ever known a person without a name?

After you die, your tombstone is a sign or symbol of your prior existence. Photo by Gary Meulemans on Unsplash

Name as Symbol of Presence, Death and Representation

Our identities spring from and are shaped by the names given to us by others from the moment we leave our mothers’ bodies. Conversely, when we die, a tombstone with a name is all that’s left behind as a symbol — a trace — of our existence. Your name stands in for your identity, your presence as an actually existing thing. Remember: you don’t exist until you have been recognized as such by another cognitive agent. The word “exist” (Latin: existere), as Heidegger points out, originally meant something like “to stand out or stand forth.” To exist is to be a difference that makes a difference, in the words of polymath cyberneticist/anthropologist Gregory Bateson. In other words, to exist is to, at the very least, comprise one bit of information. The question is whether there is anything there sensitive enough to detect this difference.

Here is one rather sketchy psychological and theological implication of this idea. A belief in an omniscient God is one way to ensure recognition of one’s existence, regardless of whether one’s identity is recognized (politically or socially) on Earth. One can then go forward in life knowing one’s actions and thoughts are recognized, and thus have meaning to at least one cognitive agent, God. This explains why God is often the final bastion of comfort for those politically or socially oppressed. Faith in God can be viewed as a psychological strategy for coping with worldly torment, for ensuring the recognition of one’s existential bit — exist (1), not exist (0).

Did you ever write your name on a desk in school? That signature served to stand in, or re-present, your presence in your absence to any cognitive agent capable of reading it. It’s a sign, a signal, re-presenting the original and authentic Being of something at a later point in time. Representation is transitive: we represent something to someone. We must always consider that. A representation can carry information, but this information is interpretable only by those who recognize it as such. The receiver must be able to decode the Being or presence encoded in the representation.

In lieu of Being, signals re-present presence to receiver. Source: Wikipedia.

Signal detection and processing theory, information theory, etc., are mathematical formalizations of Being invented in order to extract, manipulate and derive useful patterns from the experience of any cognitive agent (yes, even a sensor) capable of imposing a boundary between itself and an other, an inner and an outer — an observer and a system. Remember that existence means to stand out, or stand forth. But stand out or forth from what? From the system itself. Philosophers such as Derrida and Heidegger ask whether redundancy is essential to identity or whether compression is.

Is the true signal the one which is revealed after maximal compression (ideal), or is ever-present noise part and parcel of its identity (actual)?

Evolutionary Epistemology

Not so long ago, philosophers such as Donald Campbell, and Nobel Prize-winning biologists such as Konrad Lorenz, made Kantian (i.e., transcendental) arguments to the effect that the mechanism of evolution could not function were it not “truth-tracking” to some extent. In philosophy this view was called Evolutionary Epistemology. Organisms whose faculties of perception were not aligned, to some degree, with reality, would eventually fail to reproduce or die off. Or so they thought.

Recently, cognitive scientist Donald Hoffman claims to have shown mathematically (his “fitness beats truth” theorem or FBT) that there is no necessary connection between what we perceive and reality as it is, “out there.” Our concepts of time and space, as we know them, are mere representations — good enough for survival and reproduction — of some deeper underlying reality. What that is, we have no idea. Students of Kant will recognize this as old hat. It’s simply the distinction between Dinge (phenomena) and Dinge an sich (noumena) rehashed in the language of modern science and computer simulation.

What does all this have to do with data? Well, eventually, we all die and will be absent forever to everyone; when that happens, only our names will be evidence of our existence. Death forecloses any possibility of presence, leaving mere representation in its wake. Never again can your face be compared with a digital picture of your face, or your voice compared with a digital recording of your voice. Similarity (and distance), will then be defined by a relation of copy with copy.

Image Recognition Performance Evaluation: How to Interpret?

Compare this with modern AI. We evaluate the performance of a CNN or GAN by how well it classifies a set of test images, for example, and we blindly assume this says something useful about its performance in the “real world.” But test images are not real: they are digital representations of the reality we live in. To the extent these digital representations capture what we believe is important, the performance measures are meaningful. Put another way, if all that mattered was digital reality, then 99% accuracy would indeed be impressive and useful in reality. Conflating digital reality with analog reality is one easy way to overstate the achievements of current AI. I, for one, would like to keep the distinction because I live in analog reality.

A representation by convention. I hereby claim this picture denotes Sadness. Photo by Steve Johnson on Unsplash

Representation is Convention, not Resemblance

If we take the view of Nelson Goodman (Conventionalism), then anything can represent anything through an arbitrary act of denotation. We simply define it into existence, as we might with a logical axiom. Why? Because a resemblance theory of representation and reference is too restrictive, a vestige of Western science’s striving towards a perfectly transparent mirror of nature. (Post)modern art aims to free itself from the restrictive metaphysics of a resemblance theory of representation. Representation, unlike similarity, is not a symmetric relationship!

A heart represents love, but love doesn’t represent a heart.

This is important for machine learning because a metric space is defined using a notion of symmetry. See axiom (ii) below.

Look at axiom (ii) which defines a metric in terms of symmetry. Source:

There is no necessary connection between performance on a representation of reality and performance in reality, though Pythagoras may have disagreed.

The obsession with test set performance in image recognition tasks is a good example of failing to realize we are dealing with what Baudrillard calls the hyperreal. We have replaced the formerly real with a copy of the real and forgotten that such a replacement has occurred. We have conflated reality with appearance. If Plato had to banish the dramatists and poets from his ideal Republic because they dealt with shadows of shadows, then he would also have to banish the data scientists and engineers because they deal in representations of reality without caring about the detailed relationship between reality and its representation.

Data is dead when unreflectively used to re-present the real.

A picture of cat is not a cat. People are not pixels, and pixels are not numbers. But we can represent them as if they were. How? By number.

But only when certain conditions hold will performance of our CNN be good and useful in the real world (the place we humans live in and actually care about). A good model is isomorphic to the part of reality we wish to model — it’s one to one (each thing in reality is mapped to one thing, and only one thing, in the model) and maintains the relations among its parts. For example, if our source system has three objects, (a,b,c), we might call the relation R the set of ordered pairs (a,b) and (b,c). In this case, the relation R holds for objects a and b, and b and c, but not a and c. If these relations exist in the source system, then they must be preserved in the model system.

When this happens we call this method of mapping numbers to properties of things in the world measurement, and the view that the defining property of a model and its target system (reality) should be structural isomorphism is called the semantic view of scientific theories.

But we cannot simply assume a priori that all objects possess quantitative structure. According to controversial historian of psychology Joel Michell, a fundamental task of science, then, is to investigate whether the properties of objects have quantitative structure that can be isomorphically mapped to numbers, thus preserving their inherent relations. If so, measurements of them are meaningful. If not, they are meaningless. Michell argues on this basis that much of what modern psychology is doing is “pathological” because it has not first demonstrated that psychological attributes have quantitative structure appropriate for quantification.

Meaning, Algebra, and Arbitrary Orderings

Psychometric theorists Keith Marcus and Denny Borsboom, in their book The Frontiers of Test Validity Theory, give a nice example of how meaning, algebra and order (logos) are related. They point out that we can develop algebraic operations by positing a few basic axioms, a sequence of numbers, and a successor function that returns the next number in the sequence. Here’s how it relates to the interpretation of numbers.

Normally, most of us assume a sequence of numbers where 1 is the successor of 0, and so x+1 equals the successor of x. But if we start with different orderings, we get different algebras. In ordering (A) we start with 0, 1, 2, 3, 4, 5, 6 … and then 2+2 =4. But in ordering (B) we start with 0, 1, 2, 3, 5, 4, 6… and then 2 + 2 = 5. Both answers are correct applications of the successor function and both ‘4’ and ‘5’ represent the same quantities, though couched in a different system of ordering.

Now suppose we now tell you these numbers represent something. Maybe the second ordering is someone’s preference for numbers, where 1 is his favorite. Then in both cases ‘4’ represents the fifth favorite number and ‘5’ represents the fourth favorite. Markus and Borsboom point out that the meanings of the algebraic operations differ in the resulting two algebras. They may share theorems, such as x+1 = x+2–1, but differ in others. For example, 4x > 5x. Which is right? You could create two algebras which share many of the same theorems by ordering the numbers in the same way but for different reasons.

As Markus and Borsboom explain:

Most people learn algebraic operations without first learning their basis in assumed orderings and successor functions. People depend upon experts for that. The result is that the meaning of mathematical statements depends on the social context. (pg. 257)

Dialogue and Perspective Taking

So we might think social dialogue and logic are opposing concepts. We’d be wrong: dia-logue derives from di (two) and logos (reason). Language and logic are thus inseparable from sociality. To be named is to be recognized as having a particular kind of identity which confers membership into a moral community. Naming probably occurs in all human cultures due to its performative role in creating symbolic moral communities of equals. Once addressed with names, we can be called into dialogue with others.

Dialogical communication requires the ability to imagine the perspective of our interlocutors and have a rudimentary grasp of theory of mind, which helps us to interpret the words and deeds of others by reference to our own inner dialogue and behavior. It goes the other way around too. We understand our own behavior and inner dialogue by watching and observing the words and deeds of those around us. When I see someone smash his finger with a hammer and say “ouch!” I know he must also experience some inner feeling we collectively refer to as pain. A philosophical chestnut consists in trying to verify, solely on the basis of observable behavior, how we know my pain and your pain are referring to the same kind of thing.

This kind of intersubjective perspective-taking apparently takes vast cognitive resources, but is crucial to human development. If we are to make joint plans and coordinate our actions with other human beings we must be able to imagine what it would be like, to some extent, to take the perspective of the other.

The influential Soviet psychologist Lev Vygotsky, himself a fan of Hegel, claimed that human development occurred first socially through inter-mental categorization, and then later this process was mirrored within the individual child as an intra-mental one. On this view, human individuality is derivative of human sociality. In a section below we will explore why this idea is somewhat radical in Western thought. Hint: it has to do with some intellectual baggage we inherited from Descartes. For now, I want to mention one thing with relevance to the collection of behavioral big data (BBD).

Can I haz join ur moral community? k thx. Photo by Cong H on Unsplash

Dehumanization, Pets, and Numeric Reference

We can also put this curious fact of human sociality to other, less morally pure uses. The philosopher Charles Taylor explains that when we wish to dehumanize a person, for instance, we do not call her by her name. Numbers have historically been used for “easy” reference. As just one silly example, in Star Wars the robots’ names were C3PO and R2D2, but the Wookie was called Chewbacca. Chewy mattered more.

We also do the converse when we give animals names. We call these animals “pets.” Giving an animal a name is a performative act conferring membership into our moral community. Objectively, nothing about the animal has changed. Yet, upon receiving a name, the creature’s identity has changed symbolically. We feel obligated now to consider its interests, perspective, and well-being and we implicitly recognize its capacity for emotions, such as pleasure and pain, as we go about our lives. Have you ever cut short an activity you were enjoying because you had to get home to feed the cat?

Finally, membership into our moral community confers another practical bonus: we tend not to kill and eat animals we have given a name to. In short, names are implicit evidence of a certain kind of essence and identity that is valuable (i.e., has dignity) to us as humans. But what does this strange behavior centered around namesreveal about who and what counts as part of our moral community?

People deemed “criminals” through enaction of performative legal processes are removed from the social community and often referred to using numeric identifiers. Why? Photo by Damir Spanic on Unsplash

Think back to the most gruesome periods of recent human history. The use of numbers to refer to persons has a sordid past. There’s a reason prisoners have them. It is something best avoided unless absolutely necessary, ID numbers and drivers’ licenses notwithstanding. It is only a small step to barbarism.

Here’s why. The number “5” could represent five apples, five dollars, or a single person. We’re indifferent to which it is. This power of numeric abstraction is also a weakness. We treat all instances of “5” the same, as objects to be manipulated according to the rules of algebra. This is the level of syntax. Algebra doesn’t care about which side of the equals sign variables appear on. But in many (social) sciences we make semantic distinctions between numbers in equations: we call them IVs and DVs, exogenous and endogenous variables in systems of equations and thus specify how such variables relate to one another, according to theory (etymologically theory just means a perspective, a view). Causal modeling provides an example of why we should care about questions of syntax and semantics. From the space of all possible predictive models we might build, only a fraction of them might obey the laws of physics or conform to our best theories.

So syntax and semantics matter when applied to persons. They have ethical implications. Referring to a person by using a number is to be indifferent to one’s identity as a person. It is arguably the ultimate indignity as it places one outside the norms of the human moral community (Worse, even, than killing — as that would imply recognition of your humanity and potential for death. Numbers, after all, don’t even receive burial rites.). As Charles Taylor insightfully notes in Sources of the Self,

Beings who are just referents and not also addressees are ipso facto classed as non-human, without identity.


Care must be taken when we deal with the personal data of human persons. By referring to persons by things like “user_id” we risk overlooking our potential to enter into dialogue with another human being. user_ids do not make plans, laugh, or tell jokes. People, however, do. user_ids only exist because we needed a convenient means of reference to an object, but ideally (e.g., with enough time, energy, and money) we would engage people face to face to understand them. user_ids are not reality, but representations of reality (real persons).

We must be careful to distinguish the two.

About the author

Travis Greene is a Ph.D. candidate at the Institute of Service Science in Taipei, Taiwan, where he studies the philosophical, ethical, and judicial implications of modern data science, machine learning algorithms, and recommender systems.

If you want to follow more of his writing, visit his Medium page Datasophy:

He previously appeared on the ACIT Science Podcast, discussing how human rights should shape our online lives.

Leave a Reply