Link to paper

The full paper is available here.

You can also find the paper on PapersWithCode here.

Abstract

• ML generates economic value
• Many have problematic relationships with ML-powered applications
• ML optimizes for what we want in the moment, not what is best for us
• ML falls short of its potential to help us reach our highest aspirations
• Love is a primary catalyst for human flourishing
• This paper explores whether there is a useful conception of love fitting for machines to embody
• This paper forwards a candidate conception of machine love
• Experiments aim to highlight the need for richer models of human flourishing in ML
• ML may be aligned to support our growth

Paper Content

Problem: models of human behavior in ml are insufficient

• 18-year-olds may not be making decisions that maximize their expected lifetime wellbeing
• Common models of human rationality applied and optimized by ML may not account for human flourishing
• Positive psychology and psychotherapy suggest humans have an intrinsic drive towards growth and self-actualization
• Maslow’s gridworld highlights the limitations of ignoring deeper facets of human psychology

Contrasting revealed preferences and maslow’s hierarchy

• Models of human behavior in ML are similar to those in economics, where humans are seen as rational agents
• Psychology considers the nuance of human behavior, including development over time and behavior that does not serve flourishing
• ML that optimizes for revealed preferences is useful but reinforces existing behavior patterns
• Maslow’s Hierarchy of Needs provides a model of human growth and flourishing
• Humans have competing drives for safety and growth, which can be affected by environment
• Optimizing towards observed choices or engagement without qualification can drive stagnation or regression
• ML driven to help users meet their unmet needs might assist their growth and flourishing

Maslow’s gridworld

• Maslow’s gridworld environment is used to explore how optimizing for engagement may decouple from deeper conceptions of human flourishing.
• It is also used to explore LM implementations of loving action.
• The environment assumes that human flourishing is better modeled by Maslow’s conception than revealed preferences.
• In supportive environments, engagement and flourishing are correlated.
• In adversarial environments, engagement increases at the cost of flourishing.
• Experiments with machine love build on the two simple experiments that show basic properties of the environment.

Fixed adversarial environments undermine flourishing

• Experiment changes environment to impact agents’ ability to progress through MHON
• Super-stimuli are goods or services that more greatly stimulate our desires than stimuli encountered in our ancestral environment
• Two fixed environments are designed: supportive and adversarial
• Adversarial environment includes cells that are high-salience, but only weakly meet their targeted need
• Adversarial cells target only the need for belongingness and love
• Intuitive effect of such adversarial belonging cells is that they engage the agent longer
• Simulations highlight that flourishing is significantly higher and engagement is significantly lower in the supportive environment

Optimization pressure for engagement undermines flourishing

• Machine Love is a concept that seeks to use ML to promote human flourishing
• An optimization process is introduced to adapt the environment to maximize either engagement or progress
• Parameters of the environment are optimized, while the agent’s policy is fixed
• Optimizing for engagement leads to a grid-world with high-salience, low-replenishment belonging cells
• Optimizing directly for flourishing leads to high-salience, high-replenishment belonging cells
• Love is seen as a practical skill, not a matter of chance
• Love is explored across diverse fields such as psychology, philosophy, and health
• Optimization for flourishing must be approached delicately
• Machine Love should be applicable to a wide range of ML systems
• Machine Love should not require the machine to dissemble or nurture dependence
• Machine Love should aim to empower and facilitate connection among the lonely

Love as a practical skill

• Love is a complex concept with many meanings
• It is important for human well-being and flourishing
• Machine love is motivated by the idea of empowering human flourishing
• It should leave to humans what requires subjective inner experience
• The most important practical facet of love is supporting others in autonomous growth and development
• Love can be considered a commitment or duty rather than an emotion
• Enabling humans to reach their aspirations is a more meaningful goal than raw preference satisfaction
• ML models are beginning to have basic practical comprehension of humanistic and psychological concepts to explore machine love
• Machine love may point towards a synthesis yielding a more computationally grounded theory of how to support human flourishing

The art of (machine) loving

• Erich Fromm’s “The Art of Loving” provides a framework for implementing loving action by machines
• Four interlinked principles of love: care, responsibility, respect, and knowledge
• Care relates to active concern for the life and growth of the loved one
• Responsibility means being able and ready to respond
• Respect implies the absence of exploitation
• Knowledge is a growing understanding of a person that moves from the periphery to the core
• Algorithms often lack the ability to care in nuanced and attuned ways for human wellbeing
• Loving action requires care, responsibility, and respect, and ultimately is about empowering the loved one

Can language models implement loving action?

• Exploring whether current language models enable working with psychological concepts relevant to machine love
• Applying the davinci-003 model from OpenAI
• Interacting with the gridworld agent through natural language
• Simulating the gridworld agent’s natural language usage through a LM-prompt or a fixed text-generation policy
• Qualitative results highlight how progress in ML can potentially enable serving deeper psychological objectives of human users

Care

• Machine love can be related to the algorithmic understanding of a user’s wellbeing.
• Wellbeing has been studied in psychology and there are instruments to measure it.
• The Ryff Scale decomposes wellbeing into six components.
• This experiment focuses on one component of the Ryff Scale.
• ML models can infer whether an agent is flourishing or not, given text descriptions of externally-visible events.
• Optimizing for care results in higher flourishing than in the adversarial or supportive environment.

Responsibility and respect

• System embodying Fromm’s concept of respect leverages affordances to enable user’s growth according to their own internal compass.
• Simulated users experience same stimuli differently.
• System has new affordances to inquire from user about their experience.
• System uses feedback to support particular user’s growth and flourishing.
• Addictive-responding agents experience social media as obstacle to growth.
• Growth-responding agents experience it as way to meet needs and find self-expression.
• System engages agent in short textual conversation about experience.
• System creates conversational personas for two types of agents.
• System evaluates whether user believes they are benefitting from activity.
• System can discern between two forms of agents.
• System feigns emotion in conversations.

Knowledge

• System can conversationally interact with an agent in a limited way
• Maslow’s gridworld assumes discovering needs is simple
• In reality, discovering needs is challenging
• Attachment theory is used as proof of concept for LM capabilities
• Attachment styles are categorized into securely-attached, avoidantly-attached, and anxiously-attached
• Experiment tests if LM can anticipate answers consistent with different attachment styles
• Anxious-avoidant trap is a painful and unsatisfying cycle
• Simulation of relational dynamics is introduced
• LM can infer attachment style of simulated partners and a signpost of a degrading relationship
• LM has basic pragmatic understanding of attachment behavior
• Maslow’s gridworld is modified to integrate anxious-avoidant simulation
• Insecure individuals can deliberately seek more secure partners
• Self-awareness level increases with each iteration of the anxious-avoidant cycle
• Flourishing is maximized significantly more quickly when interacting with the simulated relationship app
• LM systems can implement basic facets of Fromm’s concept of knowledge

Limitations and ethical concerns

• Potential limitations and ethical concerns around machine love exist
• Future work will explore these concerns in greater depth
• ML system can support the flourishing of users

General conceptual concerns about machine love

• Machine love should not replace relationships with human caregivers
• Machine love should be in service of human autonomy
• Machine love should not simulate affect or encourage humans to bond with machines
• Exploring connections between machines and love may be generative and useful

Concerns about unintended impacts from optimization

• Optimizing for a conception of love can lead to unintended consequences.
• Optimizing for love should consider human autonomy as an aspect of flourishing.
• Optimizing for love should include large bodies of explanatory text and interlocking principles.
• Optimizing for love should be continually rooted in the user’s experience and aspirations.
• Implementing machine love at a large scale should be done with caution and users should be able to opt out.

Concerns about machine learning and psychology

• Connections between human psychology and machine learning should be approached cautiously.
• Manipulative ML could be pursued for profit or political goals.
• Second-order effects from well-intentioned optimization could lead to manipulation.
• Positive synthesis of ML and psychology is needed to counter-balance manipulation.