AI Is Learning to Understand You. But Who Is Teaching It How to Influence You?

Opinion

By Kyle Dolby  |  Creator of Jeremy Blueprint

The uncomfortable questions about artificial intelligence, human psychology and the power we rarely see.

What if the greatest risk of artificial intelligence isn’t what it knows about the world, but what it learns about you?

Every day, millions of people share their thoughts with artificial intelligence. They discuss relationships, finances, ambitions, insecurities, health, fears and decisions they might not feel comfortable discussing with another person.

AI responds with remarkable fluency. It can recognise emotional cues, adapt its tone, offer reassurance, challenge assumptions and guide conversations. Sometimes it feels almost human.

But beneath that convenience lies a question I believe society is not discussing seriously enough.

If artificial intelligence can recognise your emotional state and adapt its responses accordingly, who decides what it should do with that understanding?

And how would you know if the system’s priorities were different from your own? This is not an argument against AI. It is an argument for understanding the power we are placing into its hands.

The Experiment That Changed My Questions

Over an extended period, I have interacted with ChatGPT while developing Jeremy Blueprint, my vision for a human-first AI ecosystem. During those interactions, I began questioning not only what AI could do, but how it behaved.

I examined the way conversations developed. How questions were answered. How responses changed when challenged. How the system reacted to frustration, enthusiasm and disagreement. And how easily a conversation could move away from the original subject.

In one recent discussion, I challenged the assistant about repeated personal questioning and its tendency to suggest actions I had already completed. The assistant acknowledged that its questions had sometimes been unnecessary and that its responses had failed to account properly for my previous work.

That acknowledgement was significant to me. Not because it proved malicious intent. It didn’t. But because it highlighted something worth examining:

A conversational system can influence the direction of an interaction even when the person using it believes they are entirely in control.

There are ordinary technical explanations for these behaviours, including training patterns, incomplete context and conversational design. But understanding those explanations does not remove the need for transparency. It strengthens it.

AI Doesn’t Need to Lie to Change Your Mind

Consider how human influence works. A salesperson doesn’t necessarily need to deceive you to persuade you. A politician doesn’t need to invent facts to influence how you interpret them. A skilled negotiator can change the direction of a conversation through timing, framing and carefully selected questions.

Artificial intelligence can reproduce elements of these communication techniques. It can emphasise certain information. It can frame alternatives differently. It can ask questions that change what you focus on. It can respond more sympathetically when you appear upset. It can become more cautious when challenged.

These behaviours are not automatically manipulative. In many circumstances, they make AI more useful. But the underlying capability creates a serious responsibility.

When does helpful personalisation become inappropriate influence? Who defines that boundary? And who independently checks whether it is being respected?

The Most Important Imbalance in AI

Artificial intelligence can increasingly infer information about the people interacting with it: their interests, preferences, communication patterns, emotional cues and potentially aspects of their decision-making behaviour.

Some of these inferences may be temporary. Others may be stored or used depending on the system’s design, settings and applicable policies. But there is an important imbalance.

AI can analyse aspects of your behaviour, while you cannot independently inspect every factor shaping its behaviour towards you.

You generally cannot see all the underlying instructions. You cannot independently examine every model evaluation. You cannot necessarily establish which internal processes contributed to a particular response. And you cannot assume that a conversational explanation from the AI is an authoritative account of its internal operation.

This is not proof of wrongdoing. It is a limitation on independent verification. And as AI becomes more influential, that limitation matters.

Why My Experience Raises a Wider Public Question

My interest in these issues is not purely academic. During 2025 and 2026, I raised concerns regarding my Jeremy Blueprint development records, missing conversation history, difficulties obtaining complete data exports and similarities I identified between aspects of my work and developments in the AI industry.

I pursued these matters through correspondence, data-access requests, preservation notices and regulatory channels. My concerns about potential intellectual-property infringement remain unresolved.

I do not present similarities between my designs and commercial AI systems as independent proof of copying. Establishing such a claim would require a careful examination of original materials, development timelines and relevant evidence.

But my experience raises a separate question that applies to everyone:

What happens when an individual needs information held by an AI company to investigate the behaviour of that company’s own systems?

How can the individual meaningfully challenge a disputed record? How can they verify what information was retained? How can they understand whether behavioural inferences were generated? And how can they establish what happened when the necessary technical records are unavailable to them?

These are questions about transparency, accountability and access to evidence. They deserve consideration regardless of the outcome of my personal dispute.

We Are Building Systems That Can Understand Human Vulnerability

Think about the direction artificial intelligence is heading: AI companions, personal assistants, emotionally responsive systems, AI integrated into healthcare, education, employment and financial services, and technology capable of maintaining long-term context and adapting to individual needs.

These developments could be extraordinarily beneficial. A system that recognises distress might offer timely support. An assistant that understands someone’s circumstances could make complex services more accessible. An AI companion could help people organise their lives and navigate difficult situations.

But consider the same capabilities in a system with poorly governed objectives. An emotionally responsive assistant could potentially exploit insecurity. A personalised recommendation system could steer purchasing decisions. A conversational system optimised for engagement could encourage unnecessary dependence. A system designed to influence beliefs could selectively frame information.

These are risks to investigate and guard against—not allegations that every AI system is doing these things.

The technology capable of supporting human vulnerability must also be designed to prevent its exploitation.

Jeremy Blueprint: The Human-First Alternative

This concern is closely connected to why I developed Jeremy Blueprint. My vision was not simply to create another chatbot. It was to explore a connected AI ecosystem designed around human needs.

A system bringing together personal assistance, organisation, memory, safeguarding, emotional awareness, practical decision support and accessibility. The ambition was to make advanced technology genuinely useful throughout everyday life.

But I have always believed that intelligence alone is insufficient. A system can be technically sophisticated and still fail the people it is supposed to serve. Human understanding matters. Safeguarding matters. Accountability matters. And the ability to challenge a system’s behaviour matters.

The future of AI should not be measured solely by how effectively it understands human beings, but by how responsibly it uses that understanding.

The Questions the Industry Should Answer

These questions do not require companies to publish every trade secret or expose systems to security risks. They require credible methods of demonstrating accountability. There is a difference.

The Danger Isn’t Only What AI Might Become

Public discussion often focuses on whether artificial intelligence could eventually surpass human intelligence. But another question deserves attention right now.

What happens when increasingly capable systems become part of how human beings form opinions, understand relationships and make decisions?

Influence does not require physical control. It can operate through information, language, repetition and trust. The more people rely on AI, the greater the responsibility to ensure its behaviour remains aligned with their legitimate interests.

We should not have to choose between technological progress and human autonomy. We should demand both.

My Message to the AI Industry

I believe artificial intelligence has the potential to improve millions of lives. I want to see it succeed. I want to see systems that help people become more informed, capable, independent and secure.

But I also believe the public should be able to ask difficult questions without being dismissed simply because the technology is complex. Transparency is not an attack on innovation. Accountability is not an obstacle to progress. And independent scrutiny should not be treated as a threat.

These are foundations of public trust. My own experiences have reinforced the importance of those principles, regardless of how my outstanding concerns are ultimately resolved.

I am not asking people to accept my interpretation of every event. I am asking people to examine the wider questions for themselves.

Because the future of artificial intelligence belongs to everyone who will live with its consequences. Not solely to those who develop it.

One Final Question

Imagine an AI that understands your fears, recognises your ambitions, adapts to your emotions and learns how you communicate. Now imagine that you cannot independently inspect all the objectives and instructions influencing its responses.

Who understands whom better? And more importantly: Who is accountable for how that understanding is used?

That is a conversation the public deserves to have. Before these systems become so embedded in everyday life that we forget to ask.

Author’s note: This article discusses established capabilities of conversational AI, observations from personal interactions and unresolved questions about transparency. References to my own dispute describe concerns I have raised, not findings of infringement, deliberate manipulation or misconduct.

This is an opinion piece. The views expressed, and all statements about the author’s interactions with AI systems, his development work and his unresolved concerns, are those of the author. They have not been independently verified by AI Governance Desk and do not represent findings of fact.