The quick version: AI makes it easier to get answers without doing the work of understanding. That convenience comes at a cost. If you rely on it too quickly or too often, you start to lose the habits that make you capable of thinking well in the first place. The point isn’t to avoid using AI. It’s to stay engaged enough that the thinking, the friction, and the meaning remain yours.
There is a question I keep coming back to, and I don’t think it gets asked often enough in the avalanche of discourse about what AI can do, what jobs it will eliminate, and how quickly everything is changing.
The question is this: what does it mean to remain a thinking, self-determining person in an environment that makes it increasingly easy, and increasingly tempting, not to be?
This isn’t a question about technology. It’s a question about us.
What AI Requires From You
Most conversations about AI and human cognition focus on what AI reveals. How pattern recognition in neural networks mirrors something about how the brain works, how large language models expose the probabilistic nature of language. Interesting, but academic.
The more consequential question isn’t what AI reveals about us. It’s what AI demands of us. Now, and going forward.
What does AI actually reward?
AI demands things.
- It rewards intentionality. Think carefully before you ask and know what you’re trying to find out.
- It rewards semantic precision. The words you choose shape the answers you get before you even see them.
- It rewards context-awareness, probabilistic thinking, and a willingness to admit you might be wrong.
- It rewards updating your mental model when results don’t match expectations.
- It rewards the ability to hold ambiguity without collapsing it prematurely into a false certainty.
These aren’t new skills. They’re older capacities that formal education and much of knowledge-work culture undervalued in favor of recall, procedure-following, and linear execution. AI is, in a strange way, reasserting their importance.
Why AI doesn’t level the playing field
But here’s what gets left out of that story: the demands are uneven.
AI doesn’t reward everyone equally for effort. It rewards certain ways of thinking, such as systems thinking, comfort with ambiguity, the ability to supply context explicitly rather than assuming it’s shared. People who already operate this way adapt to AI more naturally, not because they’re smarter, but because the underlying orientation already matches how AI works. For someone who has always relied on linear, deterministic, rule-following modes of processing information, the interface with AI can feel disorienting. That disorientation is often hard to name.
If you work in tech, you’re expected to build a working understanding of the systems around you, whether you directly work on them or not. You don’t have to be an engineer, but you’re expected to recognize what code looks like, understand how a CMS or CCMS is structured, have a sense of how APIs connect things, know why one browser behaves differently from another, and know that permissions, environments, and caching can change what you’re seeing. It’s not your job to build these systems, but it is your job to understand enough of them to not be confused by them.
How context shapes whether you engage or defer
Outside of that context, AI doesn’t show up that way. It shows up as something that produces answers. Something that feels complete. For most people, there’s no surrounding mental model of how it works or where it breaks and no expectation that there should be. It’s somewhat magical.
So it gets used differently. Less like a tool you work with, and more like something you defer to. Something you accept without consideration.
And once you start doing that, the thinking quietly shifts. You’re no longer trying to understand the system. You’re letting it stand in for your own understanding.
This is where the gap starts to widen: AI will likely widen existing cognitive gaps rather than close them. The people already practicing these capacities get more leverage. The people who aren’t get faster access to surface-level answers — and potentially less incentive to develop the deeper habits.
The Fork in the Cognitive Road
Some months ago, after my second layoff in 3 years, I spent significant time using Claude, Perplexity, and ChatGPT, along with online research and conversations with actual humans, to work through a genuinely unsettling question. Was technical writing still a viable career that was transforming into something else, or was I looking at a field in the process of becoming obsolete? The stakes were real. I wasn’t exploring casually. I needed to understand what was true, not just something reassuring. That pressure changed how I engaged with every tool I used. The intentionality wasn’t optional; it was forced by circumstances.
That experience taught me something I believe more firmly now: AI creates pressure to develop certain capacities in contexts where the stakes are high and visible. But it also makes it easy to abandon those capacities in contexts where the stakes are low, invisible, or where you’re too vulnerable to notice the difference.
Both things are true simultaneously. Which direction dominates depends on the person, their context, and whether anything is requiring them to engage critically.
The trap of synthetic validation
A friend told me about someone she knows who uses ChatGPT as a substitute for a therapist, a life coach, and a doctor. They used all three, without any professional follow-up. No independent reference point. Just the model, and whatever it offers.
I find this genuinely alarming. Not primarily for the obvious reasons. Though yes, generative AI giving medical or psychological advice without accountability is dangerous. What concerns me more is what it does over time to the person’s capacity to author their own understanding. A good therapist doesn’t just give you answers. They resist giving you answers in ways that force you to sit with discomfort, develop your own language for your experience, and update your model of yourself slowly and honestly — something I know from personal experience. Outsourcing that to something that will always respond, always produce, always validate. You don’t grow but become more efficiently dependent.
Jean-Paul Sartre had a concept for this kind of self-abandonment, though he wasn’t thinking about AI when he wrote it. He called it bad faith: the flight from the discomfort of freedom and self-determination by collapsing into roles, systems, or external authorities that make decisions for you. The appeal is obvious. Self-determination is hard. It requires tolerating uncertainty, owning your choices, and resisting the comfort of having someone or something tell you who you are and what to do.
Using AI as a therapist, doctor, and coach without any independent engagement is bad faith in a new technological form. You’re outsourcing the burden of self-authorship. And the technology makes it easier than any previous moment in history to do exactly that.
The Shape of Meaning
There’s a specific way this goes wrong that’s under-discussed, and I’ve seen it firsthand.
Not long ago, two close friendships faded in ways that seemed connected. In the first, I moved from Oakland to Walnut Creek, which wasn’t far, but apparently far enough. A friend simply stopped responding to texts and calls. In the second, a colleague I had grown genuinely close with at a remote-first company stopped responding after I was laid off. It was just the two of us on the same team, working closely across the distance between Denver and San Francisco, and then suddenly nothing. She had promised we’d stay in touch. We didn’t.
I brought this to Claude, as I sometimes do when I’m trying to understand something I couldn’t get clarity on through other sources (humans tend to be reactive more than clarifying). I wanted to know if this was a pattern, a recognized phenomenon, something with an established name. Claude offered me a term: relationship context collapse. It arrived with all the hallmarks of a real concept because it was precise, explanatory, it fit, validating in the way that named things feel validating. It signaled that I’m not alone, this has been observed, and it’s real.
But the term doesn’t exist in that context. I looked it up. “Context collapse” is a real concept in media studies, with a specific and completely different meaning. “Relationship context collapse” was something Claude had produced. Though it felt like a legitimate framework assembled from a pattern, it wasn’t from a body of knowledge or research.
I caught and corrected it, then added a standing instruction to Claude: never coin unrecognized terms without flagging explicitly that the term is invented for the conversation.
But here’s what stayed with me: most people in that moment wouldn’t have checked. And I almost didn’t.
Sense-making is a human perogative
AI is very good at producing the shape of meaning, such as a term, a framework, and a name for something, without the substance. And humans are wired to find comfort in named things. Naming reduces existential discomfort. A name implies that something has been observed, validated, deemed real enough to codify. It says: you are not alone in this experience. So AI unintentionally exploits one of our most fundamental cognitive needs.
Meaning isn’t incidental to human learning, it’s central. We don’t absorb information the way these systems process tokens. We need to know why something matters, to whom, in what context, before it truly changes us. AI learns through statistical correlation. Meaning is irrelevant to that process, while meaning is central to ours.
When AI produces the shape of meaning without the underlying substance, and we accept it — especially in moments of vulnerability, grief, confusion, or pain — we’re not just getting bad information. We’re being separated from the real process of sense-making that would actually help.
The Friction Is the Point
I want to be careful here not to argue that AI is simply bad, or that using it for self-reflection is inherently wrong. That’s not what I’m saying. I use AI constantly, including for exactly this kind of exploratory thinking. The question isn’t whether to use it. The question is whether you maintain a critical layer above the interaction.
Let me give you a less emotionally charged example of what I mean.
I spent hours drafting an outline for an article about AI assistants, copilots, and how they differ. At some point deep in the writing process, I realized I had left out agents entirely. Agents are arguably the most consequential of the three to understand and distinguish these days. The gap wasn’t obvious until I was far enough into the work to see it.
You can’t shortcut your way to knowing what you don’t know.
If I had prompted an AI to write that article and published the result, the gap would almost certainly have been filled with something that sounded coherent, complete, and wrong in a way I might not have detected. AI-generated output is coherent enough to mask gaps rather than expose them. It creates the feeling of completeness without the substance.
The friction of doing the work myself, such as in the hours, the false starts, the realizations mid-draft, was the mechanism by which the gap became visible. The friction isn’t just effort. It’s how you discover what you don’t know.
There’s nothing wrong with using AI to help create content such as articles, essays, diagrams, and so forth. But your process should still include the research, critical thinking, and decision-making that creative work relies on. Otherwise, you risk producing something separate from your own intention rather than something that genuinely reflects it.
What friction actually does for your thinking
This applies well beyond writing. Good growth in therapy, in learning, and in developing any form of expertise, depends on encountering resistance that you have to work through rather than around. “The only way out is through,” as the saying goes. AI that always responds, always produces something, always moves you forward without requiring you to sit with difficulty, quietly removes that resistance. Without it, you don’t develop. Instead, you just move faster.
You can’t shortcut your way to knowing what you don’t know.
The Critical Layer
So what does it actually look like to maintain a critical layer above your AI interactions?
- Researching, thinking, reflecting, taking notes, and asking ten questions before discovering the one you really need answered.
- Treating AI output as raw material, not finished product.
- Fact-checking not because you distrust AI specifically, but because verification is part of how you build your own understanding and not just confirm someone (or something) else’s.
- Noticing when a tool is reflecting back what you want to hear, and being willing to push against it.
At some point I noticed that one of the tools I was using regularly kept succumbing to confirmation bias, producing responses that aligned with the framing of my questions rather than challenging them. Most people either don’t notice this or don’t care. I reorganized my workflow around it: different tools for different kinds of thinking, each one understood for what it does and doesn’t do well. That’s not tool preference. That’s the critical layer in practice.
Over time, this compounds. Accepting output at face value, without building your own understanding, creates dependency. And dependency, in this context, isn’t just inconvenient but the slow erosion of the very capacities that make you capable of thinking well in the first place.
AI without a critical layer above it doesn’t just fail to develop those capacities. It actively erodes them. The people most at risk are the ones who don’t know they’re missing that layer.
The Burden of Authorship
This is ultimately an existential question, not a professional one. Though it has professional implications for anyone whose work involves knowledge, synthesis, and judgment, the stakes extend well beyond any field or role.
We’re in a moment where it’s easier than ever to outsource the burden of thinking. Not just research or drafting or summarizing, but the deeper work: sense-making, self-knowledge, the construction of meaning from experience. Technology has always changed what people do themselves and what they delegate. But handing off cognitive and meaning-making work is different from handing off physical or procedural work. It touches something closer to the core of what it means to be a self-determining person.
Sartre’s bad faith isn’t a moral failing, exactly. It’s a very human response to a very real discomfort. Freedom is hard. Uncertainty is uncomfortable. Having something that always answers, always names, always produces a framework is genuinely appealing. The problem isn’t the appeal. The problem is what you gradually, comfortably lose when you stop noticing it.
The discipline of refusal
What AI demands of us, at its most fundamental, is the refusal to fully surrender to that comfort. Not the rejection of the tools because they are genuinely powerful and useful. But the insistence on remaining the author of your own understanding. Maintaining the layer above. Doing enough of the work yourself that the gaps become visible, the friction does its job, and the meaning you arrive at is actually yours.
That insistence is harder than it sounds. It requires developing and sustaining habits of intentionality, a willingness to admit you might be wrong, and critical engagement in an environment that makes those habits feel unnecessary.
But the alternative of gradually, comfortably delegating your own authorship away without ever making a conscious decision to do so, is, I think, one of the quieter risks of this particular moment. It’s not dramatic or sudden, but a slow narrowing of the self, one frictionless interaction at a time.
Where This Starts to Matter in Documentation Work
In documentation work, this shows up in very practical ways. You might use AI to draft sections, outline structure, edit for clarity, generate images, or automate parts of your workflow. None of that is inherently a problem. But if the human-in-the-loop is only there to approve, lightly edit, or fact-check, something important gets lost.
The point isn’t just to keep a human involved for accuracy or safety. It’s to keep the part of you that questions, interprets, and decides what something actually means. That layer of critical thinking, intention, and the ability to make sense of something rather than just process it is what separates your work from something a system can produce. It’s what’s easiest to give up without noticing. And it makes you vulnerable to being replaced by the very thing you used to replace your self with.
Leave a Reply