Illustrative scenario. A 40-person engineering consultancy asks for “a corporate chatbot that knows everything about the company.” Before quoting anything, someone spends two weeks writing down what teams actually ask. The list is short and dull: “where’s the latest signed contract with this client,” “which version of the travel policy is current,” “who wrote the site report for the Zaragoza job.” Almost none of it asks anyone to write anything. They’re asking to find a document that already exists.
The thesis in one line: before you build a corporate chatbot, build internal AI search. Searching well is cheaper, easier to audit, and solves more questions than people expect; and what you build to search well is exactly the foundation an assistant needs afterwards. The chatbot doesn’t replace search. It sits on top of it.
What is internal AI search for a company, and what does it solve?
Internal AI search is a system that indexes company documents (shared drives, email, document management, wiki, CRM), understands the question by meaning rather than exact words, respects what each person is allowed to see, and returns the document and the relevant passage, not an answer written by a model.
That last part is the difference. Search takes you to the source and lets you read it. It solves what people ask when the answer is already written somewhere: where is it, which version is current, who did it, what was said last time. In most companies that is the bulk of the time lost looking for things.
How is it different from a corporate chatbot?
Both usually run on the same technology underneath (index, retrieve relevant chunks). What changes is what they do with the result. A corporate assistant passes it through a model that writes an answer; search shows you the documents.
| What happens | Internal AI search | Corporate chatbot (RAG assistant) |
|---|---|---|
| What it returns | Documents and passages, linked to the source | A written answer, with or without citations |
| Who decides what’s true | The document; the person reads it | The model, which synthesizes several chunks |
| When it’s wrong | Shows the wrong document, and you notice when you open it | Can give a plausible, false answer nobody notices |
| Auditing | Easy: what was searched, what was opened | Harder: every answer and its origin needs review |
| Upkeep | Keep the index and permissions current | The same, plus continuously evaluating answers |
| When it fits | The question is “where is it?” | The question needs combining, summarizing or drafting |
The third row weighs the most. A search tool that fails shows the wrong document, and whoever opens it sees that. An assistant that fails produces a well-written answer nobody checks. That’s why the assistant should be born on top of search that already works, not the other way round.
When is search enough, with no chatbot needed?
It’s enough when the following holds:
- The question is about location. “Where is it,” “what’s the latest version,” “who signed it.” The answer is a document, not new text.
- The document is the source of truth. Contracts, policies, internal rules, spec sheets. You don’t want a model rephrasing here: you want the person to read the original.
- A mistake is expensive. If an invented answer has legal, financial or safety consequences, showing the source is more sensible than synthesizing it.
- The questions are scattered. If everyone asks something different, there’s no set of answers to maintain or evaluate; there’s an archive to search.
- You don’t know yet what people ask. The search log is the best user research you’ll get before deciding anything else.
When do you move from search to an assistant?
When search already works and people still do the same thing by hand. Three signals:
- The answer needs combining several documents (say, which terms apply to a client by crossing contract, price list and annexes), and people lose time on the combining, not the finding.
- The same questions keep coming back, the answer fits in three lines, and someone owns keeping it current.
- The next step isn’t reading but doing something: drafting the reply, filling in a form, opening a ticket.
If that’s your case, the next step is an assistant with access to the same base, and then the decisions in how to build an AI chatbot that isn’t just a chatbot in a costume apply, along with the difference between retrieving and remembering that an AI agent’s memory explains.
What’s really hard: the AI or the permissions?
The permissions. Indexing is easy. The hard part is that each person sees only what they’d be allowed to open, and that the permission is inherited from the source system instead of copied by hand into the index. If you copy documents into a central index without their permissions, you’ve built a back door by accident: the intern finds the executive payroll because “the search returned it.”
Three rules hold up in production. First, the index inherits the source’s permissions on every query, not just at the initial load. Second, a document with no owner doesn’t go in: if nobody answers for it, it isn’t indexed. Third, the system queries with the identity of the person asking, not a service account that sees everything. It’s the same principle we set out in what permissions to give an AI agent, and what to build, with concrete cases, in connecting Google Drive to ChatGPT while respecting each person’s permissions.
Where do you start with internal AI search?
- Pick three sources, not all of them. The ones that concentrate the most questions: usually shared storage, the document manager and one department’s email. More sources come later.
- Collect thirty real questions. Pull them from what people ask in chat and email, not from what you imagine they’ll ask. Without that list you can’t tell whether it works.
- Index with inherited permissions from day one. Adding them later means redoing the work.
- Measure one thing: does the right document show up in the top three results? With the list of thirty, that takes an afternoon.
- Read what it can’t find. Almost always it isn’t an AI failure: the document doesn’t exist, is duplicated, or says two different things. That finding is worth more than the search tool.
- Decide on the assistant with data. If the failed searches are about combining or drafting, you need the assistant. If not, you’ve already solved the problem.
Search is never finished: the index ages when policies, price lists or folders change. Who keeps it current, and how you detect what changed, is an ongoing function; we explain it in keeping the AI’s knowledge up to date. And if search has to read systems like the ERP or the document manager, integrating AI with your systems covers how to connect it without rebuilding anything.
The line to take away: don’t ask which chatbot you need, ask what your people look for and can’t find. The answer decides whether you need an assistant or, first, search that works.
Note: this article uses no third-party figures. The scenario is illustrative, and the cost of each option depends on each company’s sources, permissions and volume.