Knowledge Base AI in HighLevel

Updated: February 4, 2026  | 
Author: Bill Stilwell

This post covers how the Web Crawler works, how to write effective Custom Bot Responses, which takes priority, how to test training accuracy, and how to maintain the knowledge base over time.

Reading time: about 8 minutes.

What Is Knowledge Base AI in HighLevel?

Knowledge Base AI is the training layer of HighLevel’s Conversation AI bot.

When a contact sends an inbound message and the AI bot responds, the accuracy of that response depends entirely on what the bot has been trained on. Without good training, the bot either gives vague non-answers or, worse, generates plausible-sounding but incorrect information about the business.

The Knowledge Base AI system gives you two tools to prevent that. The Web Crawler feeds the bot real content from the business website.

Custom Bot Responses give it exact, controlled answers for specific high-priority questions.

Find both tools at Settings, then Conversation AI, then the Bot Training tab.

Why Training Quality Matters

An untrained or poorly trained AI bot is worse than no bot. It creates a worse customer experience than silence because it confidently gives wrong answers.

A prospect asks “Do you serve the [City] area?” and the bot says yes – because the city appeared somewhere on the website in a different context – when the answer is actually no. That contact drives two hours to a service call that was never booked.

That is a real business problem that training prevents.

The quality of the Knowledge Base is the single most important variable in Conversation AI performance. Better than the persona settings.

More impactful than the mode configuration. The bot can only be as good as the information it has access to.

The Web Crawler

The Web Crawler fetches content from the business website and adds it to the bot’s knowledge base.

Enter a URL in the crawler section and choose the crawl type. Single URL fetches just that specific page.

Directory fetches all pages under a specific path – for example, crawling /services/ captures all service sub-pages. Full domain attempts to crawl the entire website.

After crawling, review the list of pages that were indexed. Remove irrelevant pages – contact forms, privacy policies, login pages, error pages – that do not contribute useful business information.

The bot does not benefit from knowing the content of a GDPR compliance page.

The crawl indexes the text content of each page. Images, embedded videos, and form content are not captured.

If important information is only in an image or a form, it needs to be added as a Custom Bot Response rather than relying on the crawl.

Custom Bot Responses

Custom Bot Responses are the highest-value training input in the Knowledge Base.

You write each question exactly as a customer would ask it, and then write the exact answer the bot should deliver. When an incoming message matches a custom question, the bot returns that specific answer rather than generating a response from the web crawl or general AI training data.

This matters for precision. The web crawl gives the bot broad knowledge, but it produces variable response quality – the bot synthesizes an answer from indexed page content, which may be accurate or may pull the wrong details from the wrong context.

Custom Bot Responses eliminate that variability for the questions that matter most. Pricing, service areas, hours, booking process, cancellation policy – any question where an inaccurate answer creates a real problem should have a custom response.

Which Takes Priority?

Custom Bot Responses always take priority over web crawl content.

When an incoming message is detected as matching a custom question, the bot uses that specific answer and does not draw from the crawled content at all. The crawl is only used for questions that do not match any custom entry.

This means the knowledge base has a clear hierarchy: Custom Bot Responses for high-stakes questions, web crawl for general background knowledge, and the general AI model as the final fallback for everything else.

Build the custom responses first for the questions that matter most. Let the web crawl fill in the gaps for everything else.

How to Write Effective Q&A Pairs

The question part of each pair is how the bot identifies a match. The more precisely it matches how customers actually phrase the question, the better the matching accuracy.

Write questions in the customer’s voice, not the business’s internal language. “What’s the cost of a cleaning?” not “Residential cleaning service pricing inquiry.” Pull actual phrasing from inbound messages, call transcripts, and email threads if available.

Add multiple entries for the same answer. “What are your hours?”, “When are you open?”, “Are you open on Saturdays?”, and “Do you work on weekends?” all need the same answer but will not all match a single entry.

Adding each variation as a separate Custom Bot Response improves matching significantly.

Write answers as the bot should deliver them – complete, accurate, and appropriate for a chat format. Avoid long paragraphs.

A two to four sentence answer that covers the key information clearly works better than a comprehensive essay.

For answers with multiple parts – for example, a service area that covers several cities – use a simple list format in the answer text rather than a long run-on sentence.

Testing with Bot Trial

The Bot Trial tab in Conversation AI settings is the validation step before any live deployment.

After adding your custom responses and running the web crawl, go to Bot Trial and simulate the conversations you expect the bot to handle. Send the exact questions you added as custom responses and verify the bot returns the correct answer.

Then send variations of those questions that you did not explicitly add – “Can you help me on Sunday?” when you added “Do you work on weekends?” – to test whether the AI matches close variations.

Also test questions that are not in the custom list and should be answered from the web crawl. A question about a specific service the business offers, or about the team, or about the process – does the bot produce a reasonable answer or does it hallucinate something inaccurate?

Every inaccurate response in Bot Trial is a gap to fix before the bot goes live. Add a custom response for it, re-crawl the relevant page if the content exists on the site, or update the persona instructions to handle that topic differently.

Maintaining the Knowledge Base

The knowledge base goes stale as the business changes. Prices change.

Hours change. Services are added or discontinued.

Service areas expand or contract.

The web crawl does not update automatically. When significant content changes on the website, you need to manually re-crawl the affected pages to keep the bot’s general knowledge current.

Custom Bot Responses also need updating when the specific answers they contain become outdated. A custom response with the wrong hours is worse than no custom response – it confidently delivers incorrect information.

A good practice is to include a knowledge base review in any process that updates business information. When hours change, update the hours on the website AND update the custom bot response for hours questions.

When a new service is added, re-crawl the services page AND add a custom Q&A for the most common questions about that service.

What Can You Do With It?

• Train the bot on real business content in minutes: Enter the website URL, run the crawl, and the bot immediately has broad knowledge about the business’s services, team, and offerings – without manually entering every piece of information.
• Control exactly what the bot says for high-priority questions: Custom Bot Responses lock in specific answers for pricing, service area, hours, and cancellation policy – questions where an inaccurate AI-generated response creates real problems.
• Eliminate the most common bot failure modes: Hallucinated service areas, wrong prices, outdated hours – these failures happen when the bot draws from insufficient or incorrect training. Custom responses prevent them for the questions that matter most.
• Improve bot accuracy without changing anything else: The same persona, the same mode configuration, the same channel settings – better training produces better responses. Knowledge base quality is the highest-leverage variable in bot performance.
• Set up bots for agency clients efficiently: Run the web crawl on the client’s website, then add 15 to 20 custom Q&As based on the business type. A well-structured training session for a home services bot takes 30 to 45 minutes and produces dramatically better results than a default unconfigured deployment.
• Keep the bot current with minimal effort: A quarterly knowledge base review – re-crawl changed pages, update outdated custom responses – takes 20 to 30 minutes and keeps the bot’s accuracy at the level it was at launch.

Key Definitions

Use Cases by Industry

Home Services

A plumbing company configures its bot’s knowledge base with a full website crawl and 20 custom Q&As covering service areas, typical job pricing ranges, emergency call availability, and the booking process.

Without the custom responses, the bot’s answer to “Do you serve [City]?” was generated from web content and was sometimes wrong – the city name appeared in a testimonial but was not actually in the service area. The custom response for service area questions now returns the exact list of covered zip codes every time.

Result: The service area question – previously the most common source of bot errors – is now answered correctly 100% of the time because the custom response locks it in.

Dental Practices

A dental practice adds 18 custom Q&As covering insurance accepted, new patient process, appointment cancellation policy, appointment types and durations, and location and parking. The website crawl handles general information about the practice and team.

During Bot Trial testing, the practice manager discovers the bot was generating an answer about financing options that no longer existed. The outdated content was still on an old website page that had been crawled.

The page is removed from the crawl and a custom response clarifies the current financing situation.

Result: A potentially embarrassing bot error – telling patients about financing that was discontinued – is caught in testing rather than in a live conversation with a new patient.

Real Estate

A real estate agent adds custom responses for the most common buyer and seller questions – what documentation is needed, how long the process takes, what areas they specialize in, how their commission structure works, and what the first step is.

Multiple phrasings are added for each question. “What areas do you cover?”, “What neighborhoods do you know?”, and “Do you work in [specific city]?” all route to the same custom answer about the agent’s geographic focus.

Result: The bot handles the most common pre-qualification questions consistently and accurately, and the agent receives leads who have already been screened for fit rather than contacts who got vague or wrong information.

Marketing Agencies – Client Onboarding

An agency has a standard knowledge base setup process for every new client. The process: crawl the client website, remove irrelevant pages, then add 15 to 20 custom Q&As based on the client’s industry template (home services template, dental template, real estate template).

The template questions are customized with the specific client’s pricing, hours, and service area. The entire setup takes 30 to 45 minutes per client.

Bot Trial confirms accuracy before the bot goes live on any channel.

Result: A repeatable, consistent process produces well-trained bots for every client without starting from scratch each time. The template approach also means new team members can execute the setup correctly without deep AI knowledge.

Insurance Agencies

An insurance agency adds custom responses for the questions prospects most commonly ask before requesting a quote – which carriers are represented, what types of coverage are available, how the quoting process works, and how long a quote takes.

The bot trial reveals the general crawl was producing answers about specific policy types the agency no longer offers. Those pages are excluded from the crawl and replacement custom responses clarify what is currently available.

Result: The bot accurately describes current offerings and sets correct expectations before a prospect goes through the quote process – reducing the number of prospects who drop off after finding out the agency does not offer what they expected.

Who Is This For?

How to Train the Knowledge Base AI

Step 1: Open the Bot Training tab

Go to Settings, then Conversation AI in the sub-account.

Click the Bot Training tab. Both the Web Crawler and Custom Bot Responses sections are here.

Step 2: Run the Web Crawler

In the Web Crawler section, enter the business website URL.

Choose whether to crawl by specific URL, directory, or full domain. Click Crawl to start indexing.

Wait for the crawl to complete before moving to the next step.

Step 3: Review and clean the crawl results

After crawling, review the list of indexed pages. Remove irrelevant pages – contact forms, privacy policies, login pages, error pages, old blog posts with no business relevance.

The goal is a knowledge base of pages that accurately represent what the business currently offers.

Step 4: List the 15 to 25 most common questions

Before adding Custom Bot Responses, compile the questions the business receives most frequently. Use inbound message history, call notes, front desk logs, and social message threads as sources.

These are the questions with the highest volume and the highest cost when answered incorrectly.

Step 5: Add Custom Bot Responses

In the Custom Bot Responses section, add each question and its exact answer.

Write the question using the customer’s natural phrasing. Write the answer as the bot should deliver it – clear, accurate, and appropriately brief for a chat message.

Step 6: Add multiple phrasings for the same question

For each high-priority question, add two to three additional phrasings as separate entries with the same answer.

“What are your hours?”, “When are you open?”, and “Are you open Sundays?” all need the same answer. Adding each improves matching coverage.

Step 7: Test with Bot Trial

Go to the Bot Trial tab. Send the custom questions you added and verify the bot returns the correct answers.

Also test phrasing variations and questions from the web crawl content.

Note any inaccurate responses – each one is a gap to fix before the bot goes live.

Step 8: Fix identified gaps

Add custom responses for any questions where the bot performed poorly in testing. Re-crawl relevant pages if important content was missed.

Adjust the agent instructions if the bot’s tone or approach was off-target.

Re-test after each change until the most common question types are handled correctly.

Step 9: Schedule quarterly knowledge base reviews

Set a reminder to review the knowledge base every 90 days.

Re-crawl any pages where content has changed. Update or add Custom Bot Responses when business information shifts – hours, pricing, services, service area, or policies.

How Does It Connect to HighLevel?

• Conversation AI: Knowledge Base AI training directly determines the quality of Conversation AI responses. The web crawl and custom Q&As are the primary knowledge sources the bot draws from when generating replies across all channels.
• AI Conversation Agent: The AI Conversation Agent configuration – persona, objective, channel settings – shapes how the bot communicates. Knowledge Base AI training shapes what it knows. Both are needed for a well-performing bot.
• Conversation AI Flow Builder: Flow Builder structured paths handle specific high-value intents with defined responses. Knowledge Base AI handles the general conversational knowledge that fills everything outside those flows.
• AI Voice Agents: AI Voice Agents have their own knowledge base configured in Agent Studio – separate from the conversation agent’s training. The principles for building effective knowledge are the same: specific controlled answers for high-priority questions alongside broader business context.
• Agent Studio: Agent Studio is where all agents are managed. Knowledge Base AI training is accessed through the Bot Training tab in Conversation AI settings – one of the configuration areas accessible from or linked to the Agent Studio workspace.

Common Questions

What is Knowledge Base AI in HighLevel?

The training system for the Conversation AI bot – two components in the Bot Training tab: the Web Crawler for indexing website content and Custom Bot Responses for specific Q&A pairs. Together they define what the bot knows and how accurately it responds.

Where do I set up the Knowledge Base AI in HighLevel?

Go to Settings, then Conversation AI in the sub-account. Click the Bot Training tab.

The Web Crawler and Custom Bot Responses sections are both here.

How does the HighLevel Web Crawler work?

Enter a website URL, choose to crawl by single URL, directory, or full domain, and click Crawl. HighLevel fetches and indexes the text content of specified pages.

Review and remove irrelevant pages after crawling to keep the knowledge base clean.

What are Custom Bot Responses in HighLevel?

Specific question-and-answer pairs written manually and added to the bot’s knowledge base. When an incoming message matches a custom question, the bot uses that specific answer.

Custom responses take priority over web crawl content for matched questions.

Which takes priority – Custom Bot Responses or the Web Crawler content?

Custom Bot Responses always take priority. When a match is detected, the bot returns the custom answer.

Web crawl content is the fallback for questions not covered by a custom entry.

How many Custom Bot Responses should I add?

Cover the 15 to 25 questions the business receives most frequently. Add multiple phrasing variations for the same question as separate entries.

Quality and coverage of high-frequency questions matters more than total entry count.

Can I add documents or PDFs to the HighLevel Knowledge Base AI?

The primary training inputs are the Web Crawler and Custom Bot Responses. For document-based content, convert key information into Custom Bot Response entries or publish it on a website page that can be crawled.

How do I know if the Knowledge Base is working correctly?

Use the Bot Trial tab to simulate conversations before activating on live channels. Test custom questions, phrasing variations, and questions the crawl should cover.

Each inaccurate response is a gap to fix before the bot goes live.

Does the Web Crawler update automatically when the website changes?

No. The crawl is not automatically refreshed. Manually re-crawl affected pages when significant content changes – new services, updated pricing, changed hours – to keep the bot’s knowledge current.

Can I use Knowledge Base AI for AI Voice Agents as well?

AI Voice Agents have their own knowledge base configured in Agent Studio – separate from the conversation agent’s Bot Training settings. The principles are the same but the configuration locations are different for each agent type.

To Wrap It Up

The knowledge base is where Conversation AI success or failure is determined.

Most people configure the persona, enable the channels, and go live without investing time in the training. The bot works – it responds to messages – but it produces variable and sometimes wrong answers because it is drawing from general AI knowledge rather than business-specific content.

Thirty minutes of focused knowledge base work changes that. A web crawl that covers the key service and information pages, plus 15 to 20 well-written custom Q&As for the questions that matter most, produces a bot that consistently handles the conversations it was deployed for.

The custom responses are where the real accuracy gains come from. The web crawl provides background.

Custom responses provide precision. A bot with 20 high-quality custom responses outperforms one with a thorough web crawl but no custom training for the specific questions that actually drive conversion or failure.

Here is how to get started:

1. Go to Settings, then Conversation AI, then Bot Training
2. Run the Web Crawler on the business website – start with the homepage and main service pages
3. Review the crawl results and remove irrelevant pages
4. Compile the 15 to 25 questions the business receives most frequently from real inbound sources
5. Add each as a Custom Bot Response using the customer’s natural phrasing
6. Add two to three additional phrasing variations for each high-priority question
7. Go to Bot Trial and test – send every question you just added and evaluate the responses
8. Fix every inaccurate response in Bot Trial before activating on any live channel
9. Set a quarterly reminder to re-crawl changed pages and update stale custom responses

The bot trial step is non-negotiable. Every inaccurate response you catch in testing is one your contacts will not receive.

The 20 minutes it takes to test thoroughly is the most valuable 20 minutes in the entire bot deployment process.