Website Crawl Errors

Open Training → Knowledge, then the website source. Find the page marked Action needed and open it for the reason.

Check the reported problem

  • The page cannot be opened: verify the address and whether it redirects or requires a login.
  • A firewall blocks access: ask your website administrator to inspect the blocked request. Do not disable protection for the whole website just to train a page.
  • The website could not connect to its database: ask your website administrator to restore the database connection. During background training, Boei tries twice more. If the problem continues, training stops for the remaining pages of that website. After the website works again, retry the affected pages.
  • No readable content: check that the page actually contains useful text, or add that content as a file or text source.
  • Search engine results cannot be used as a knowledge source: add the original page address instead. Search links in a bulk import stay visible with this reason. If a previously trained search page is retried, Boei removes its old knowledge and frees its training space.
  • The address is a sitemap: add it as a website source for discovery rather than treating the XML itself as knowledge.
  • Not enough training space: free space or upgrade, then resume. This is a capacity problem, not a website error.
  • Training stopped or only partly succeeded: inspect the details, then retry the affected content.

Retry carefully

After correcting the cause, retry one affected page first. Check its extracted content and try a question with Test chatbot. If that succeeds, retry the remaining affected selection.

A discovered address is not automatically usable knowledge. Wait for the ready status before expecting the chatbot to answer from it. Existing usable content remains available during replacement processing.

Large or slow sites

Training runs in the background and is paced per website. A large site can take longer even when Boei has spare workers, because repeatedly requesting the same host too quickly can cause blocks or overload. Leave the run processing and return to its progress, or use the completion-email option.

If a problem persists, contact support with the page address, the displayed error and when it happened. Do not include passwords or private access tokens.