How do I add content from my website or sitemap?

Live site content is some of your most accurate, up-to-date material. Importing a sitemap into a Knowledge Base gives your AI tools direct access to it, so every workflow pulling from that base reflects your live site. Automatic syncing keeps it current without manual effort.

Key takeaways

  1. A sitemap gives AirOps a structured list of your website pages, making your content searchable by AI tools inside a Knowledge Base.
  2. You can import all pages, select specific ones, or filter by URL pattern, and for sites with multiple sitemaps, use sitemap_index.xml as the source URL.
  3. Scheduled automatic re-syncs keep your Knowledge Base up to date without manual effort.
  4. JavaScript-rendered and login-gated pages won't import correctly, so stick to publicly accessible, server-rendered content.
  5. The "Scrape Metadata Only" toggle imports titles and descriptions without full-page content, useful when you need lightweight reference data.

Tutorial overview

Your website holds some of your most current, brand-accurate content. Importing a sitemap into a Knowledge Base pulls all of it into AirOps at once, making every page searchable and available to power your workflows. Instead of adding pages one by one, you import a full sitemap in a few clicks.

The process is straightforward. Open your Knowledge Base, select Add Data, choose Web Scrape, and pick Multiple Pages via Sitemap. Paste your sitemap URL, select the pages you want, and save. AirOps fetches the content from each URL and indexes it for semantic search.

You can schedule recurring syncs on an hourly, daily, weekly, or monthly cadence so your Knowledge Base stays current as you publish new content. There's also a Scrape Metadata Only toggle if you only need titles and descriptions rather than full page content. A few things to keep in mind: use sitemap_index.xml if your site splits sitemaps across multiple files, and test a small batch first if your site uses JavaScript rendering or gated content.

Practice

  1. Open Your Knowledge Base and Click Add Data

    The Add Data button in AirOps Knowledge Bases opens content ingestion options. Navigating to a Knowledge Base in the left sidebar and clicking Add Data is the starting point for importing sitemap content.

  2. View the Add Data Options (Including Web Page)

    The Add Data dropdown in AirOps Knowledge Bases lists all supported ingestion methods, including Web Page. It opens from the Add Data button in the top-right corner or the empty state of the Files tab. Select Web Page to add content from a single URL: enter the URL in the dialog and confirm to begin scraping and indexing the page.

  3. Configure and Start a Sitemap Import

    The Sitemap option in the AirOps Add Data dropdown imports an entire website at once using a sitemap file. Sitemaps list every page on the site, enabling AirOps to process all pages automatically without individual URL entry. Enter a sitemap.xml URL in the Sitemap URL field and use filter options to scope the import to specific sections or URL patterns. Toggle Scrape Metadata Only to limit the import to page titles and descriptions rather than full page content, then click Start Scraping to begin the import process.

  4. Monitor the Sitemap Import Progress

    The sitemap import queue in AirOps shows each page URL as it is processed and added to the Knowledge Base, updating in real time as pages are indexed. The import progress view displays the total number of pages being ingested and their current processing status. The sync runs in the background after initiating. You can navigate to other parts of the platform while the import completes without interrupting the process.