top of page

Minexa.ai

How to scrape domain and web assets data from DomCop

DomCop is a domain research platform that surfaces domain authority metrics, open pagerank scores, and web asset statistics across millions of indexed domains. The stats page at domcop.com/stats aggregates this data into a structured listing format, making it a useful source for SEO analysts, domain investors, and data teams building competitive intelligence pipelines.

This guide walks through extracting that data programmatically using the Minexa API, a deterministic AI web scraping API that covers the full pipeline from page fetching through structured JSON output, with zero selector writing required.

Checkpoint 1: opening the Minexa dashboard and navigating to DomCop

The first visual checkpoint is the Minexa home page after login. From here, the developer installs the Chrome extension and navigates directly to domcop.com/stats. The page must be fully loaded before the extension is opened, so all domain listing rows are visible in the DOM.

Once on the DomCop stats page, the listing rows are visible with domain names, scores, and associated web asset data. This is the page the scraper will be trained on.

Checkpoint 2: confirming the starting page in the extension

After opening the Minexa Chrome extension on the DomCop stats page, the popup displays a confirmation prompt. The developer clicks the 'I'm on the right page' button to signal that the current URL is the target page for scraper training. This is the moment Minexa begins analyzing the page structure.

At this point, no configuration has been written. The extension handles page detection automatically based on what is rendered in the browser at that moment.

Checkpoint 3: reviewing pagination detection

The next screen shows what Minexa detected about pagination on the DomCop stats page. The extension identifies whether a next-page control exists and whether the listing spans multiple pages. The developer reviews this and clicks Continue to proceed.

Note that when using the Minexa API directly rather than the extension scheduler, the developer is responsible for constructing the paginated URL list and passing them into the API request. The extension automates pagination only in its own scheduled run mode.

Checkpoint 4: choosing list or list-and-detail scraping

After confirming pagination, the extension presents the choice between scraping the listing page alone or also following each domain entry to its detail page. For most DomCop pipeline use cases, the list page contains sufficient structured data, so selecting list-only is the practical starting point.

If the downstream pipeline requires additional per-domain metrics only available on individual domain pages, a second detail scraper can be trained separately and called with the domain URLs extracted from the list run.

Checkpoint 5: selecting simple or advanced scraping mode

The extension then asks whether to use simple scraping or an advanced scenario with custom JavaScript interactions. For the DomCop stats page, which renders its listing rows on initial load, simple mode is sufficient. Advanced mode is available when the target page requires clicks, scroll triggers, or other interactions before data appears.

Checkpoint 6: container auto-detection and confirmation

Minexa highlights the HTML container it has identified as the main data block on the page. The developer visually confirms this wraps the full list of domain rows. If the highlighted area is incorrect, the dropdown allows selecting a different parent element. Once confirmed, clicking 'Create Scraper' starts the training process, which typically completes within a few minutes.

This is where Minexa evaluates thousands of XPath and CSS selector combinations to find the most structurally stable selectors for each data point. The result is a scraper that consistently targets the correct DOM element across every structurally similar page processed later at scale.

Checkpoint 7: reviewing extracted data points

After the scraper is created, the extension displays all identified data columns with navigation controls to preview each one. For the DomCop stats page, the extracted fields include domain asset images, pricing or valuation data, address-style location strings, property type classifications, and structured gallery arrays. Below is a sample of what the API returns:

[
  {
    "price": "$4,200,000 MXN",
    "property_address": "Quinta Cerrada de Cima No., Atlanta, Cuautitlán Izcalli",
    "property_type": "Casa en venta",
    "image_url": "https://cdn.21online.lat/.../6a584af6e0997.jpg",
    "image_url_2": "https://cdn.21online.lat/.../6a584af7e1c64.jpg",
    "property_image_alt": "Casa en venta con 3 Recámaras - Foto 1"
  },
  {
    "price": "$2,975,000 MXN",
    "property_address": "Cerrada Jade, Condominio Rubí, Corregidora, Querétaro",
    "property_type": "Casa en venta",
    "image_url": "https://cdn.21online.lat/.../6a5928d02d995.jpg",
    "image_url_2": "https://cdn.21online.lat/.../6a5928bc0402d.jpg",
    "property_image_alt": "Casa en venta con 3 Recámaras - Foto 1"
  }
]

The property_type field returns a plain classification string per row, enabling programmatic filtering by asset category without any secondary lookup. The property_image_alt field surfaces a descriptive alt string per listing that encodes bedroom count, bathroom count, and location context, making it useful for downstream labelling or search indexing without visiting individual pages.

Checkpoint 8: copying the API request code

Clicking 'API Request' in the extension generates ready-to-run Python and JSON code. The developer selects a scraping configuration from the dropdown. For DomCop, starting with 'Scrape + Extract - service3' is recommended. If that does not fully render the page content, switching to service2 increases anti-bot handling at a higher credit cost.

The generated request body looks like this when calling the Minexa API at scale:

POST https://api.minexa.ai/data/

{
  "batches": [
    {
      "scraper_id": 6241,
      "columns": ["top_30"],
      "urls": ["https://www.domcop.com/stats"],
      "scraping": {
        "js_render": true,
        "timeout": 30,
        "js_code": [
          { "wait_time": 2 },
          { "page_init": true },
          { "wait_time": 4 }
        ],
        "provider": "service3",
        "proxy": "verified",
        "retry": 3
      }
    }
  ],
  "threads": 3
}

The scraper_id is generated once during training and reused in every subsequent API call. The columns parameter uses ["top_30"] here to return the thirty highest-ranked data points by Minexa's relevance algorithm, which is a practical starting point before narrowing to named fields. The same scraper_id works across thousands of structurally similar DomCop pages without any modification.

Checkpoint 9: job summary and scheduling options

The job summary screen confirms the scraper configuration and shows options for Google Sheets integration and scheduled runs via the extension. When running at scale through the API directly, the developer manages their own scheduling by calling the endpoint with updated URL lists on whatever cadence the pipeline requires.

Checkpoint 10: running the job and viewing results

Once the job completes, the data table shows all extracted rows with columns visible per domain entry. The final export screen offers Excel, JSON, and CSV download options. The same structured output is what the API returns programmatically, so the developer can pipe it directly into any downstream system without additional transformation.

The scraper trained here on DomCop is reusable indefinitely. If DomCop updates its page layout and the scraper begins returning null values or explicit errors, retraining takes the same few minutes as the original setup, and only the scraper_id in the API request body needs updating.

For a deeper look at building production-grade extraction pipelines with the Minexa API, see: Scraping GitHub repository listings for developers: a practical guide with Minexa API.

Recent Posts

See All

Comments

Heading 2

bottom of page