> ## Documentation Index
> Fetch the complete documentation index at: https://help.lobstr.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Re-running a Squid: duplicates, Unique Results and credits

> What happens when a lobstr.io Squid runs again: which results come back, what Unique Results really removes, what gets billed again, and how to collect only new results.

Every run of a Squid processes its whole [task list](/core-concepts/tasks) from scratch. So when you launch the same Squid again, manually or on a [schedule](/core-concepts/scheduling), results collected by earlier runs can come back.

This page explains what **Unique Results** removes, what you pay for again, and how to set up a Squid that only collects new results.

## What Unique Results does

**Unique Results** is a toggle in your Squid's **Settings**, under **Advanced Settings**.

<Frame>
  <img src="https://mintcdn.com/lobstrio-8dcae32c/8Hff-_iT1Cf92GQx/images/core-concepts/repeat-runs/unique_results_toggle.png?fit=max&auto=format&n=8Hff-_iT1Cf92GQx&q=85&s=87e20395f4bd278467031e7a2107bd96" alt="Unique Results toggle switched on in the Advanced Settings of a Squid" width="1766" height="494" data-path="images/core-concepts/repeat-runs/unique_results_toggle.png" />
</Frame>

It removes duplicates **inside a run**. If the same result is returned by several tasks of one run (for example two overlapping searches), it appears only once in that run's results and export.

It does **not** skip results collected by previous runs. A listing collected on Monday that is still online on Tuesday will also be in Tuesday's run.

<Note>
  On or off, a result is billed **once per run**, even if several tasks of that run return it. The toggle only changes how many times it appears in the export.
</Note>

## What gets billed again

A result that comes back in a later run is a new row for that run, so its base cost is billed again. Most enrichment functions work differently: when this Squid has already run the function on a result, it skips that result on later runs and doesn't bill it again.

**Why is it billed again?** Even if a result is a duplicate for you, lobstr.io fetches it again from the website on every run, and that costs real resources: compute, proxies, storage and captcha solving.

| | Same run | Later run of the same Squid |
| - | - | - |
| **Base cost** (collecting the result) | Billed once | Billed again |
| **Enrichment function** already run on this result by this Squid (most functions) | Billed once | Skipped, not billed |

**Example: Facebook Marketplace Scraper with Get Listing Details on.** A new listing costs 1 credit for the listing plus 4 credits for its details. A listing this Squid already collected and enriched costs 1 credit when it comes back: its details are not fetched or billed again.

<Tip>
  Each function shows its price in your Squid's settings. To see what a run actually spent on base rows vs. each function, open the run's [Credits tab](/core-concepts/credits#per-run-the-credits-tab-on-a-run).
</Tip>

## Max Unique Results counts results from earlier runs

**Max Unique Results** caps the number of unique results in a run. A result collected by an earlier run still counts toward that cap.

<Frame>
  <img src="https://mintcdn.com/lobstrio-8dcae32c/6ihPF5ZmeylxZIOP/images/core-concepts/repeat-runs/max_unique_results.png?fit=max&auto=format&n=6ihPF5ZmeylxZIOP&q=85&s=34e58bb43ccfeec17cb1d06f820e9f56" alt="Max Unique Results field in the Advanced Settings of a Squid, with its tooltip open" width="1600" height="354" data-path="images/core-concepts/repeat-runs/max_unique_results.png" />
</Frame>

So if your search returns the same results in the same order every time, a re-run can fill the cap with results you already have. Combine it with **Newer than** (below) so the cap is only filled with recent results.

## How to collect only new results

### Use the Newer than setting

Many scrapers have a **Newer than** setting in their Squid settings. It stops the run once it reaches results older than the threshold you set.

<Steps>
  <Step title="Open Squid Settings">
    Navigate to your Squid and click **Settings**.
  </Step>

  <Step title="Set Newer than">
    Choose **Relative**, then enter a duration that matches your schedule: `24` hours for a Squid that runs every day, `168` hours for a weekly one.
  </Step>

  <Step title="Schedule the Squid">
    Under **Launch**, click **Repeatedly** and pick the same interval. See [Scheduling](/core-concepts/scheduling).
  </Step>
</Steps>

<Frame>
  <img src="https://mintcdn.com/lobstrio-8dcae32c/8Hff-_iT1Cf92GQx/images/core-concepts/repeat-runs/newer_than_setting.png?fit=max&auto=format&n=8Hff-_iT1Cf92GQx&q=85&s=b94bc3f8b232a14efd2228d00979b912" alt="Newer than setting in a Squid, set to a relative duration in hours" width="1666" height="260" data-path="images/core-concepts/repeat-runs/newer_than_setting.png" />
</Frame>

### Filter the search on the website

When a scraper has no **Newer than** setting, check whether the website's search has its own date filter, such as "listed in the last 24 hours". If it does, apply it on the website, copy the URL from your browser, and add it as a task in your Squid. Then schedule the Squid with the same interval.

### Use the scraper's skip option

Two scrapers can skip everything the Squid has already collected, so each run returns only new results. Turn the option on in your Squid's settings:

* **Sales Navigator Leads Scraper**: [Skip Collected Leads](/guides/sales-navigator/skip-collected-leads)
* **LinkedIn Leads Scraper**: **Only New Leads**

<Frame>
  <img src="https://mintcdn.com/lobstrio-8dcae32c/TloeTGFix98ENRqm/images/guides/sales-navigator/skip-collected-leads/toggle.png?fit=max&auto=format&n=TloeTGFix98ENRqm&q=85&s=87b8230da7674f29d2d43d6287cc86ab" alt="The Skip Collected Leads toggle in the Sales Navigator Leads Scraper settings" width="824" height="726" data-path="images/guides/sales-navigator/skip-collected-leads/toggle.png" />
</Frame>

## Related

* [Tasks](/core-concepts/tasks): every run re-processes every task
* [Scheduling](/core-concepts/scheduling): run a Squid automatically
* [Credits](/core-concepts/credits): what credits buy and how to track them
* [Resurrect a run](/core-concepts/resurrect-runs): rows re-scraped by a resurrected run aren't billed again


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.