> ## Documentation Index
> Fetch the complete documentation index at: https://help.lobstr.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Export scraped data automatically to Amazon S3

> Attach an AWS S3 bucket to your Squid so every completed run uploads its results there automatically — including how to create the bucket, the IAM policy, and the access keys.

If your data pipeline already reads from S3, you can skip the download-and-upload step entirely. Attach a bucket to your Squid and lobstr.io writes the results there itself every time a run completes.

Most of the work is on the AWS side: a bucket, a narrow permissions policy, and a dedicated user whose keys you paste into lobstr.io. The lobstr.io part is three fields.

## Before you start

* A [lobstr.io account](https://app.lobstr.io) with a Squid you've already configured.
* An AWS account. If you don't have one, [sign up with AWS](https://portal.aws.amazon.com/billing/signup) first.

## Part 1 — Set up AWS

<Steps>
  <Step title="Create an S3 bucket">
    Open the [S3 console](https://s3.console.aws.amazon.com/s3/buckets) and click **Create bucket**. Set the region using the dropdown in the top-right corner before you start.

    Enter a globally unique **bucket name** using only lowercase letters, numbers, and hyphens.

    <Frame>
      <img src="https://mintcdn.com/lobstrio-8dcae32c/lanZnXBNnxHNeAbm/images/integrations/amazon-s3/create-bucket.png?fit=max&auto=format&n=lanZnXBNnxHNeAbm&q=85&s=b0398fd2a9eb4e77fe71c711b357b58d" alt="Create an S3 bucket with a globally unique name in the AWS console" width="1882" height="756" data-path="images/integrations/amazon-s3/create-bucket.png" />
    </Frame>

    Leave everything else at its defaults — **Bucket type** on General purpose, **Bucket namespace** on Global, and **Block all public access** switched on. Click **Create bucket**.

    <Tip>
      Avoid dots in the bucket name. They're technically allowed, but they break TLS certificate matching for virtual-hosted-style requests and cause problems that are annoying to diagnose later.
    </Tip>
  </Step>

  <Step title="Create a permissions policy">
    Go to **IAM** → **Policies** → **Create policy**. Switch the editor to the **JSON** tab and replace the contents with this, substituting your own bucket name:

    ```json theme={null}
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": "s3:PutObject",
          "Resource": "arn:aws:s3:::your-bucket-name/*"
        }
      ]
    }
    ```

    <Frame>
      <img src="https://mintcdn.com/lobstrio-8dcae32c/lanZnXBNnxHNeAbm/images/integrations/amazon-s3/IAM_policy.png?fit=max&auto=format&n=lanZnXBNnxHNeAbm&q=85&s=fe06f321529067f1d648df2168d292e7" alt="Paste the S3 PutObject policy into the IAM JSON policy editor" width="1869" height="675" data-path="images/integrations/amazon-s3/IAM_policy.png" />
    </Frame>

    Click **Next**, name it `lobstr-delivery-write`, and click **Create policy**.

    This grants permission to write files to that one bucket and nothing else — it can't read, can't delete, and can't touch any other bucket in your account.
  </Step>

  <Step title="Create a dedicated IAM user">
    In IAM, go to **Users** and click **Create user**. Name it `lobstr-delivery` and leave the console-access checkbox unticked — this user authenticates with keys only.

    <Frame>
      <img src="https://mintcdn.com/lobstrio-8dcae32c/lanZnXBNnxHNeAbm/images/integrations/amazon-s3/add_iam_useer.png?fit=max&auto=format&n=lanZnXBNnxHNeAbm&q=85&s=9f3b777bb1fb4821346fc92df99f9fed" alt="Create an IAM user without AWS Management Console access" width="1905" height="655" data-path="images/integrations/amazon-s3/add_iam_useer.png" />
    </Frame>

    On the next screen, choose **Attach policies directly**, tick `lobstr-delivery-write`, then click through and create the user.

    <Frame>
      <img src="https://mintcdn.com/lobstrio-8dcae32c/lanZnXBNnxHNeAbm/images/integrations/amazon-s3/set_iam_user_permissions.png?fit=max&auto=format&n=lanZnXBNnxHNeAbm&q=85&s=466dc4e4d99878ae259221eb16cfe252" alt="Attach the lobstr-delivery-write policy directly to the new IAM user" width="1851" height="638" data-path="images/integrations/amazon-s3/set_iam_user_permissions.png" />
    </Frame>
  </Step>

  <Step title="Generate access keys">
    Open the new user and click **Create access key** in the **Access key 1** panel.

    <Frame>
      <img src="https://mintcdn.com/lobstrio-8dcae32c/lanZnXBNnxHNeAbm/images/integrations/amazon-s3/create-access-key-1.png?fit=max&auto=format&n=lanZnXBNnxHNeAbm&q=85&s=4610abe6bdd3d21e875a0557c7edbdf2" alt="Create access key from the IAM user summary panel" width="1883" height="774" data-path="images/integrations/amazon-s3/create-access-key-1.png" />
    </Frame>

    Choose **Application running outside AWS**, skip the description tag, and click **Create access key**.

    <Frame>
      <img src="https://mintcdn.com/lobstrio-8dcae32c/lanZnXBNnxHNeAbm/images/integrations/amazon-s3/create-access-key-2.png?fit=max&auto=format&n=lanZnXBNnxHNeAbm&q=85&s=cbd4aa7d5579bbf3690f042d5cbe3d63" alt="Select Application running outside AWS as the access key use case" width="1512" height="686" data-path="images/integrations/amazon-s3/create-access-key-2.png" />
    </Frame>

    Copy both the **Access key** and the **Secret access key** now.

    <Frame>
      <img src="https://mintcdn.com/lobstrio-8dcae32c/lanZnXBNnxHNeAbm/images/integrations/amazon-s3/create-access-key-3.png?fit=max&auto=format&n=lanZnXBNnxHNeAbm&q=85&s=f031312da0840da5adf6bf707ff0cbc7" alt="Retrieve the access key and secret access key on the final screen" width="1856" height="711" data-path="images/integrations/amazon-s3/create-access-key-3.png" />
    </Frame>

    <Warning>
      The secret access key is shown **only on this screen** and can't be retrieved afterwards. Copy it now, or download the `.csv`. If you lose it, you'll have to create a new access key and deactivate the old one.
    </Warning>
  </Step>
</Steps>

## Part 2 — Connect it to your Squid

<Steps>
  <Step title="Add S3 as a delivery method">
    In your lobstr.io dashboard, open the Squid, go to its **Delivery** settings, and expand the **Amazon s3** section.

    Enter the bucket name in **Bucket**, then paste the access key and secret key into their fields.

    <Frame>
      <img src="https://mintcdn.com/lobstrio-8dcae32c/lanZnXBNnxHNeAbm/images/integrations/amazon-s3/s3_setup_lobstr_app.png?fit=max&auto=format&n=lanZnXBNnxHNeAbm&q=85&s=0dc58ea867acd441eee1dcf736e697a7" alt="Fill in the bucket, AWS access key, and AWS secret key in the Amazon S3 delivery settings" width="632" height="746" data-path="images/integrations/amazon-s3/s3_setup_lobstr_app.png" />
    </Frame>

    <Note>
      Enter the bucket name on its own — `my-bucket`, not the full `s3://my-bucket` URL.
    </Note>
  </Step>

  <Step title="Test the connection">
    Click **Test Amazon S3** before saving. A pass confirms the credentials and permissions are correct.

    If it fails with an access-denied error, the bucket name in the policy ARN doesn't match the bucket name in the form.
  </Step>

  <Step title="Enable and save">
    Tick the checkbox beside **Amazon s3** to activate the integration, then click **Save**.

    <Warning>
      Filled-in credentials with the box left unticked means deliveries won't fire. The fields and the checkbox are separate — you need both.
    </Warning>
  </Step>

  <Step title="Run the Squid and verify">
    Click **Launch**. When the run completes, open the bucket in the S3 console — the results file appears there within about a minute.

    Files land at the **root of the bucket**, named after the Squid plus the run's UTC timestamp:

    ```
    Google Maps Reviews Scraper (12)_2026-09-03T16:03:06.899628+00:00.csv
    ```

    <Frame>
      <img src="https://mintcdn.com/lobstrio-8dcae32c/lanZnXBNnxHNeAbm/images/integrations/amazon-s3/s3-received-csv.png?fit=max&auto=format&n=lanZnXBNnxHNeAbm&q=85&s=0e1da79fba1db7a414cc4f16a17e9689" alt="The results CSV delivered to the root of the S3 bucket after a run completes" width="1912" height="572" data-path="images/integrations/amazon-s3/s3-received-csv.png" />
    </Frame>

    Every subsequent run drops a new file automatically, including scheduled ones. Nothing is overwritten — the timestamp keeps each run's file distinct.
  </Step>
</Steps>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Access denied">
    The bucket name in the policy ARN doesn't match the bucket you entered in lobstr.io. Open the policy in IAM and check the `Resource` line — it needs your real bucket name and must end in `/*`.
  </Accordion>

  <Accordion title="No such bucket">
    Either the bucket name is mistyped, or the bucket is in a different region than the one you're looking at. Bucket names are global, so a typo can also mean you're pointing at a bucket that isn't yours.
  </Accordion>

  <Accordion title="Invalid credentials">
    Usually the secret key was copied with trailing whitespace. Re-paste both values carefully. If the secret is lost, create a new access key in IAM and deactivate the old one.
  </Accordion>

  <Accordion title="The test passes but no files appear">
    Check the **Amazon s3** checkbox is ticked and that you clicked **Save**. A successful test doesn't activate the delivery on its own.
  </Accordion>
</AccordionGroup>

## Things to know

* **Delivery applies to future runs only.** Runs that finished before you set this up aren't uploaded retroactively. To backfill, use the green **Download** button to grab the combined CSV and upload it yourself.
* **S3 delivery is per Squid.** Configure it separately on each Squid you want writing to a bucket.
* **You can combine delivery channels.** Amazon S3, [Google Sheets](/integrations/google-sheets), SFTP, email, and webhooks each have their own checkbox in the Delivery panel and can all be active at the same time.
* **Rotate the key if it's ever exposed.** Create a new access key in IAM, paste it into the Delivery panel, save, then deactivate the old key. The narrow policy means the worst case is writes to one bucket — which is exactly why the dedicated user is worth the extra two minutes.
