> ## Documentation Index
> Fetch the complete documentation index at: https://www.octogen.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Subscribe to catalogs in BigQuery

> Create linked BigQuery datasets for your granted Octogen catalogs, either one listing at a time or automatically from cron.

For analytical and bulk workloads, Octogen can publish a granted catalog as a [BigQuery Analytics Hub](https://cloud.google.com/bigquery/docs/analytics-hub-introduction) **listing**. Subscribing to the listing creates a **linked dataset** in your Google Cloud project, so you can query the full catalog with standard SQL while Octogen updates flow through automatically.

Use BigQuery when you want to join Octogen catalog data against your own data, run large scans, or build analytics pipelines. Use the [Catalog Search API](/docs/guides/catalog-search) or [MCP](/docs/mcp/overview) for interactive product lookup and agent workflows.

<Note>
  BigQuery subscription uses your **Google Cloud** credentials and your destination GCP project. It does not use `OCTO_API_KEY`. The helper commands are part of the Python SDK's optional `bigquery` extra.
</Note>

## How access works

BigQuery subscription is a two-party flow:

1. **Octogen grants listing access.** Octogen publishes the catalog as an Analytics Hub listing and grants your Google principal `roles/analyticshub.subscriber` on that listing.
2. **You create the linked dataset.** You run the subscription step from your own environment using Google Application Default Credentials. The linked dataset is created in your GCP project and billed to you.
3. **Octogen observes the subscription.** After the linked dataset exists, Octogen refreshes the subscription status to `active`.

The grant is asynchronous. It can take a few minutes for a listing to move from `pending` to `awaiting_subscription`.

## Prerequisites

<Steps>
  <Step title="Ask Octogen to enable BigQuery listing access">
    Your Octogen contact must enable BigQuery listing access for your organization.
  </Step>

  <Step title="Choose a subscriber principal">
    Choose the Google principal that will create linked datasets. For automation, use a service account such as:

    ```text theme={null}
    serviceAccount:bq-reader@my-gcp-project.iam.gserviceaccount.com
    ```
  </Step>

  <Step title="Install the Python SDK with the BigQuery extra">
    The SDK ships from the public [`octogen-dev`](https://github.com/octogen-ai/octogen-dev) repository and is managed with [uv](https://docs.astral.sh/uv/). Clone the repository and sync the `sdks/python` project with the `bigquery` extra:

    ```bash theme={null}
    git clone https://github.com/octogen-ai/octogen-dev.git
    cd octogen-dev
    uv sync --project sdks/python --extra bigquery
    ```

    This installs `octogen-bq-subscribe`, `octogen-bq-autosubscribe`, and `octogen-mcp-login` into the project environment. Confirm the install:

    ```bash theme={null}
    uv run --project sdks/python --extra bigquery octogen-bq-subscribe --help
    ```
  </Step>

  <Step title="Authenticate to Google Cloud">
    For local testing:

    ```bash theme={null}
    gcloud auth application-default login
    ```

    In production, run with service account credentials that can create linked datasets in the destination project.
  </Step>
</Steps>

<Note>
  Run the `octogen-bq-*` and `octogen-mcp-login` commands below from the `octogen-dev` repository. Either activate the synced environment once with `source sdks/python/.venv/bin/activate` so the commands are on your `PATH` exactly as written, or prefix each one with `uv run --project sdks/python --extra bigquery`.
</Note>

## Manual subscription

Use `octogen-bq-subscribe` when you already have a listing resource name and want to subscribe one catalog.

<Steps>
  <Step title="Get the listing resource">
    Use the Platform UI or the MCP `list_bigquery_listing_resources` tool. A listing resource looks like:

    ```text theme={null}
    projects/octogen-catalog-exchange/locations/US/dataExchanges/catalogs_prod/listings/catalog_farfetch_exported_product_view_v1
    ```
  </Step>

  <Step title="Preview the subscription">
    The command is a dry run by default.

    ```bash theme={null}
    octogen-bq-subscribe \
      --listing projects/octogen-catalog-exchange/locations/US/dataExchanges/catalogs_prod/listings/catalog_farfetch_exported_product_view_v1 \
      --project my-gcp-project
    ```
  </Step>

  <Step title="Apply">
    ```bash theme={null}
    octogen-bq-subscribe \
      --listing projects/octogen-catalog-exchange/locations/US/dataExchanges/catalogs_prod/listings/catalog_farfetch_exported_product_view_v1 \
      --project my-gcp-project \
      --apply
    ```

    Re-running is safe. If the linked dataset already exists, the command reports the existing subscription instead of creating a duplicate.
  </Step>

  <Step title="Query the linked dataset">
    ```sql theme={null}
    SELECT *
    FROM `my-gcp-project.octogen_catalog_farfetch_exported_product_view_v1.products_current_v1`
    LIMIT 10;
    ```
  </Step>
</Steps>

### Manual CLI options

| Flag              | Required | Default                     | Description                                                              |
| ----------------- | -------- | --------------------------- | ------------------------------------------------------------------------ |
| `--listing`       | Yes      | —                           | Analytics Hub listing resource name.                                     |
| `--project`       | Yes      | —                           | Destination GCP project for the linked dataset.                          |
| `--dataset`       | No       | Derived from the listing id | Linked dataset id.                                                       |
| `--location`      | No       | Listing location            | BigQuery dataset location.                                               |
| `--friendly-name` | No       | Derived from the dataset id | Linked dataset display name.                                             |
| `--apply`         | No       | Off                         | Create the linked dataset. Without it, the command prints the plan only. |
| `--json`          | No       | Off                         | Emit JSON output for automation.                                         |

## Automatic subscription from cron

Use `octogen-bq-autosubscribe` when you want automation that polls for new BigQuery listings and subscribes to anything ready.

The command performs one idempotent sync pass:

1. Lists BigQuery listings granted to your organization.
2. Lists your registered BigQuery subscribers.
3. Registers your Reader if it is missing.
4. Subscribes every catalog whose status is `awaiting_subscription`.
5. Refreshes Octogen status after each linked dataset is created.

### Bootstrap MCP OAuth credentials

`octogen-bq-autosubscribe` calls Octogen MCP to list and register subscriber state. Bootstrap MCP credentials once from an interactive terminal:

```bash theme={null}
octogen-mcp-login \
  --client-id-file /secure/octogen-mcp.client-id \
  --redirect-uri-file /secure/octogen-mcp.redirect-uri \
  --refresh-token-file /secure/octogen-mcp.refresh
```

The login command opens Octogen AuthKit in your browser, registers a public MCP OAuth client if needed, requests an MCP-audience refresh token, and writes the client id, registered redirect URI, and rotating refresh token to the requested files. It does not print the refresh token.

### Dry run

```bash theme={null}
OCTOGEN_MCP_CLIENT_ID_FILE=/secure/octogen-mcp.client-id \
OCTOGEN_MCP_REFRESH_TOKEN_FILE=/secure/octogen-mcp.refresh \
octogen-bq-autosubscribe \
  --project my-gcp-project \
  --principal serviceAccount:bq-reader@my-gcp-project.iam.gserviceaccount.com \
  --json
```

### Apply from cron

```cron theme={null}
*/15 * * * * cd /path/to/octogen-dev && OCTOGEN_MCP_CLIENT_ID_FILE=/secure/octogen-mcp.client-id OCTOGEN_MCP_REFRESH_TOKEN_FILE=/secure/octogen-mcp.refresh uv run --project sdks/python --extra bigquery octogen-bq-autosubscribe --project my-gcp-project --principal serviceAccount:bq-reader@my-gcp-project.iam.gserviceaccount.com --apply --json
```

`OCTOGEN_MCP_REFRESH_TOKEN_FILE` should contain an Octogen MCP OAuth refresh token. If the identity provider rotates that refresh token during exchange, the command writes the replacement back to the same file while holding a sibling `.lock` file, so overlapping cron runs do not overwrite the same rotating token at once.

### Autosubscribe options

| Flag                       | Required | Default                          | Description                                                                |
| -------------------------- | -------- | -------------------------------- | -------------------------------------------------------------------------- |
| `--project`                | Yes      | —                                | Destination GCP project for linked datasets.                               |
| `--principal`              | Yes      | —                                | Google subscriber principal Octogen should grant.                          |
| `--catalog`                | No       | All granted catalogs             | Restrict the run to one catalog. Repeat for multiple catalogs.             |
| `--friendly-name`          | No       | Derived per dataset              | Friendly name for created linked datasets.                                 |
| `--apply`                  | No       | Off                              | Register and subscribe. Without it, the command reports what would happen. |
| `--json`                   | No       | Off                              | Emit JSON output.                                                          |
| `--mcp-url`                | No       | `https://mcp.octogen.ai/mcp/`    | Octogen MCP endpoint.                                                      |
| `--mcp-client-id-file`     | No       | `OCTOGEN_MCP_CLIENT_ID_FILE`     | File containing the public OAuth client id from `octogen-mcp-login`.       |
| `--mcp-refresh-token-file` | No       | `OCTOGEN_MCP_REFRESH_TOKEN_FILE` | File containing the rotating MCP refresh token.                            |
| `--mcp-token-command`      | No       | `OCTOGEN_MCP_TOKEN_COMMAND`      | Command that prints a short-lived MCP access token.                        |
| `--mcp-access-token`       | No       | `OCTOGEN_MCP_ACCESS_TOKEN`       | Short-lived MCP access token for manual runs.                              |

## MCP tools behind the automation

Agents and custom clients can implement the same flow through MCP tools:

| Tool                                   | Purpose                                                                                      |
| -------------------------------------- | -------------------------------------------------------------------------------------------- |
| `list_bigquery_listing_resources`      | Lists BigQuery listings granted to your organization.                                        |
| `list_bigquery_subscribers`            | Lists registered subscriber projects, principals, and per-catalog status cells.              |
| `register_bigquery_subscriber`         | Registers the GCP project and principal that should receive Analytics Hub subscriber access. |
| `refresh_bigquery_subscription_status` | Checks whether a linked dataset subscription exists and updates the Octogen status.          |
| `delete_bigquery_subscriber`           | Disables a registered subscriber.                                                            |

Most customers should use `octogen-bq-autosubscribe` instead of calling these tools directly. It handles idempotency, pending grants, existing subscriptions, and status refreshes for you.

## Programmatic subscription

`subscribe_to_listing` is a synchronous Python helper that talks to Google Cloud using Application Default Credentials.

```python theme={null}
from octogen_ai_sdk import (
    OctogenBigQueryAccessPendingError,
    OctogenBigQueryError,
    subscribe_to_listing,
)

try:
    result = subscribe_to_listing(
        listing_resource="projects/octogen-catalog-exchange/locations/US/dataExchanges/catalogs_prod/listings/catalog_farfetch_exported_product_view_v1",
        destination_project="my-gcp-project",
    )
except OctogenBigQueryAccessPendingError:
    print("Access is not ready yet. Retry in a few minutes.")
    raise
except OctogenBigQueryError as exc:
    print(f"Subscribe failed: {exc}")
    raise

verb = "Already subscribed" if result.already_subscribed else "Subscribed"
print(f"{verb}: {result.linked_project}.{result.linked_dataset} [{result.state}]")
print(result.sample_query)
```

## Result statuses

`octogen-bq-autosubscribe --json` returns one catalog result per granted or requested catalog.

| Action               | Meaning                                                                                      |
| -------------------- | -------------------------------------------------------------------------------------------- |
| `already_active`     | Octogen already sees an active linked dataset subscription.                                  |
| `already_subscribed` | BigQuery reported an existing linked dataset, and Octogen status was refreshed.              |
| `subscribed`         | The command created a linked dataset and refreshed Octogen status.                           |
| `would_subscribe`    | Dry run: the catalog is ready and would be subscribed with `--apply`.                        |
| `pending`            | The Reader or listing grant is still being prepared, or Octogen has not observed status yet. |
| `skipped`            | The catalog is not ready or not enabled for this subscriber.                                 |
| `error`              | The catalog needs attention or the subscribe/refresh step failed.                            |

## Error handling

| Exit code | Command                | Meaning                                                 |
| --------- | ---------------------- | ------------------------------------------------------- |
| `0`       | Both                   | Success, dry run, or idempotent no-op.                  |
| `1`       | Both                   | Subscribe, MCP, or Google Cloud operation failed.       |
| `2`       | Both                   | Usage or invalid arguments.                             |
| `3`       | `octogen-bq-subscribe` | Analytics Hub access is not granted yet. Retry shortly. |

<Tip>
  Both commands are idempotent. Schedule `octogen-bq-autosubscribe --apply` as often as you need; completed catalogs report `already_active` and new ready catalogs are picked up automatically.
</Tip>