Subscribe to catalogs in BigQuery

For analytical and bulk workloads, Octogen can publish a granted catalog as a BigQuery Analytics Hub listing. Subscribing to the listing creates a linked dataset in your Google Cloud project, so you can query the full catalog with standard SQL while Octogen updates flow through automatically. Use BigQuery when you want to join Octogen catalog data against your own data, run large scans, or build analytics pipelines. Use the Catalog Search API or MCP for interactive product lookup and agent workflows.

BigQuery subscription uses your Google Cloud credentials and your destination GCP project. It does not use OCTO_API_KEY. The helper commands are part of the Python SDK’s optional bigquery extra.

How access works

BigQuery subscription is a two-party flow:

Octogen grants listing access. Octogen publishes the catalog as an Analytics Hub listing and grants your Google principal roles/analyticshub.subscriber on that listing.
You create the linked dataset. You run the subscription step from your own environment using Google Application Default Credentials. The linked dataset is created in your GCP project and billed to you.
Octogen observes the subscription. After the linked dataset exists, Octogen refreshes the subscription status to active.

The grant is asynchronous. It can take a few minutes for a listing to move from pending to awaiting_subscription.

Prerequisites

Ask Octogen to enable BigQuery listing access

Your Octogen contact must enable BigQuery listing access for your organization.

Choose a subscriber principal

Choose the Google principal that will create linked datasets. For automation, use a service account such as:

serviceAccount:bq-reader@my-gcp-project.iam.gserviceaccount.com

Install the Python SDK with the BigQuery extra

The SDK ships from the public octogen-dev repository and is managed with uv. Clone the repository and sync the sdks/python project with the bigquery extra:

git clone https://github.com/octogen-ai/octogen-dev.git
cd octogen-dev
uv sync --project sdks/python --extra bigquery

This installs octogen-bq-subscribe, octogen-bq-autosubscribe, and octogen-mcp-login into the project environment. Confirm the install:

uv run --project sdks/python --extra bigquery octogen-bq-subscribe --help

Authenticate to Google Cloud

For local testing:

gcloud auth application-default login

In production, run with service account credentials that can create linked datasets in the destination project.

Run the octogen-bq-* and octogen-mcp-login commands below from the octogen-dev repository. Either activate the synced environment once with source sdks/python/.venv/bin/activate so the commands are on your PATH exactly as written, or prefix each one with uv run --project sdks/python --extra bigquery.

Manual subscription

Use octogen-bq-subscribe when you already have a listing resource name and want to subscribe one catalog.

Get the listing resource

Use the Platform UI or the MCP list_bigquery_listing_resources tool. A listing resource looks like:

projects/octogen-catalog-exchange/locations/US/dataExchanges/catalogs_prod/listings/catalog_farfetch_exported_product_view_v1

Preview the subscription

The command is a dry run by default.

octogen-bq-subscribe \
  --listing projects/octogen-catalog-exchange/locations/US/dataExchanges/catalogs_prod/listings/catalog_farfetch_exported_product_view_v1 \
  --project my-gcp-project

Apply

octogen-bq-subscribe \
  --listing projects/octogen-catalog-exchange/locations/US/dataExchanges/catalogs_prod/listings/catalog_farfetch_exported_product_view_v1 \
  --project my-gcp-project \
  --apply

Re-running is safe. If the linked dataset already exists, the command reports the existing subscription instead of creating a duplicate.

Query the linked dataset

SELECT *
FROM `my-gcp-project.octogen_catalog_farfetch_exported_product_view_v1.products_current_v1`
LIMIT 10;

Manual CLI options

Flag	Required	Default	Description
`--listing`	Yes	—	Analytics Hub listing resource name.
`--project`	Yes	—	Destination GCP project for the linked dataset.
`--dataset`	No	Derived from the listing id	Linked dataset id.
`--location`	No	Listing location	BigQuery dataset location.
`--friendly-name`	No	Derived from the dataset id	Linked dataset display name.
`--apply`	No	Off	Create the linked dataset. Without it, the command prints the plan only.
`--json`	No	Off	Emit JSON output for automation.

Automatic subscription from cron

Use octogen-bq-autosubscribe when you want automation that polls for new BigQuery listings and subscribes to anything ready. The command performs one idempotent sync pass:

Lists BigQuery listings granted to your organization.
Lists your registered BigQuery subscribers.
Registers your Reader if it is missing.
Subscribes every catalog whose status is awaiting_subscription.
Refreshes Octogen status after each linked dataset is created.

Bootstrap MCP OAuth credentials

octogen-bq-autosubscribe calls Octogen MCP to list and register subscriber state. Bootstrap MCP credentials once from an interactive terminal:

octogen-mcp-login \
  --client-id-file /secure/octogen-mcp.client-id \
  --redirect-uri-file /secure/octogen-mcp.redirect-uri \
  --refresh-token-file /secure/octogen-mcp.refresh

The login command opens Octogen AuthKit in your browser, registers a public MCP OAuth client if needed, requests an MCP-audience refresh token, and writes the client id, registered redirect URI, and rotating refresh token to the requested files. It does not print the refresh token.

Dry run

OCTOGEN_MCP_CLIENT_ID_FILE=/secure/octogen-mcp.client-id \
OCTOGEN_MCP_REFRESH_TOKEN_FILE=/secure/octogen-mcp.refresh \
octogen-bq-autosubscribe \
  --project my-gcp-project \
  --principal serviceAccount:bq-reader@my-gcp-project.iam.gserviceaccount.com \
  --json

Apply from cron

*/15 * * * * cd /path/to/octogen-dev && OCTOGEN_MCP_CLIENT_ID_FILE=/secure/octogen-mcp.client-id OCTOGEN_MCP_REFRESH_TOKEN_FILE=/secure/octogen-mcp.refresh uv run --project sdks/python --extra bigquery octogen-bq-autosubscribe --project my-gcp-project --principal serviceAccount:bq-reader@my-gcp-project.iam.gserviceaccount.com --apply --json

OCTOGEN_MCP_REFRESH_TOKEN_FILE should contain an Octogen MCP OAuth refresh token. If the identity provider rotates that refresh token during exchange, the command writes the replacement back to the same file while holding a sibling .lock file, so overlapping cron runs do not overwrite the same rotating token at once.

Autosubscribe options

Flag	Required	Default	Description
`--project`	Yes	—	Destination GCP project for linked datasets.
`--principal`	Yes	—	Google subscriber principal Octogen should grant.
`--catalog`	No	All granted catalogs	Restrict the run to one catalog. Repeat for multiple catalogs.
`--friendly-name`	No	Derived per dataset	Friendly name for created linked datasets.
`--apply`	No	Off	Register and subscribe. Without it, the command reports what would happen.
`--json`	No	Off	Emit JSON output.
`--mcp-url`	No	`https://mcp.octogen.ai/mcp/`	Octogen MCP endpoint.
`--mcp-client-id-file`	No	`OCTOGEN_MCP_CLIENT_ID_FILE`	File containing the public OAuth client id from `octogen-mcp-login`.
`--mcp-refresh-token-file`	No	`OCTOGEN_MCP_REFRESH_TOKEN_FILE`	File containing the rotating MCP refresh token.
`--mcp-token-command`	No	`OCTOGEN_MCP_TOKEN_COMMAND`	Command that prints a short-lived MCP access token.
`--mcp-access-token`	No	`OCTOGEN_MCP_ACCESS_TOKEN`	Short-lived MCP access token for manual runs.

MCP tools behind the automation

Agents and custom clients can implement the same flow through MCP tools:

Tool	Purpose
`list_bigquery_listing_resources`	Lists BigQuery listings granted to your organization.
`list_bigquery_subscribers`	Lists registered subscriber projects, principals, and per-catalog status cells.
`register_bigquery_subscriber`	Registers the GCP project and principal that should receive Analytics Hub subscriber access.
`refresh_bigquery_subscription_status`	Checks whether a linked dataset subscription exists and updates the Octogen status.
`delete_bigquery_subscriber`	Disables a registered subscriber.

Most customers should use octogen-bq-autosubscribe instead of calling these tools directly. It handles idempotency, pending grants, existing subscriptions, and status refreshes for you.

Programmatic subscription

subscribe_to_listing is a synchronous Python helper that talks to Google Cloud using Application Default Credentials.

from octogen_ai_sdk import (
    OctogenBigQueryAccessPendingError,
    OctogenBigQueryError,
    subscribe_to_listing,
)

try:
    result = subscribe_to_listing(
        listing_resource="projects/octogen-catalog-exchange/locations/US/dataExchanges/catalogs_prod/listings/catalog_farfetch_exported_product_view_v1",
        destination_project="my-gcp-project",
    )
except OctogenBigQueryAccessPendingError:
    print("Access is not ready yet. Retry in a few minutes.")
    raise
except OctogenBigQueryError as exc:
    print(f"Subscribe failed: {exc}")
    raise

verb = "Already subscribed" if result.already_subscribed else "Subscribed"
print(f"{verb}: {result.linked_project}.{result.linked_dataset} [{result.state}]")
print(result.sample_query)

Result statuses

octogen-bq-autosubscribe --json returns one catalog result per granted or requested catalog.

Action	Meaning
`already_active`	Octogen already sees an active linked dataset subscription.
`already_subscribed`	BigQuery reported an existing linked dataset, and Octogen status was refreshed.
`subscribed`	The command created a linked dataset and refreshed Octogen status.
`would_subscribe`	Dry run: the catalog is ready and would be subscribed with `--apply`.
`pending`	The Reader or listing grant is still being prepared, or Octogen has not observed status yet.
`skipped`	The catalog is not ready or not enabled for this subscriber.
`error`	The catalog needs attention or the subscribe/refresh step failed.

Error handling

Exit code	Command	Meaning
`0`	Both	Success, dry run, or idempotent no-op.
`1`	Both	Subscribe, MCP, or Google Cloud operation failed.
`2`	Both	Usage or invalid arguments.
`3`	`octogen-bq-subscribe`	Analytics Hub access is not granted yet. Retry shortly.

Both commands are idempotent. Schedule octogen-bq-autosubscribe --apply as often as you need; completed catalogs report already_active and new ready catalogs are picked up automatically.

​How access works

​Prerequisites

​Manual subscription

​Manual CLI options

​Automatic subscription from cron

​Bootstrap MCP OAuth credentials

​Dry run

​Apply from cron

​Autosubscribe options

​MCP tools behind the automation

​Programmatic subscription

​Result statuses

​Error handling

How access works

Prerequisites

Manual subscription

Manual CLI options

Automatic subscription from cron

Bootstrap MCP OAuth credentials

Dry run

Apply from cron

Autosubscribe options

MCP tools behind the automation

Programmatic subscription

Result statuses

Error handling