GCP usage export

Getting your Google Cloud usage data is a two-part change. Part 1 enables the Cloud Billing detailed usage cost export to BigQuery. Part 2 sets up a daily job that exports that table to a Cloud Storage bucket you own.

Enable part 1 as early as possible. Billing data flows only from the date the export is enabled. For regional dataset locations such as europe-west2, Google does not backfill earlier usage, while US and EU multi-region locations receive some retroactive backfill that can take up to 5 days. History from before enablement cannot be recovered later.

Before you start

You need:

• Billing Account Administrator on the billing account, and BigQuery User or higher on the host project.
• A project and BigQuery dataset to receive the export. Choose the dataset location per your data residency policy, noting the backfill behaviour above.
• A Cloud Storage bucket to receive the part 2 delivery. Create one, or use an existing bucket. If you plan to use Greenpixie's cloud usage report enrichment service, this is the same bucket you will connect later, so a name that includes your company or team is recommended, for example abc-billing-data. See Connect Google Cloud Storage.

Check these before you begin:

• Whether the detailed export is already enabled. Look for a table named gcp_billing_export_resource_v1_* in an existing billing dataset. The standard export table gcp_billing_export_v1_* is not sufficient; the resource-level table is required. If it already exists, skip to part 2.
• If an organisation policy restricts which identities can be granted IAM roles, known as domain-restricted sharing, pre-approve an exception. Enablement adds Google's billing export service account ([email protected]) as a dataset owner and fails without it.
• If VPC Service Controls apply to BigQuery, add the ingress and egress rules covered in Google's setup guide. For part 2, the dataset and destination bucket must sit in the same perimeter or have an egress rule.

If usage spans more than one billing account, repeat these steps per billing account. Each writes its own table.

Part 1. Enable the BigQuery export

1. In the Google Cloud console, go to Billing, select the billing account, then Billing export.
2. On the BigQuery export tab, click Enable Detailed usage cost export.
3. Select the host project and dataset, then save.
4. Confirm the table gcp_billing_export_resource_v1_<billing_account_id> appears and receives data. It is partitioned by day. Cost data usually appears within a few hours, and any retroactive backfill for multi-region locations can take up to 5 days.

Part 2. Daily delivery to the bucket

5. Create a daily scheduled job that exports the previous day's partition to the bucket. A BigQuery scheduled query using EXPORT DATA is the simplest option, and a Composer or Workflows job is equally fine. Base it on the query below, which writes Parquet with zstd and shards output with a wildcard.

EXPORT DATA OPTIONS (
  uri = 'gs://<destination-gcs-bucket>/input/gcp/<billing_account_id>/<year_month>/*.parquet',
  format = 'PARQUET',
  compression = 'ZSTD',
  overwrite = true
) AS
SELECT *
FROM `<project>.<dataset>.gcp_billing_export_resource_v1_<billing_account_id>`
WHERE DATE(_PARTITIONTIME) = DATE_SUB(CURRENT_DATE(), INTERVAL 1 DAY);

Adapt paths, scheduling and date handling to your own standards. This is the shape, not final code.

6. Run a one-off export of all existing partitions to the bucket to cover history from the enablement date to now.

Required directory structure

If you will be using Greenpixie's cloud usage report enrichment service, the delivery path must follow the shape below.

<bucket_name>/<input|output>/<platform>/<account_id>/<year_month>/{partition files}

gs://<destination-gcs-bucket>/input/gcp/<billing_account_id>/2026-07/...

<bucket_name>/<input|output>/<org_id>/<platform>/<account_id>/<timestamp>/{partition files}

gs://<destination-gcs-bucket>/input/org_12345/gcp/<billing_account_id>/20260722T120000Z/...

Data and format notes

Export all columns and keep the full detailed-export schema. The complete set gives you the most context for later analysis.

The detailed export contains nested (RECORD) columns, which Parquet preserves. BigQuery cannot export nested data to CSV, so if you need CSV, flatten the columns in the SELECT.

If you will be using Greenpixie's cloud usage report enrichment service, Parquet with zstd compression is preferred, and the query above produces exactly that. No single file should exceed 500 MB. A single EXPORT DATA file can be up to 1 GB and file sizes are not guaranteed, so the * wildcard alone will not hold files under 500 MB. To split further, partition the query on an extra column.

Check it worked

You should have:

• The detailed usage cost table existing and updating daily.
• Daily files landing in the agreed bucket path, in Parquet (nested) or flattened CSV.
• Parquet files, with zstd preferred, and no single file larger than 500 MB.
• A one-off backfill covering the enablement date to present.

To remove the export later, disable it under Billing export settings and delete the scheduled job. Nothing else depends on them.

Next steps

Once your export is running, you can enrich it with energy, carbon and water metrics in two ways.