Data format and structure
How to format your billing data and where to place it.
Overview
This page outlines how our daily enrichment finds and processes your billing data every day.
We run enrichments between 01:00 and 04:00 UTC each day, processing the entire month-to-date, every day.
All dates/times referenced in this document are in UTC timezone.
Storage platforms
Greenpixie can read data from S3, Azure Blob Storage or Google Cloud Storage.
Data from any platform can be placed in any storage provider. You can use the same bucket/container for multiple platforms, or you can provide separate buckets/containers for each platform.
Data format
Billing data can be stored in any of the following formats:
.csv.csv.gz.parquet.snappy.parquet.zstd.parquet
Data must be split into separate files (partitions) of less than 500MB each. Each partition must have the exact same column names and data types across a single directory. Separate directories can have different schemas.
Directory structure
Data must arrive in one of two layouts:
- Your cloud provider's default export layout — point the billing export directly at the connected bucket/container and leave the delivered files untouched.
- The Greenpixie structure — required whenever you restructure the data or deliver it through your own pipeline. If you change the layout at all, it must match the Greenpixie structure exactly; other custom layouts are not supported.
Partners always use the partner variant of the Greenpixie structure.
Default export layout
If your billing export delivers straight into the connected bucket/container, no restructuring is needed — Greenpixie reads the provider's default layout as delivered.
AWS Data Exports (CUR 2.0) deliver to:
// <bucket_name> / <prefix> / <export_name> / data / BILLING_PERIOD=<year_month> / {partitions here...}Azure Cost Management exports deliver to:
// <container_name> / <directory> / <export_name> / <date_range> / {partitions here...}Where <date_range> is the export period in the format YYYYMMDD-YYYYMMDD.
For GCP there is no provider-written default: the delivery job you author sets the path, so use the Greenpixie structure below. The export query in the GCP usage export guide already produces it.
Greenpixie structure
If you restructure the data yourself, it must be placed in a directory structure that matches the following pattern. Greenpixie data pipelines rely on this structure to infer which platform, account and time period the data is for.
// <bucket_container_name> / <input | output> / <platform> / <account_id> / <year_month> / {partitions here...}Where:
<bucket_container_name>can be any bucket name you decide<input | output>can beinputoroutputto indicate whether the data is for input to Greenpixie or output from Greenpixie. Even if you are using separate buckets/containers for input/output, it's essential to still include this base directory.<platform>is the name of the platform that the billing data relates to from the following list:aws,azure,gcp. Even if you are only sending one platform, it's essential to still include this directory.<account_id>is the ID of any billing accounts in your environment (e.g.1234567890). If you do not have separate accounts for a platform, please enterall.<year_month>is the year and month of the data in the format:YYYY-MM. e.g2025-01- Partitions can be named anything you wish, but must be less than 500MB each.
If you are providing multiple platforms in separate buckets/containers, you must still include the <platform> directory.
Examples:
ABC Corp, placing AWS billing data in S3 for accounts 1111111111 and 2222222222 in the month of November 2025:
ABC Corp placing Azure billing data for all accounts (not separated by account) in Azure Blob Storage in the month of November 2025:
If you are onboarding as a Greenpixie partner, the directory structure differs from the customer pattern in two ways:
- An
<org_id>directory sits between<input | output>and<platform>, identifying the organisation the data belongs to. Greenpixie shares these IDs with you during partner onboarding. - The
<year_month>directory is replaced by a<timestamp>directory, identifying a single upload. Use ISO 8601 basic format in UTC:YYYYMMDDTHHMMSSZ, e.g.20260722T120000Z.
// <bucket_container_name> / <input | output> / <org_id> / <platform> / <account_id> / <timestamp> / {partitions here...}Example: a partner placing AWS billing data for organisation org_12345, account 1111111111, uploaded at 12:00:00 UTC on 22 July 2026:
To trigger an enrichment of an uploaded timestamp directory, see Partner trigger.
Next steps
At this stage, you should have formatted data in a bucket/container, and assigned permissions to give Greenpixie access.
Next, Greenpixie will check the permissions and minimum columns required for CUR enrichment.
Head back to How it works to continue.