Data format and structure
How to format your billing data and where to place it.
Overview
This page outlines how our daily enrichment finds and processes your billing data every day.
We run enrichments between 01:00 and 04:00 UTC each day, processing the entire month-to-date, every day.
All dates/times referenced in this document are in UTC timezone.
Storage platforms
Greenpixie can read data from S3, Azure Blob Storage or Google Cloud Storage.
Data from any platform can be placed in any storage provider. You can use the same bucket/container for multiple platforms, or you can provide separate buckets/containers for each platform.
Data format
Billing data can be stored in any of the following formats:
.csv.csv.gz.parquet.snappy.parquet.zstd.parquet
Data must be split into separate files (partitions) of less than 500MB each. Each partition must have the exact same column names and data types across a single directory. Separate directories can have different schemas.
Directory structure
Data must be placed in a directory structure that matches the following pattern. Greenpixie data pipelines rely on this structure to infer which platform, account and time period the data is for.
// <bucket_container_name> / <input | output> / <platform> / <account_id> / <year_month> / {partitions here...}Where:
<bucket_container_name>can be any bucket name you decide<input | output>can beinputoroutputto indicate whether the data is for input to Greenpixie or output from Greenpixie. Even if you are using separate buckets/containers for input/output, it's essential to still include this base directory.<platform>is the name of the platform that the billing data relates to from the following list:aws,azure,gcp. Even if you are only sending one platform, it's essential to still include this directory.<account_id>is the ID of any billing accounts in your environment (e.g.1234567890). If you do not have separate accounts for a platform, please enterall.<year_month>is the year and month of the data in the format:YYYY-MM. e.g2025-01- Partitions can be named anything you wish, but must be less than 500MB each.
If you are providing multiple platforms in separate buckets/containers, you must still include the <platform> directory.
Examples:
ABC Corp, placing AWS billing data in S3 for accounts 1111111111 and 2222222222 in the month of November 2025:
ABC Corp placing Azure billing data for all accounts (not separated by account) in Azure Blob Storage in the month of November 2025:
If you are onboarding as a Greenpixie partner, the directory structure differs from the customer pattern in two ways:
- An
<org_id>directory sits between<input | output>and<platform>, identifying the organisation the data belongs to. Greenpixie shares these IDs with you during partner onboarding. - The
<year_month>directory is replaced by a<timestamp>directory, identifying a single upload. Use ISO 8601 basic format in UTC:YYYYMMDDTHHMMSSZ, e.g.20260722T120000Z.
// <bucket_container_name> / <input | output> / <org_id> / <platform> / <account_id> / <timestamp> / {partitions here...}Example: a partner placing AWS billing data for organisation org_12345, account 1111111111, uploaded at 12:00:00 UTC on 22 July 2026:
To trigger an enrichment of an uploaded timestamp directory, see Partner trigger.
Next steps
At this stage, you should have formatted data in a bucket/container, and assigned permissions to give Greenpixie access.
Next, Greenpixie will check the permissions and minimum columns required for CUR enrichment.
Head back to How it works to continue.