Warehouse Misconfigurations That Quietly Break Reliability
Your Snowflake or BigQuery setup might be running smoothly on the surface while quietly producing unreliable numbers. Here are the configuration mistakes we see most often.
By Pallisade Team
When a warehouse query fails, you notice. When it succeeds but returns a slightly wrong answer because of a misconfiguration, you might not notice for months. These are the subtle setups that bite teams later.
Time Zones Set at the Wrong Layer
A query runs correctly for the analyst in one timezone and incorrectly for the one in another, because the warehouse session timezone, the transformation layer default, and the dashboard display are all set differently. Pick one canonical timezone for all stored timestamps (UTC), and do conversions only at the display layer.
Warehouse Auto-Suspend Too Aggressive
Snowflake warehouses that auto-suspend after 60 seconds will cold-start every scheduled query, inflating wall-clock time and sometimes exceeding job timeouts. For critical pipelines, use a dedicated warehouse with a longer suspend window.
Clustering Keys That Never Actually Cluster
Clustering keys on a table that is rarely filtered by those columns cost money and buy nothing. Run system$clustering_information and compare against your actual query patterns before adding them.
Partitioning on the Wrong Column
BigQuery partitioned on created_at performs well for queries that filter on created_at. But if your analysts mostly query by updated_at, you are scanning the full table every time. Partition by the column you filter on, not the column that feels like the natural primary key.
Retention Policies That Delete Useful History
Setting a 7-day time-travel window on a table you occasionally need to restore from is fine — until the one day you need day 9. Critical tables should have retention windows that match how long it takes to actually notice a problem, plus a safety margin.
Permissions That Allow Schema Changes Downstream
When transformation users can ALTER TABLE on source schemas, upstream changes can happen without producer awareness. Enforce the direction: sources flow downstream, not the other way.
Column Masking That Hides Quality Issues
Data masking policies that replace values with ** before your quality checks run will let the bad data through undetected. Run quality checks before masking, not after.
Implicit Type Coercion in Joins
Joining a string column to a number column silently coerces one side. Some warehouses do this without warning. The join will look like it works, but may drop rows where coercion fails. Always check types before joining across sources.
What to Audit
Run this checklist against your warehouse quarterly:
- [ ] All stored timestamps are UTC
- [ ] Critical pipelines have dedicated compute
- [ ] Clustering and partitioning match query patterns
- [ ] Time-travel and fail-safe windows are long enough
- [ ] Source schemas are protected from downstream writes
- [ ] Quality checks run on unmasked data
- [ ] Join keys have matching types
Need a reliability audit of your warehouse? Get in touch.*
Tags:
Need Help With Your Security Posture?
Our team can help you identify and fix vulnerabilities before attackers find them.