Build the pipelines behind datastore: decoding years of on-chain history into clean, versioned Parquet that researchers can trust.
Datastore sells something unusual: files, not API access. Customers download decoded on-chain history as Parquet and run their own queries. That only works if the data is actually right, which makes correctness, lineage and reproducibility the product.
You will build and run the pipelines that decode Solana and Hyperliquid history at scale: backfills over billions of rows, schema design, checksums, manifests and the quality checks that let a quant trust a file they did not produce themselves.
What you will do
Design and run large-scale decoding and backfill pipelinesModel typed schemas for instructions, events and state across dozens of protocolsBuild validation that catches bad data before customers doKeep versioning, checksums, manifests and lineage docs accurate on every deliveryTune storage layout and partitioning so files query rapid in DuckDB and PolarsAdd new protocols and chains to the catalogueSupport customer questions about schemas and coverageWhat we are looking for
Experience building production data pipelines at meaningful scaleStrong SQL plus one of Python, Rust or GoReal familiarity with columnar formats,



ideally Parquet, and query engines like DuckDB, Polars or SparkCare for data correctness: testing, validation and reconciliationComfort owning pipelines in production, including when they break at nightAble to work from our Toronto office part of the weekNice to have
Experience with blockchain data or other messy, high-volume event streamsFamiliarity with warehouse ecosystems your customers use, like Snowflake, BigQuery or ClickHouseBackground in quantitative research support or backtesting infrastructureExperience with orchestration tools and with knowing when a cron job is enoughWho you are
You think an unverified number is worse than no numberYou write pipelines you would be happy to debug at 2am, so they rarely need itYou like schemas that make the next person's query obviousYou get satisfaction from a backfill that reconciles to the last rowHow we hire
1 Intro call with an engineer, about 30 minutes2 Short take-home assignment working with a real decoded dataset3 Technical conversation about your assignment and pipelines you have run4 Conversation with the founders5 Offer
#J-18808-Ljbffr

📌 Data Engineer (Winnipeg)
🏢 CLR3
📍 Winnipeg