ticklake

Fail-safe local DhanHQ tick ingestion stack for NSE spot, futures, and options.

Architecture

• 5-socket topology: mkt-0 for spot+futures, opt-0..opt-3 for option shards.
• Durable ingestion: websocket packet -> local spool (disk) -> parser -> TimescaleDB + Parquet.
• Dhan v2 binary feed decode for ticker/quote/full packets into canonical ticks.
• No-loss recovery: spool checkpoint replay on restart/crash.
• Live query while writing: Timescale hypertables with indexed symbol/instrument time queries.
• Exact replay: raw websocket payloads archived in partitioned Parquet.

Quick Start

1. Install dependencies:

python -m pip install -e .[dev]

2. Start TimescaleDB:

docker compose -f deploy/docker-compose.timescale.yml up -d

3. Create your .env from template:

Copy-Item config/.env.example .env

4. Replace instruments.sample.csv with your live instrument master or update INSTRUMENT_MASTER_PATH in .env.

5. Verify setup:

ticklake doctor

6. Run ingestion:

ticklake run

7. Run query API (separate terminal):

ticklake serve-api

8. Open dashboard:

http://127.0.0.1:8080/dashboard

Commands

• ticklake run: start ingestion service.
• ticklake doctor: health checks for token/master/Timescale.
• ticklake backfill-index: ensure DB schema/indexes.
• ticklake compact --event-date YYYY-MM-DD: compact canonical Parquet partition.
• ticklake serve-api: start local query API.

If Docker/Timescale is unavailable, run ingestion with:

$env:STORAGE_MODE='parquet'; ticklake run

Key Paths

• src/ticklake/ws: websocket client and connection manager.
• src/ticklake/pipeline: spool, parser, router.
• src/ticklake/sinks: Timescale and Parquet sinks.
• src/ticklake/query: FastAPI query endpoints.
• sql/: DB DDL/index policy scripts.
• deploy/docker-compose.timescale.yml: Timescale runtime.

Security Note

Rotate any tokens/secrets currently present in .env and keep .env out of version control.