Factor Foundry
LLM-driven evolutionary factor search that reached a 0.826 public leaderboard score.
- Google Gemini
- Anthropic Claude
- Evolutionary Program Search
- LLM Agent Orchestration
- Status
- Private repository. Two commits, both 2026-08-04 ("Initial commit", "Initial Push"), a single snapshot push, no activity since. The submission artifacts are internally dated 2026-07-23 through 2026-07-27, against a stated final-pick freeze of 2026-08-02, so the work itself is complete but the post-freeze outcome is not recorded here.
By the numbers · 8
0.826
Best public leaderboard score (team maximum, as of 2026-07-27)
70%
Share of the competition score set by joint Elastic-Net uniqueness (Part B)
7 of 9
Submitted factors whose logic was LLM-generated
IC +0.0092 / ICIR +0.271 / LS Sharpe +1.543 / stress-ICIR +0.433
Platform-verified Part-A legs for factor 09 on the 2024 window, all four positive
min 99.2% / mean 99.9%
Per-day panel coverage measured for the order-book factors
|corr| <= 0.012
Measured cross-sectional correlation of factor 09 to every live holding
4
Distinct logged agent mining runs behind the submissions
13
Verbatim LLM audit-trail files shipped with the submissions
Summary
Factor Foundry is an LLM-driven evolutionary search system for quantitative stock-ranking factors, built for the BigAlpha 2026 AI Factor Mining track on Chinese small caps. A Gemini model acts as the composition and mutation operator inside a closed loop: each prompt embeds the scored archive, the rejection reasons, and live leaderboard feedback from prior generations; generated code passes a perturbation-based causality gate and a static SQL gate before an offline replica of the official two-part scoring engine ranks it on both strength and decorrelation from the competitor pool. The architecture matters because 70% of the competition score comes from a joint Elastic Net fitted over every team's factors, correlated entries share weight and unselected ones score zero, so the search objective is uniqueness under a shared model, not standalone predictive power.
The problem
A quantitative-finance competition scores stock-ranking signals for Chinese small caps on a public leaderboard, and it scores them jointly against every other team's entries. A signal that genuinely works but resembles what everyone else submitted earns close to nothing. Teams get at most fifty submissions and are ranked on their single best one. The task is therefore not to find one good signal but to search a large idea space quickly, discard the crowded ideas before spending a submission on them, and prove where each survivor came from, the track additionally requires an auditable record of exactly how the machine contributed.
Approach
A large language model (Google Gemini, temperature 0.9, 8192 max output tokens) is used as the composition and mutation operator of an evolutionary search loop rather than as a coding assistant, no training, no fine-tuning, no loss function. Every generation's prompt embeds the scored archive, the previous rejection reasons, and live leaderboard results, making the search evaluation-guided prompting.
Machine gates run before any scoring. A perturbation-based causality gate mutates future rows and requires interior factor values not to move; a static gate machine-rejects lookahead constructs, nonexistent columns, unverified SQL functions, and any output that violates the contract shape.
An offline replica of the official scoring engine reproduces the competition's two-part objective, winsorize, cross-sectional z-score, style neutralization, then Part A (IC mean, ICIR, long-short Sharpe, stress-regime ICIR) and Part B (ModelScore = mean(|w|)/std(|w|) from a rolling 60d/20d Elastic Net over the crowd and the archive), so decorrelation, not raw predictive power, drives selection.
A second pipeline variant handles order-book data, which exists only inside the platform. The loop runs generation-only, the model additionally emits a structured JSON self-critique ranking its own candidates, and platform evaluation results are byte-copied into a feedback file that steers the next generation.
Generated factor logic ships verbatim between BEGIN/END AI-GENERATED FACTOR LOGIC markers, wrapped in fixed human boilerplate: month-chunked queries to stay inside the 6 GB kernel, a constituent-panel merge, and missing-trading-day and per-day coverage gates that raise loudly rather than silently return an invalid panel.
Every submission hard-codes a SIGN constant set by a single mandatory on-platform first-run check, because the engine scores the post-BARRA residual and does not auto-flip orientation. Repeated observe-then-flip is explicitly banned as leaderboard fitting.
A third design lineage uses an Anthropic Claude multi-agent research workflow: a factor-designer agent emits a structured hypothesis, then an independent adversarial verifier agent demands measured correlations and a measured null rate and warns that the sign story runs backwards, a warning the platform check subsequently confirmed.
The AI Application Description Document records per-submission provenance (run id, generation, archive rank, verbatim prompt and response files) plus a proactive errata section disclosing defects found by the team's own post-submission audit, including a coverage gate rendered vacuous because fillna executed before the check.
Architecture
Prompt assembly (objective + causality rules + primitive vocabulary + scored archive + rejection reasons + leaderboard feedback)Gemini composition operator emits candidate factor codeparse and restricted compilecausality gate (future-row perturbation) and static gate (lookahead constructs, unknown columns, unverified SQL functions, contract shape)scoring: offline replica computes Part A (IC, ICIR, long-short Sharpe, stress ICIR) and Part B (rolling 60d/20d Elastic Net ModelScore vs crowd + archive), or, for order-book factors, on-platform bigalpha_evaldecorrelated archive selectionscores fed back into the next promptnotebook packaging wraps the verbatim factor logic in month-chunked query boilerplate and output gatesBigQuant AI Studio run with a mandatory one-time sign checksubmission, and platform metrics return to the prompt as feedback
| Component | Role |
|---|---|
| Prompt assembler (build_prompt) | Composes each generation's prompt: search objective, strict point-in-time causality rules, verified data schema, primitive vocabulary of unmined data axes, incumbent factors to decorrelate from, the scored archive, and logged rejection reasons. Dictates constraints, never formulas. |
| LLM composition operator (Google Gemini) | Writes executable factor code, pandas functions for the daily-bar campaign, DAI/DuckDB SQL for the order-book campaign. The core AI stage; output ships unedited. |
| Structured self-critique stage | In the order-book pipeline the model additionally emits a JSON block ranking its own candidates on point-in-time risk, memory risk, decorrelation, stress design, and SQL validity risk, parsed verbatim into critiques/gen_*.json. |
| Parser and restricted compiler | Extracts fenced candidate blocks and compiles them in a restricted namespace, no imports, no I/O, no dunder access. |
| Causality gate | Perturbation test: future rows are mutated and interior factor values must not move. Catches lookahead that static inspection misses. |
| Static gate (book pipeline) | Machine-rejects banned constructs (LEAD, negative LAG), nonexistent columns, SQL functions unverified on the platform, and contract-shape violations. |
| Scoring harness (part_a, part_b_en) | Offline replica of the official engine: winsorize, cross-sectional z-score, style neutralization, then Part A percentile legs and a rolling 60d/20d Elastic Net over candidate plus crowd plus archive yielding ModelScore = mean(|w|)/std(|w|). Composite fitness 0.3*ICIR + 0.7*ModelScore. |
| Decorrelated archive (corr_to_archive) | Keeps survivors that are both strong and low-correlation to everything already held, then feeds their scores and the rejection reasons back into the next prompt. This is the loop closure. |
| Notebook packager | Wraps the verbatim AI factor logic in fixed human boilerplate: month-chunked dai.query on the injected bar table, constituent-panel merge, missing-day and per-day coverage gates, no-inf enforcement, and the official evaluation cell. |
| On-platform evaluator (BigQuant AI Studio, bigalpha_eval) | Ground truth. Runs the packaged notebook against the 2024 panel with real BARRA style and industry neutralization, producing the IC, ICIR, long-short Sharpe and stress-ICIR readings used for the mandatory one-time sign check. |
| Claude multi-agent design workflow | A separate lineage for factor 09: a factor-designer agent emits a structured hypothesis and SQL, an independent adversarial verifier agent hunts for fatal flaws and imposes pre-submission conditions (measured correlations, measured null rate, sign-story caution), and the result is evaluated unchanged. |
| AI Application Description Document | The compliance and audit artifact: pipeline-stage table marking which stages are AI, per-submission provenance rows citing run ids and verbatim prompt/response files, self-assessed AI participation level, and an errata section that supersedes any disagreeing notebook. |
Trade-offs
Chose
A generation-only LLM loop with a human running evaluation on the platform and feeding results back
Over
The fully automated offline-scored loop used for daily-bar factors
Order-book minute data exists only inside the competition platform, so offline scoring of book factors is impossible; the loop was restructured so the model still steers on real evaluation numbers, byte-copied into a feedback file (AI Application Document §2, book-campaign table).
Chose
Artifact-level reproducibility, every invocation, prompt, and raw response logged verbatim
Over
Seed-level determinism
LLM sampling at temperature 0.9 is stochastic and cannot be replayed exactly; the audit can instead re-validate the exact outputs that were actually produced (AI Application Document §3, disclosed as a determinism caveat).
Chose
Selecting for decorrelation against a scored archive and the visible crowd
Over
Maximising standalone predictive power
Part B is 70% of the total score and comes from a rolling Elastic Net fitted over all competitors' factors, correlated factors share weight and rotate selection, and L1-unselected factors score zero (prompts_gen_02.md scoring block; README's ban on correlated variants).
Chose
Month-chunked queries with expressions projected once in a first CTE before windowing
Over
Single full-window queries
The platform kernel has a hard 6 GB limit; a full-year window-function query OOMs, and recomputing an expression inside the window function crashed the kernel outright (prompt ground-truth block; comment in 02_m1_imbalance_persistence/submission.ipynb).
Chose
A restricted portable-SQL vocabulary with correlations rebuilt manually from AVG and SQRT
Over
Built-in statistical SQL functions
CORR, REGR_*, MEDIAN, QUANTILE, PERCENTILE_* and FILTER(WHERE) are unverified on the platform's DAI dialect and are machine-rejected by the static gate, so the prompt hands the model the explicit covariance formula instead (prompts_gen_02.md).
Chose
Shipping AI-generated logic verbatim, with the single human patch on factor 08 explicitly disclosed
Over
Hand-tuning generated formulas before submission
The AI-track finals audit must reproduce the submitted code from the logged provenance; the one deviation (a CAST AS DOUBLE fix for INT32 overflow) is recorded with the original verbatim block preserved (AI Application Document §5, row 08).
At scale
11 submission notebooks across 9 factor folders; GitHub reports 86,304 bytes of Jupyter Notebook as the repository's only language.
13,418-byte AI Application Description Document containing a 9-row per-submission provenance table, a stage-by-stage AI attribution table for both pipelines, and a 5-item errata and known-limitations section.
13 verbatim LLM audit-trail markdown files (6 prompts, 6 raw responses, 1 multi-agent design provenance) shipped alongside the notebooks.
Two distinct miner pipelines referenced by path, kit/agent_miner.py (daily-bar, fully closed offline-scored loop) and kit/agent_miner_book.py (order-book, generation-only with platform feedback), plus a third Anthropic Claude multi-agent design lineage for factor 09.
Evaluation window is the full 2024 trading year over the CSI 1000 constituent panel, queried month-chunked from a 1-minute order-book bar table carrying five price, volume and order-count ladder levels.
One prompt (05/prompts_gen_04.md) embeds an 8-entry scored archive with per-factor IC, ICIR, ModelScore and selection rate, plus 4 logged rejection reasons, as feedback for the next generation.
Competition budget capped at 50 total submissions with the team ranked on its single best; final picks limited to 2 and frozen 2026-08-02.
My role
Not fully evidenced by the repository alone. Muneeb (munib123) owns this private repository and is its sole committer, and the notebooks, prompts and provenance document are consistent in style and process. However the AI Application Description Document is written in team voice ("our team", "the team's multi-agent research workflow"), so the split between individual and team contribution must be confirmed with him before the case study states a role. Treat as: contributor to a competition team, with authorship of the miner architecture, prompt design and scoring harness to be verified.
