The scorecard, as committed before the run

Public copy of the scorecard committed to the QuantAscent repository on 2026-10-15 (commit af41393), with Amendments 1 and 2 (commits 80730ad and 8d95e9c) added the same day, before any strategy was drawn. The run started on 2026-10-15 at 21:21. Two changes were made for publication: the data vendor's name is replaced with "the data vendor", and this note is added. Nothing else is edited. In this file the three arms carry the study's internal labels. They map to the article as follows: "placebo" is version 2, Library only; "design space" or "sweep" is version 3, Adaptive, any settings; "as designed" is version 4, Adaptive, shipped settings.

Pre-registered 2026-10-15. This file was written and committed before any draw was run. The git hash of the commit that first added it is the timestamp. Anything that changes after that commit is recorded under Amendments at the bottom, with the date and the reason. Results never change a hypothesis or a bar.

1. The question

Does scoring companies on their financial statements with statistical methods give a stock picker an edge over picking names at random from the same pool, and over owning the pool or the market as a whole?

"Edge" is scored three ways because the summer study found they move independently:

• Return: does the typical software portfolio make more money?
• Risk: does it lose less in its worst stretch, and swing less?
• Risk-adjusted return: does it make more money per unit of risk?

And "edge" is tested against three comparison groups, from easiest to hardest:

1. Random portfolios from the same pool (same universe filter, same number of holdings, same quarters). This isolates the scoring method.
2. Owning the whole pool equally. This isolates the picking.
3. The market: the S&P 500 cap-weighted (SPY), the S&P 500 equal-weighted (RSP), and for single-sector pools the sector ETF.

2. What is being tested

The QuantAscent Adaptive Builder as shipped in the app, driven through the Sweep Explorer's design-space sampler, on a research panel built with the shipped panel builder on a longer history than the app's pack carries (section 3, Amendment 1). In plain terms, at every quarter the engine:

1. looks only at data available before that quarter,
2. measures which financial metrics have recently predicted returns in the chosen pool (information coefficient, split-half sign agreement),
3. fills a recipe's slots with the qualifying metrics, optionally narrows the pool with a single statistically chosen floor metric,
4. scores every company in the pool and buys the top N equally weighted,
5. holds one quarter, then repeats.

The walk-forward is out of sample at every quarter by construction. Code paths: quant_engine/user_library.py run_walk_forward and derive_recipe, quant_engine/sweep_space.py sample_sweep, quant_engine/luck_band.py draw_returns. The harness calls these; it does not reimplement them.

3. Data

The harness records the panel file's SHA-256 and the DB's max price date in out/ at run time. A result produced on a different panel is a different study.

Survivorship disclosure, fixed before the run. Dead companies are well covered from 2021 on, partially covered 2016 to 2018, thin in 2019 and 2020, and essentially absent before 2016. The first scored quarter is 2016-08-31. The engine's record starts in 2011 (it needs eight panel dates of history), but no quarter before 2016-08-31 is ever scored, so 2009 to mid-2016 is training history only. The bias flatters the software's portfolios, the random portfolios, and the pool average alike, so comparisons 1 and 2 are largely unaffected; it flatters the software against the market (comparison 3). The write-up says this.

4. The universe matrix

One sweep per cell. A universe is fixed for its sweep. Every cell carries a $5 share-price floor (the panel itself already excludes sub-$1 names at build time) and the freshness rule DaysSinceFiling <= 135 as a metric filter (Amendment 2), so the engine's candidate pool, the random cloud, and the pool average all exclude stale filers alike. The engine's own freshness screen uses the same 135-day rule, so this adds nothing to the engine and removes an asymmetry from the comparisons.

Sector sets (16): the eleven singles (Basic Materials, Communication Services, Consumer Cyclical, Consumer Defensive, Energy, Financial Services, Healthcare, Industrials, Real Estate, Technology, Utilities), four groups, and all sectors.

Size sets (9), market cap in USD:

Gate sets (3), as metric_filters entries, NaN fails closed:

How the solvent threshold was derived. Rows: panel dates before the first scored quarter (2009-10-07 to 2016-06-02, 28 dates, 73,590 stock-quarters), $5 floor, all sectors. Label: tail loss, next-quarter return below -30% (base rate 4.3%). Rule: the single Z split with the largest Gini impurity decrease, scanning a 0.1-percentile grid, allowed to exclude at most 30% of rows. Result: Z > -1.282, which excludes 4.2% of rows; the tail-loss rate is 14.6% below the cut and 3.8% above it. It keeps 94% or more of every sector except Healthcare (84%). Output: out/z_calibration.json.

Why not the textbook 1.8: in this panel the median Z is 1.68, so "above 1.8" is a median split (keeps 47%), and it is a sector tilt by value, not because Z is undefined for anyone: it keeps 18% of Financial Services, 17% of Real Estate, and 11% of Utilities, whose balance sheets make Z structurally low. On the calibration rows the tail-loss rate above 1.8 is slightly higher than below it, so 1.8 is not a distress screen in this data. One reference cell A-all-solvent18 (MeanZscore > 1.8, all sectors, all caps, 500 draws) is kept so the piece can compare.

The calibration years are survivors-only, and a distress screen is about the companies that go bust, which are the ones missing. The derived cut is therefore lenient, which makes the gate axis a conservative test of "does a screen help."

The profit gate keeps 69% of the pool overall but only 29% of Healthcare (the unprofitable-biotech effect) and 58% of Technology. It is partly a sector tilt and is labelled as such.

The engine's own statistically chosen gate (the universe_floor dial) is not a universe cell. It is varied on and off inside every sweep, so "did letting the statistics choose the gate help" is reported as its own finding (section 8, F4).

Tier A, the main factorial: all sectors x 9 sizes x 3 gates = 27 cells, 500 draws each, plus the reference cell A-all-solvent18 (28 cells).

Tier B, sector detail: 15 sector sets (11 singles + 4 groups) x {all, ex_micro} x {none, profit} = 60 cells, 200 draws each.

88 cells, 26,000 draws. Each draw is one full-series walk-forward, so every cut point in section 6 comes from the same run.

Cell id format: A-<size>-<gate> and B-<sector_slug>-<size>-<gate>. Sweep seed for a cell = 101 + its ordinal in the cell list the harness writes to out/cells.json (list order: Tier A in the table order above, then A-all-solvent18, then Tier B alphabetical by sector slug). Luck cloud seed = 7, the product convention.

Thin-cell rule. Before the run, the harness counts each cell's pool per traded quarter from the panel. A cell whose pool averages fewer than 45 names (three times the maximum holdings count) is marked thin, still run, but excluded from every verdict aggregate and listed by name in the write-up as "too thin to test." Pool counts are not outcome data, so this check does not break pre-registration.

5. The design space, identical for every cell

structure: mode "template", flavors_allowed = all seven (Momentum, Growth, Quality,
           Value, Financial Strength, Defensive, Equal Weighting),
           max_blend = 6 (the composer's full range), holdings = [5, 15]
dials:     lookback             ["expanding", 12, 16, 20, 24]
           stability            ["standard", "strict"]
           ic_floor             [0.010, 0.020, 0.030]
           cadence              ["quarterly", "boundary"]
           universe_floor       [true, false]
           floor_stability_gate [true, false]
           freshness_trained    true   (inert once freshness is in the universe,
                                        Amendment 2; fixed on)

Every leaf is a value the Sweep tab exposes. The sampler is non-adaptive: draws never move toward winners. Custom-cluster mode is out of scope for this study.

Three further arms on the nine no-gate Tier A cells (A-<size>-none, the headline cell among them), Amendment 2. Same universe, same quarters, same random cloud as the design-space arm, so the four rungs read as a ladder:

The placebo's slot filler is the only non-product code in the study. It is a 20-line substitute for _fill_slots that ignores the IC halves; everything else (universe, scoring, portfolio construction, walk-forward) is the shipped path. Placebo seed for a cell = 1001 + the cell's ordinal.

6. Spans and cut points

Panel date indices in brackets (the extended panel, Amendment 1). The engine trades from index 8 (2011-09-xx) but quarters before 2016-08-31 [28] are survivors-only and are never scored. 41 scored quarters are available, 2016-08-31 [28] through 2026-07-10 [68], the same dates as the shipped panel's [8..48].

Selection spans expand from the first scored quarter. Holdouts are two years, matching the product's own sealed span. Tests T1 to T3 and T5 are also reported per holdout. Every training window (section 5) is fully formed at the first scored quarter: the engine has 28 panel dates of history there, more than the longest trailing window.

Stood-aside quarters (the engine held cash because nothing qualified):

• For comparisons against random portfolios (T1 to T3), traded quarters only, on both sides. This is the product's luck-cloud convention.
• For comparisons against the pool and the market (T5), the full span with cash quarters at 0% while the benchmark keeps compounding. This is what an adopter would have lived through.
• The stood-aside rate per cell is reported either way.

Refusals. A draw with fewer than 4 traded quarters in a span is refused for that span (product convention REFUSED_MIN_Q). Refused and errored counts are reported per cell and never dropped silently.

7. Metrics

Per draw, per span, from quarterly returns:

Luck percentile. For a draw and a span, 500 random portfolios are drawn from the same universe filter, redrawn every quarter, over the draw's traded quarters, seed 7. The draw's percentile is the share of random portfolios it beats on the metric. Under no edge, percentiles are uniform and the median across draws is 50.

Two versions are computed (Amendment 2). Matched-count (primary): each quarter's random portfolios hold exactly as many names as the engine actually held that quarter (the engine holds between 5 and the requested count, fewer when fewer qualify; a fixed-count cloud would be less concentrated than the engine and tilt the drawdown comparison against it). This is a per-date-size variant of luck_band.draw_returns with the same conventions, pinned by a test to reproduce draw_returns bit-for-bit when every size is equal. Product convention (secondary): luck_band.draw_returns with the requested holdings count. On the product's own design span this must reproduce the nfp the sweep runner stamped on the row exactly; the harness asserts it, which validates the scoring arithmetic.

Percentile direction: higher is better for CAGR and Sharpe; for max drawdown and volatility the percentile is the share of random portfolios with a worse (deeper or higher) value, so higher is still better.

Uncertainty. Draws within a cell share quarters, so they are not independent. Every confidence interval is a 4-quarter block bootstrap over the span's quarters (1,000 resamples): resample quarter blocks, recompute every draw's and every random portfolio's metric on the resampled path, recompute the percentile and the cross-draw median. The CI is on the cross-draw median percentile.

Secondary arm, cap-weighted picks. From each draw's recorded quarterly picks, reweight by the panel's MarketCap at the as-of date and recompute the series. Its random cloud is cap-weighted too (same drawn tickers, cap weights). Reported in T5 against SPY only. This arm never decides a headline verdict.

8. Tests and verdict bars

Each test returns one of three verdicts per cell and span, always printed with the median and its 95% interval (Amendment 2: the middle verdict is NOT DETECTED, not "no edge", because ten years of quarterly data cannot distinguish a small edge from none, and the interval says which it is closer to):

• EDGE: the cross-draw median percentile is at or above 60 and its 95% CI excludes 50.
• NOT DETECTED: the CI includes 50, or the median sits between 40 and 60.
• HARM: the median is at or below 40 and the CI excludes 50.

T4 verdict bars, per metric: PERSISTS if the correlation is at or above +0.20 at three or more of the four seams; DOES NOT PERSIST otherwise. The top-decile-by- selection draws' median holdout excess versus the remaining draws is reported next to it, since that is what "picking the best dot on the map" would have bought.

T5 verdict bars, per benchmark: BEATS if the share of draws ahead is at or above 60% and the 4-quarter block-bootstrap 95% CI on that share excludes 50%; TIES if the CI includes 50%; TRAILS if the share is at or below 40% and the CI excludes 50%. The pool-average comparison is the picking test. The RSP comparison is the fairest market bar for an equal-weight portfolio. The SPY comparison is the one the survivorship disclosure says is tilted in our favor.

T6, the statistics rung (Amendment 2). On each of the nine no-gate Tier A cells, the design-space arm's cross-draw median percentile minus the placebo arm's, for T1, T2, and T3 on the FULL span, with a 4-quarter block-bootstrap 95% CI on the difference. ADDS if the difference is at or above +5 points and the CI excludes 0; NOT DETECTED if the CI includes 0 or the difference is within 5 points of 0; SUBTRACTS if at or below -5 and the CI excludes 0. The as-designed arm is scored on T1 to T3 and T5 with the section 8 bars like any other arm, and its difference from the design-space arm is reported the same way as T6.

T7, selection discipline (Amendment 2). The product's documented way to choose from a sweep is to reject draws below the luck bar and prefer the lowest-risk candidates. On each seam's selection span, two pre-stated selections are applied to the design-space draws of each cell: (a) the shallowest-drawdown quartile, and (b) draws above the product's bar for a 500-draw sweep (the 500/501 percentile of the product-convention cloud). For each, the selected draws' median holdout CAGR percentile and max-drawdown percentile are compared with the remaining draws'. PERSISTS if the selected draws lead by at least 5 points at three or more of the four seams; DOES NOT PERSIST otherwise. This is the test of "use the software the way the methodology page says," and it costs no extra draws.

Secondary findings, reported but not verdicts:

• F1 The cell map: every test's verdict per cell, as a grid and as a distribution, never as a leaderboard.
• F2 Size: T1 to T3 medians across the nine size sets within Tier A.
• F3 Gates: T1 to T3 medians for none vs profit vs solvent within Tier A.
• F4 Engine-chosen floor: T1 to T3 medians for draws with universe_floor on vs off, pooled across Tier A.
• F5 Sectors: T1 to T3 and T5-sector-ETF per sector set in Tier B.
• F6 Holdings count and blend size: T1 to T3 medians by holdings bucket (5 to 7, 8 to 11, 12 to 15) and by number of flavors in the blend.
• F7 Stood-aside rate and refusal rate per cell.
• F8 Training window: T1 to T3 medians for expanding vs trailing 12, 16, 20, and 24 quarters, pooled across Tier A.
• F9 Turnover: median share of names replaced per quarter, per cell and per arm, so a reader can apply their own cost haircut to the gross figures.
• F10 Portfolio size: distribution of names actually held versus requested, per cell, with the stood-aside rate.

9. Headline and exploration

The headline verdict comes from one cell, fixed now: A-all-none (all sectors, all caps, no gate, $5 floor), on the FULL span for T1, T2, T3, and T5, and across the four seams for T4, and (Amendment 2) the ladder on that cell: random stocks, placebo engine, design space, as designed, each with its T1 to T3 medians and intervals, plus T6 and T7. The write-up's answer to "is there an edge" is these verdicts, in the words of section 8, read rung by rung.

Everything else is exploration. With 88 cells, about 4 would clear a 5% threshold by luck alone if the method did nothing. Exploration is therefore reported as distributions across cells (what share of non-thin cells returned EDGE on T1, on T3, and so on) and as the factor summaries F2 to F7. A single non-headline cell is never presented as a result on its own. The broad claim "the method works across pools" requires the share of non-thin cells returning EDGE on a test to exceed the upper 95% binomial bound of what chance predicts at the 5% level.

10. Compute plan

• Workers: 4 by default (the pool deadlocked at 8 inside the app); the headless harness may use up to 8 if the probe shows it stable. The panel costs 0.4 GB per worker; the machine has 32 logical cores and 32 GB.
• Timing probe first: 20 draws of A-all-none, extrapolate to the roughly 31,100 draws and runs across all arms (26,000 design space, 4,500 placebo, 576 as designed). The probe's draws are the first 20 of that cell under its seed; they are kept and are not scored or looked at until every cell finishes.
• If the projection exceeds three nights, reduce in this order and record it under Amendments: Tier B to 100 draws per cell, then Tier A to 300 draws per cell. Cells and arms are never dropped to save time.
• The harness is committed before the full run starts. No scoring code runs until every cell and arm has finished.
• Smoke test with hard assertions before the run: every pick passes its cell's universe filter on its date; pick counts sit within 5 and the requested holdings; the product-convention percentile reproduces the row's stamped nfp exactly; the matched-count cloud equals draw_returns bit-for-bit when sizes are constant.
• Runs are resumable per cell (the sweep runner appends rows and skips finished draws). The harness writes out/run_manifest.json with panel hash, DB price max date, cell list, seeds, start and end times.

11. Known weaknesses, listed before the run

1. Survivorship: survivors-only before 2016 (training history only, 2009 to 2016), partial 2016 to 2018, thin 2019 and 2020, dense 2021 on. The engine therefore learned its early recipes from a sample missing its deaths; the direction of that effect on the method is unknown and it does not inflate any scored quarter. 43 symbol-reuse names and the builder's splice-guard exclusions are out.
2. Gross of trading costs, slippage, and taxes. Quarterly rebalance on panel dates.
3. Returns clipped to plus or minus 100% per quarter (the readout convention; this mutes both the best and worst single-name outcomes, for the software and the random portfolios alike).
4. Micro-cap cells are likely untradeable at any meaningful size; they are reported with that caveat.
5. The solvent gate removes most financials and REITs.
6. XLRE begins 2015-10 and XLC 2018-06; the engine falls back to SPY before those dates, so Real Estate and Communication Services sector-ETF comparisons are partly SPY comparisons in early quarters.
7. One panel vendor (the data vendor). An in-house fundamentals pack is not used here.
8. Ten years, one market regime dominated by large-cap growth. The seams exist to show how much the answer moves with the cut, not to escape this.
9. The method's shape was designed on this same history. The metric clusters, the flavor templates and their weights, the IC floor options, and the stability rule were chosen in research during 2026 on this panel. Each scored quarter's recipe is out of sample, but the ingredient list is not. Mitigations: every dial is varied across its full range so no single tuned setting is tested, the placebo arm shows what the ingredient list does without the statistics, and the piece says this plainly.
10. Companies trading under $1 today are absent from the database entirely, their history included, because the active universe was enumerated with a $1 floor. The fallen angels are missing. Both sides share it; it tilts the market comparison in our favor.
11. Seam 1's holdout contains the 2020 crash with 2019 and 2020 deaths mostly missing from the panel.
12. Current sector labels are applied to history; companies reclassified since are in their present sector throughout.
13. A dead company's final quarter records the return to its last exchange print, which for bankruptcies is usually above the ultimate recovery.
14. Our figures are gross; the ETF series are net of their expense ratios.

12. Reporting rules for the write-up

• Every number is an aggregate across draws. No draw is named as "the strategy."
• No stock is named. No prediction is made. No return is tied to subscribing.
• Verdicts are stated in section 8's words. "No edge" is published as such.
• The disclosure line appears on every channel: "Hypothetical backtested research statistics about our software, gross of costs and taxes; not investment, financial, or tax advice. Past or simulated performance does not predict future results."
• The commit hash of this scorecard is cited in the piece.

Amendments

Amendment 1, 2026-10-15, before any sweep draw was run. Original scorecard commit af41393. Changes, each made in place above and listed here:

1. Panel extended backward for training only. The shipped panel starts 2014-09-11, which left the engine 8 quarters of history at the first scored quarter and truncated every trailing window until 2019 to 2022. The study panel now starts 2009-10-07 on the same 90-day grid (built with the shipped builder; scored quarters verified identical, section 3). Scored quarters, seams, and every bar are unchanged. Reason: survivorship bias inflates a result only when survivors-only quarters are scored; none are. The engine learns its early recipes from a sample missing its deaths, disclosed in section 11.
2. Solvent gate threshold derived instead of ported. The original MeanZscore > 1.8 was justified with a wrong mechanism ("undefined for banks and REITs"; Z is present for 99.6% of rows and is simply low for balance-sheet-heavy sectors) and turned out to be a median split. Replaced with a threshold from a pre-specified rule on the training-only years (section 4): MeanZscore > -1.282. The rule, its inputs, and its output were fixed and run before any sweep.
3. Reference cell added. A-all-solvent18 keeps the textbook 1.8 on the all-sectors, all-caps pool so the piece can compare. 88 cells, 26,000 draws.
4. Profit gate tilt disclosed. It keeps 29% of Healthcare.
5. F8 added. Training-window dial (expanding vs trailing 12/16/20/24) reported as an exploration finding.

Amendment 2, 2026-10-15, before any sweep draw was run. A soundness review of the design, in two parts: is the comparison fair, and is the process being tested the one the software embodies rather than a user turning every knob at random.

1. Stale-filer asymmetry removed. The engine always screened filers older than 135 days; the random cloud and pool average did not (1.6% of the pool, and on the training years those names earned about 3.5 points less per quarter). The freshness rule is now part of every cell's universe. freshness_trained becomes inert and is fixed on.
2. Random cloud matched to actual portfolio size (primary), with the product convention kept as a secondary figure and as the harness validation.
3. "NO EDGE" renamed NOT DETECTED, interval always printed.
4. Placebo engine arm added on the nine no-gate Tier A cells: the engine with its statistical metric test removed. Test T6 compares the design space against it. This is the direct test of "do the statistics add anything beyond using fundamentals at all."
5. As-designed arm added on the same nine cells: the Build tab's defaults with all 64 template structures enumerated. The ladder (random, placebo, design space, as designed) joins the headline.
6. T7 selection discipline added: the product's documented selection logic applied ex ante on each seam's selection span, scored on the holdout.
7. F9 turnover and F10 portfolio size added.
8. Disclosures 9 to 14 added, led by the fact that the method's ingredient list was designed on this same history.
9. Process rules added: harness committed before the run, probe draws kept and unseen, no scoring until every arm finishes, smoke-test assertions.
10. Draw count is now about 31,100 across all arms.