Building the Scoring Rubric
William Thomson (Lord Kelvin), Popular Lectures and Addresses (1891):
When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind.
Chapter 3 left you with ranked criteria and a paragraph-length definition of “great” for each one. This chapter turns those paragraphs into points. The rubric doesn’t replace judgment — it is judgment, written down cold, made repeatable, and made comparable across roles you’ll evaluate weeks apart. Without it, you recalibrate every time. And recalibrating every time means you renegotiate your own standards every time, which means the market gets to move your floor.
Mine is 100 points. Six dimensions. Four tiers from Strong Yes to Do Not Pursue. It took three rewrites before it stopped rewarding roles that looked right in a DM and felt wrong on the third interview.
Company names below are composites. The scores and the structural patterns are real; the labels are not. Gossip doesn’t teach triage. Structure does.
From throw-out to chase
Section titled “From throw-out to chase”Dealbreakers and the rubric are sequential, not parallel. The dealbreakers run first. If anything fires, the role never reaches the rubric. This matters because a role can look genuinely interesting — strong founders, great domain fit, compelling equity story — and still fail before you score a single dimension. Skipping the gate to get to the score is how you end up outreaching on roles you would have declined if you’d been honest with yourself an hour earlier.
This came up concretely with Company F: a middleware and integration play, domain-expert founder, adapter-layer work that matched patterns I’d already shipped in my side company. The founder reached out. We had a meeting scheduled. I was genuinely interested. And then the comp dealbreaker fired. No confirmed salary, equity-heavy, “base when funded.” Floor: $230K hard. Nothing to score.
That doesn’t mean you kill the meeting. The rubric didn’t close Company F off — it prevented me from going in without a number in my head. I walked in knowing a fractional structure was the only viable path until a funding event, and I asked about it in the first ten minutes instead of the last. The gate isn’t a closed door. It’s a forced conversation.
For roles that clear all gates, the rubric ranks what dealbreakers can’t distinguish. Two roles both remote, both above $230K, both tolerant of an active side role. Now what? The rubric answers that. It doesn’t ask “does this role exist?” It asks “does this role deserve your time?”
The 100-point structure
Section titled “The 100-point structure”Six dimensions. The weights came from the ranking exercise in chapter 3 — not from a template. If your ranking is different, your weights should be too. This is mine, at this stage of my life, in this search.
| Dimension | Points | What “great” earns the max |
|---|---|---|
| Compensation alignment | 25 | At or above $300K target; cash-primary; bonus meaningful |
| Role shape | 20 | Architectural authority, hands-on builder, not execution-only under a CTO |
| Leadership & culture | 20 | Technical CEO or CTO, founder-led, merit-based, low process overhead |
| Stage & momentum | 15 | Series B+ with a clear model, or seed/A with funded runway and technical founders |
| Technical domain fit | 15 | Direct match: agent orchestration, middleware, cloud-native, AI-native stack |
| Long-term upside | 5 | Equity meaningful at current valuation; path to VP/SVP or fractional-to-perm |
Compensation gets 25 points because it ranked first in chapter 3 and the floor is non-negotiable. But the gate only asks whether you cleared $230K. Scoring distinguishes between “passes the gate” and “hits the target.” A role at $235K passes the comp dealbreaker and earns maybe 10 points here. A role at $300K earns 20. $350K+ earns 25. The rubric rewards not just clearing the floor but clearing it with room.
Role shape gets 20 because the last thing I want is a title that sounds senior and feels like middle management. The dimension asks: do I have architectural authority, or am I executing someone else’s decisions? VP Engineering reporting to a deeply engaged CTO who makes all the real calls is a different job from VP Engineering with genuine ownership. The rubric can’t fully answer this from a job description — but leadership profiles, org charts, and early-call questions fill in the picture.
Leadership and culture gets 20 because culture eats everything. The single most costly pattern I’ve named — I call it the Par Pattern — is a non-technical founder, or one whose technical contribution peaked at “I used to code,” who puts an MBA or ex-consultant in the CTO seat and runs engineering by committee. It looks fine on paper. It is a specific hell in practice. A technical CEO who shipped a product before the Series A, alongside a CTO who co-founded the company, scores the max here. A CEO with an MBA and a light coding background whose CTO’s public trail shows no engineering craft after the early career docks hard.
Stage and momentum gets 15 because the earlier the company, the more upside on paper and the more volatility in practice. Neither is inherently wrong. But the rubric scores for my current calibration, which at this stage of life weights against anything that requires me to wait three years for a liquidity event that might not materialize.
Technical domain fit gets 15 because I have a specific stack — agent orchestration, middleware adapters, event-driven systems, AI-native tooling, cloud-native infrastructure — and I am not trying to pivot out of it. A company in this exact space scores the max. Adjacent scores partial. A company where “AI” means a thin wrapper on a third-party SaaS scores low.
Long-term upside gets 5 because it matters, but it is the dimension most susceptible to storytelling. Great founders tell compelling upside stories. Rubrics don’t get seduced by them.
Four tiers
Section titled “Four tiers”The tier structure is binary within the tier: you are not half a Strong Yes. Every scored role lands in one tier and the tier dictates the action.
| Tier | Range | Action |
|---|---|---|
| Strong Yes | 85–100 | Outreach today |
| Yes | 70–84 | Outreach this week |
| Maybe | 50–69 | Watchlist — revisit if something changes |
| Do Not Pursue | 0–49 | Archive |
Two roles scored during an active sweep illustrate what the tiers look like with real numbers attached.
Company A, VP Engineering: 71 — Yes. Technical CEO and co-founder CTO — both had shipped and sold real systems before this company. The Par Pattern does not fire. Builder culture is explicit in how they talk about engineering, not just in marketing copy. Series B with institutional backing. Comp in a band above the $230K floor and below the $300K target. Domain adjacent to my stack (infrastructure SaaS with an AI-native engineering push); moderate fit, not a direct hit.
What kept it out of the Strong Yes tier: the VP Engineering role reports into a deeply engaged co-founder CTO. That means execution ownership without full architectural authority — the role shape dimension pays for it, losing points in the category that weights second only to comp. The right move is outreach, but “outreach this week” not “clear the afternoon and write a DM right now.” 71 is honest. Acting like it’s an 85 would be a rationalization.
Company B, VP Engineering: 59 — Maybe. Commerce / revenue-optimization SaaS, Series A. Comp listed in a range whose floor sat a few thousand under the dealbreaker, though the top of the range cleared it. Relevant enough to hold for a phone call. CEO background: MBA plus light product-engineering history. The Par Pattern gets flagged — not definitively fired, but audible. Public employee reviews mixed, with engineering-specific complaints about leadership style. Leadership and culture dimension takes a significant dock. This lands on the watchlist, not in outreach. If a first call surfaces a genuinely technical CTO, or the culture signals improve, it could move. As filed: archive and watch.
The gap between 71 and 59 is not about how much I would enjoy either role. It’s about what the rubric can verify in a first pass, without being emotionally attached to either. Calibration is only possible when you score without attachment.
Turning a feeling into a line item
Section titled “Turning a feeling into a line item”The first version of my rubric had twelve dimensions. Four of them were measuring the same thing — some variant of “does this company respect engineers?” — because I had not named the actual pattern I was trying to catch. The rubric rewarded companies that passed a vibe check, not companies that had structural indicators of the thing I’d call “culture.”
The rewrite that fixed it was naming the Par Pattern as a specific, scoreable concept. Par is not “non-technical CEO” as a general category. Some of the best outcomes I’ve seen came from non-technical founders who hired genuinely strong engineering leadership and stayed out of the way. Par is a specific configuration: a non-technical or lightly technical CEO who doesn’t hire a strong CTO, who manages engineering by consensus or metrics alone, and whose engineering team’s work is invisible to the board. You can check for it. You look at the CEO’s background for any indication of hands-on engineering past the college years. You look at the CTO’s public trail for code, papers, open-source contributions, or founding credit. You look at employee-review sites for “micromanagement” appearing in engineering-specific reviews. You look at the careers page for whether engineering is treated as a cost center or a value creator. Each check is a signal, not a verdict. The rubric turns the aggregate into a score.
The second rewrite separated comp floor (dealbreaker) from comp alignment (rubric dimension). I had originally made comp binary — you either cleared the floor or you didn’t. That worked for the gate but left the rubric treating $235K and $350K identically. Once comp became a scored dimension above the gate, the rubric started correctly distinguishing roles that technically pass from roles that genuinely hit the target.
The third rewrite removed stage as a proxy for quality. I had originally weighted early-stage down because I’d been burned on equity that evaporated. But I was baking in a past outcome rather than a current criterion. The correct version scores for runway clarity and technical founder presence — not for Series letter. A seed-stage company with two technical co-founders and twelve months of runway might score better on the stage dimension than a Series B with an MBA-led board micromanaging engineering decisions. Stage matters. It just isn’t a quality filter by itself.
Write it before you’re attached
Section titled “Write it before you’re attached”The Company F meeting almost went wrong not because the opportunity was bad but because I was interested before I was calibrated. The middleware fit was real. The founder’s domain knowledge was deep. Geography made me a natural candidate on paper. I was genuinely curious about the problem. And none of that changed the fact that the comp dealbreaker fired before the rubric even opened.
The lesson isn’t “don’t get curious.” It’s that the rubric has to be written — and the dealbreakers have to be internalized — before curiosity converts to momentum. If I’d gone into that meeting without a clear number in my head, I would have had a great conversation about adapters and domain plumbing and probably scheduled a follow-up before ever asking about the comp structure. The “salary when funded” answer would have arrived three weeks and two calls later, at which point I’d have been emotionally committed to a role that had not changed.
Company A is the subtler version of the same lesson. I was drawn to that company before I scored it. The founders were the kind I respect — deeply technical, built something real, sold it, came back for more. The builder culture signals were strong. The score came back at 71. Not a Strong Yes. Not even a Yes-outreach-today. A Yes-outreach-this-week, docked meaningfully on role shape because the VP Eng seat reports into a co-founder CTO who isn’t stepping back.
The rubric did not stop me from reaching out to Company A. It stopped me from overcommitting before I knew whether the role shape was salvageable in a first call. That is the practical value: not blocking enthusiasm, but protecting you from spending the first conversation lobbying for a role you haven’t been honest about.
Score cold. Not on the day the DM arrived. Not on the day you’re twelve rejections in and the market feels thin. Write the rubric before you have a specific company in mind, calibrate the weights on roles you don’t care about, and trust the number when it comes back lower than the feeling. The feeling is data. The rubric is the second opinion.
The worksheet
Section titled “The worksheet”A sanitized version lives in templates/rubric.md. Replace the dimensions, weights, and max-score anchors with yours. The columns that matter:
| Dimension | Max pts | Max-score anchor (one paragraph) | Weight rationale |
|---|---|---|---|
| your #1 criterion | __ pts | paragraph from chapter 3 | because it ranked first |
| your #2 criterion | __ pts | paragraph from chapter 3 | because… |
| … |
Rules that kept mine from drifting:
- Weights should reflect the ranking from chapter 3, not a template’s defaults. If comp isn’t your top criterion, it shouldn’t have the most points.
- Every max-score anchor is a paragraph, not an adjective. “Great culture” is not an anchor. A three-sentence description of the specific leadership configuration you’re chasing is.
- Score cold. Not against a role you already want. Not on a day when the market feels slow.
- If a role’s score is lower than you expected, read the dimension that docked it before you argue with the number. Usually the number is right.
- Revisit the weights after a sustained thin streak (chapter 10). Sometimes the market has moved; sometimes your criteria have.
The dealbreakers protect the floor. The rubric protects you from optimizing for the wrong ceiling.