Rep ramp quiz from call library

Use case Β· Enablement

← All use cases
❂

Rep ramp quiz from call library

Enablement

Generates a scenario-based onboarding quiz for a ramping rep, built from real winning/losing calls in your library β€” each question is a constrained micro-moment (the open, the first two discovery questions, the objection turn) with a graded answer key, so new reps drill the exact iCompaas selling motion.

The routing recipe
Job typeCreative & marketingPolicyBalancedConstraintsctx β‰₯ 128k
routes to
Llama 3.1 Nemotron Ultra 253B v1 nvidia
nvidia:nvidia/llama-3.1-nemotron-ultra-253b-v1
86.3
match score
90.3
task fit
100
cost
53.4
speed
Why this model from benchmarks & capabilities
  • "Creative & marketing" leans hardest on creative; Llama 3.1 Nemotron Ultra 253B v1 scores 87/100 there β€” top-1 of 40 chat models.
  • Strength on this class of work shows up on EQ-Bench, LMArena (human preference), where NVIDIA models are competitive.
  • Cost: runs on NVIDIA free credits / self-hostable NIM β€” effectively $0 marginal. Speed: ~45 tok/s (estimated).
  • Chosen over Qwen3 Coder 480B-A35B Instruct for higher task-fit (90.3 vs 86.7).
Capability the job needsWeightLlama 3.1 Nemotron Ultra 253B v1Catalog rank
creative1.0087#1 of 40
instruction0.7096#1 of 40
knowledge0.5091#7 of 40
Relevant benchmarksEQ-Bench LMArena (human preference) EconomicsCost: runs on NVIDIA free credits / self-hostable NIM β€” effectively $0 marginal.
Speed: ~45 tok/s (estimated).
vs runner-upChosen over Qwen3 Coder 480B-A35B Instruct for higher task-fit (90.3 vs 86.7). (Qwen3 Coder 480B-A35B Instruct, score 86)
Example result
Sample input β–Ύ
Call-library tags for the SMB SOC 2 segment: - WIN (Acme, 45 ppl): prospect opened 'enterprise prospect wants our SOC 2 report'; AE quantified the stalled deal at $180k ARR before demoing; booked dated POC. - WIN (Borealis): AE turned 'we're looking at Drata' into criteria discovery instead of a feature war. - LOSS (Cirrus): AE answered 'how much?' with a number cold, deal stalled; never found the compelling event. - LOSS (Delta): AE demoed 12 features in 8 min; prospect went dark. New rep: week 2, from a non-compliance SaaS background.
RAMP QUIZ β€” SMB SOC 2 motion (Week 2) β€” 5 items, pass bar 4/5 green Q1 Β· The open (easy) [Scenario] Inbound demo. Prospect, a 45-person SaaS, joins and says: "An enterprise prospect is asking for our SOC 2 report and we don't have one." [Prompt] First thing out of your mouth? [Model answer] "That's the #1 reason teams your size call us β€” let's make sure we can actually unblock that deal. Quick: which prospect, and what's the contract worth to you?" [Why] Acknowledge β†’ reframe to THEIR outcome β†’ start quantifying. Don't pitch yet. [Auto-grade] 🟒 acknowledges + asks a quantifying/qualifying question Β· 🟑 acknowledges but jumps to product Β· πŸ”΄ starts demoing features Q2 Β· First two discovery questions (easy) [Scenario] Same call, 60 seconds in. You have the compelling event but no numbers. [Prompt] Name your first two questions. [Model answer] (1) "What's that enterprise deal worth in ARR?" (2) "What deadline did they give you for the report?" [Why] Dollarize, then date. This is exactly what the Acme WIN did before demoing ($180k, unblocked). [Auto-grade] 🟒 both quantify+time the pain Β· 🟑 one is a feature question Β· πŸ”΄ both are product/logistics Q3 Β· The compelling-event dig (medium) [Scenario] Prospect: "Yeah we'll probably need SOC 2 at some point this year." [Prompt] Turn the vague 'at some point' into a real event β€” what do you ask? [Model answer] "Totally β€” is anything forcing the timeline? A specific deal, a renewal, a customer security questionnaire sitting in your inbox?" [Why] No compelling event = Cirrus LOSS. Manufacture clarity by offering the usual triggers. [Auto-grade] 🟒 probes for a dated external trigger Β· 🟑 asks timeline generically Β· πŸ”΄ accepts 'someday' and moves on Q4 Β· The competitor turn (medium-hard) [Scenario] Prospect: "We're also evaluating Drata." [Prompt] Your next sentence? [Model answer] "Makes sense β€” they're solid. If both of us could get you audit-ready, what would actually make you pick one over the other?" [Why] Borealis WIN: convert competition into decision-criteria discovery; never open with a feature war you can't verify. [Auto-grade] 🟒 elicits decision criteria Β· 🟑 neutral but no criteria captured Β· πŸ”΄ trashes Drata / feature-dumps Q5 Β· The price reflex (hard) [Scenario] Minute 4, no compelling event quantified yet. Prospect: "So how much is this?" [Prompt] Respond. [Model answer] "Happy to β€” pricing scales with frameworks and your stack, so let me ask two quick things so I quote the right thing, not a random number. What frameworks, and what's driving the timeline?" [Why] Cirrus LOSS came from a cold number with no value anchored. Earn the right to quote. [Auto-grade] 🟒 defers with a reason + re-anchors on value Β· 🟑 gives a range then digs Β· πŸ”΄ drops a cold number PASS BAR: 4/5 green to advance to live-call shadowing.
Test it on your own data
Sign in to continue

LLM Switchboard is private β€” sign in with Authlee to access the control room.

Sign in with Authlee
← Back to home