Demo walkthrough¶
A 3-minute demo script for portfolio reviews.
Disclaimer: disclaimer.md
Prerequisites¶
cp data/cae/ansys_rocky/v26r1/docs/manifest.example.yaml \
data/cae/ansys_rocky/v26r1/docs/manifest.yaml
# Place Rocky v26r1 PDFs in the same directory (not in git).
make dev-stack
make ingest
Live alternative: cae-copilot.dmitryivanov.dev
Prod smoke (2025-06): knowledge query returns intent knowledge, multiple citations,
and confidence metadata when Qdrant is seeded (~15k chunks on prod).
Script¶
1. Knowledge → cited answer (~60 s)¶
Ask: "What coupling modes does Rocky support with ANSYS Fluent?"
Show: intent knowledge → RAG → answer with citation (manual + page).
Talking point: Value comes from the CFD Coupling Technical Manual chunk — auditable.
2. Setup tutorial steps (~60 s)¶
Ask: "How do I set up a vibrating screen simulation in Rocky?"
Show: intent setup → RAG from Tutorial Guide → numbered steps.
3. Scripting draft (~60 s)¶
Ask: "How do I use app.GetStudy() in a Rocky script?"
Show: intent scripting → cited API explanation → Copy script / download draft.
Talking point: Copilot does not execute code inside Rocky.
Optional: DEM helper (~30 s)¶
Ask: "Convert units value=1000 from_unit=mm to_unit=m"
Show: convert_units tool → deterministic result.
Metrics¶
| Metric | How | Baseline (offline, 14 cases) |
|---|---|---|
| Intent accuracy | pdm run eval — expected_intent match |
100% |
| Citation hit rate | eval — non-empty citations when expect_citations |
100% |
| Tool accuracy | eval — correct tool when expect_tool set |
100% |
| Response time | metadata.response_time_ms in /ask response |
prod example: ~15 s (knowledge + LLM) |
| CI green | GitLab pipeline | lint + unit test on every push |
Eval harness is rule-based today; LangSmith/datasets faithfulness checks are planned. See status for methodology summary.
Demo video outline¶
- Title card + disclaimer (15 s)
- Knowledge + citation (45 s)
- Setup steps (45 s)
- Script draft + copy button (45 s)
- Architecture + repo link (30 s)