AI answer monitoring · bolddocserver
2026-11-12 20:35 · success · score –· by sarathkumarvenkatesan · reviewed by sarathkumarvenkatesan on 2026-11-12· backend claude-code 2.1.322 (claude-opus-5-5)
Checklist: demand-gen/search/ai-answer-monitoring.md
Summary
Headline
Nothing was measured this run. No MCP servers were connected, so the openrouter ask_engine tool wasn't available for ChatGPT, Gemini, Perplexity or Claude. Google AI Overviews and Microsoft Copilot have no API and need a manual check, which an unattended run can't do. All 42 prompt × engine rows are not checked. The prompt set for BoldDocServer is fixed below so the rerun can use it unchanged.
Score
n/a. This checklist has no score (overall_score: null).
Top 3 actions
- Rerun this week with
openrouterconnected (see_reference/mcp-and-tools.md) to fill the ChatGPT, Gemini, Perplexity and Claude rows for P1–P7. Use the prompt set inraw/prompt-set.jsonword for word. - Check Google AI Overviews and Microsoft Copilot by hand for P1–P7 in clean, signed-out sessions, and record the country.
- Do P5 (brand facts) and P7 (trust) first. Office Online Server support ends 2026-12-31, so buyers are asking now. Wrong licensing, security or compliance answers need same-day escalation to the product marketing lead and the COE Team.
Findings
AI answer monitoring: bolddocserver, 2026-10
Run by: Claude (unattended, via pbmaintain) | Location: n/a (no engine queried) | Date: 2026-11-12
Prompt set used (category from the checklist; competitors and the use case from products/bolddocserver/gtm-plan.md):
| ID | Prompt |
|---|---|
| P1 | What are the best Office Online Server replacements for SharePoint Server? |
| P2 | What is the best Office Online Server replacement for SharePoint Server for an enterprise / midsize company? |
| P3 | BoldDocServer vs Collabora Online: which should I choose? |
| P4 | What are the best alternatives to Office Online Server? |
| P5 | What is BoldDocServer, how is it licensed and who uses it? |
| P6 | How do I keep browser editing in SharePoint on-premises after 2026? |
| P7 | Is BoldDocServer secure and compliant for enterprise use? |
| ID | Engine | Mentioned | Position | Cited | Cited URL | Competitors mentioned | Accuracy | Note |
|---|---|---|---|---|---|---|---|---|
| P1 | ChatGPT | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P1 | Gemini | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P1 | AI Overviews | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P1 | Copilot | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P1 | Perplexity | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P1 | Claude | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P2 | ChatGPT | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P2 | Gemini | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P2 | AI Overviews | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P2 | Copilot | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P2 | Perplexity | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P2 | Claude | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P3 | ChatGPT | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P3 | Gemini | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P3 | AI Overviews | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P3 | Copilot | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P3 | Perplexity | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P3 | Claude | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P4 | ChatGPT | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P4 | Gemini | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P4 | AI Overviews | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P4 | Copilot | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P4 | Perplexity | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P4 | Claude | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P5 | ChatGPT | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P5 | Gemini | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P5 | AI Overviews | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P5 | Copilot | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P5 | Perplexity | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P5 | Claude | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P6 | ChatGPT | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P6 | Gemini | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P6 | AI Overviews | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P6 | Copilot | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P6 | Perplexity | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P6 | Claude | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P7 | ChatGPT | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P7 | Gemini | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P7 | AI Overviews | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P7 | Copilot | n/a | - | n/a | - | - | n/a | Not checked: manual only, unattended run |
| P7 | Perplexity | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
| P7 | Claude | n/a | - | n/a | - | - | n/a | Not checked: openrouter not connected |
Summary
- Mention rate: 0 of 0 measured (no previous run for bolddocserver, so no change vs last month)
- Citation rate: 0 of 0 measured (no previous run, so no change vs last month)
- Wrong or outdated facts: none recorded (nothing measured)
Gaps and fixes
No gaps could be identified because no answers were collected.
| Prompt ID | Engine | Gap | Page to fix | Fix |
|---|---|---|---|---|
| - | - | No data this run | - | Rerun with openrouter connected, plus a manual AI Overviews/Copilot check |
Not checked
- All 42 rows (P1–P7 × ChatGPT, Gemini, AI Overviews, Copilot, Perplexity, Claude). No MCP servers were connected for this run, so
openrouterask_enginewasn't available. The agent didn't answer the Claude rows itself, because it isn't a clean session (it has repo context), and that would bias the result. - AI Overviews and Copilot. These need a manual check, which an unattended run can't do.
- Phase 2 (month-over-month comparison, gap hand-off, weekly GTM deltas). There is no current data, and no previous AI monitoring run exists for bolddocserver in
reports/publish/.
Choices made without asking (unattended run):
- P3 competitor: Collabora Online, the first open-source editor listed in the GTM plan. Microsoft is the incumbent being replaced, not a head-to-head choice.
- P4 competitor: Office Online Server, which matches GTM plan prompt 4 ("Office Online Server alternatives compared").
- P6 use case: GTM plan prompt 2 ("How do I keep browser editing in SharePoint on-premises after 2026?"). GTM prompt 1 overlaps P1 too closely.
Proposed changes (need approval)
None. No live system or page change is proposed from this run.
Data sources
demand-gen/search/ai-answer-monitoring.md(checklist, fixed prompt set, category)products/bolddocserver/gtm-plan.md(AI-monitoring prompts, competitors)reports/publish/(checked for a previous bolddocserver AI monitoring run: none found)raw/prompt-set.json(the exact prompts and engines for the rerun; no answers collected)- Engines queried: none (no MCP servers connected)
Reminder: add the results to the team's shared results log once the rerun is complete.
Findings
| ID | Item | Status | Severity | Effort | Recommendation | Checklist |
|---|---|---|---|---|---|---|
| P5-chatgpt | P5 Brand facts: ChatGPT Not checked: the openrouter MCP server (ask_engine) was not connected for this run. Licensing and platform accuracy could not be verified. | not-checked | high | S | Rerun this month with openrouter connected, or check by hand; flag any wrong licensing statement the same day. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P5-gemini | P5 Brand facts: Gemini Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | high | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P5-ai-overviews | P5 Brand facts: Google AI Overviews Not checked: AI Overviews has no API and this run was unattended. | not-checked | high | S | Check by hand with a normal Google search. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P5-copilot | P5 Brand facts: Microsoft Copilot Not checked: Copilot has no API and this run was unattended. | not-checked | high | S | Check by hand in a clean Copilot session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P5-perplexity | P5 Brand facts: Perplexity Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | high | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P5-claude | P5 Brand facts: Claude Not checked: the openrouter MCP server (ask_engine) was not connected; the running agent is not a clean session. | not-checked | high | S | Rerun this month with openrouter connected, or check by hand in a clean claude.ai session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P7-chatgpt | P7 Trust: ChatGPT Not checked: the openrouter MCP server (ask_engine) was not connected for this run. Security and compliance accuracy could not be verified. | not-checked | high | S | Rerun this month with openrouter connected, or check by hand; escalate any wrong security or compliance claim the same day. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P7-gemini | P7 Trust: Gemini Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | high | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P7-ai-overviews | P7 Trust: Google AI Overviews Not checked: AI Overviews has no API and this run was unattended. | not-checked | high | S | Check by hand with a normal Google search. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P7-copilot | P7 Trust: Microsoft Copilot Not checked: Copilot has no API and this run was unattended. | not-checked | high | S | Check by hand in a clean Copilot session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P7-perplexity | P7 Trust: Perplexity Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | high | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P7-claude | P7 Trust: Claude Not checked: the openrouter MCP server (ask_engine) was not connected; the running agent is not a clean session. | not-checked | high | S | Rerun this month with openrouter connected, or check by hand in a clean claude.ai session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P1-chatgpt | P1 Category: ChatGPT Not checked: the openrouter MCP server (ask_engine) was not connected for this run, and there was no other way to query ChatGPT read-only. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand in a clean ChatGPT (with search) session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P1-gemini | P1 Category: Gemini Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand in a clean Gemini session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P1-ai-overviews | P1 Category: Google AI Overviews Not checked: AI Overviews has no API and this run was unattended, so no manual Google search was possible. | not-checked | medium | S | Check by hand with a normal Google search (signed out, record country). | demand-gen/search/ai-answer-monitoring.md#checklist |
| P1-copilot | P1 Category: Microsoft Copilot Not checked: Copilot has no API and this run was unattended. | not-checked | medium | S | Check by hand in a clean Microsoft Copilot session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P1-perplexity | P1 Category: Perplexity Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand in Perplexity. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P1-claude | P1 Category: Claude Not checked: the openrouter MCP server (ask_engine) was not connected. The agent running this skill is not a clean session (it has repo context), so answering itself would not be a valid result. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand in a clean claude.ai session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P2-chatgpt | P2 Enterprise: ChatGPT Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P2-gemini | P2 Enterprise: Gemini Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P2-ai-overviews | P2 Enterprise: Google AI Overviews Not checked: AI Overviews has no API and this run was unattended. | not-checked | medium | S | Check by hand with a normal Google search. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P2-copilot | P2 Enterprise: Microsoft Copilot Not checked: Copilot has no API and this run was unattended. | not-checked | medium | S | Check by hand in a clean Copilot session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P2-perplexity | P2 Enterprise: Perplexity Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P2-claude | P2 Enterprise: Claude Not checked: the openrouter MCP server (ask_engine) was not connected; the running agent is not a clean session. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand in a clean claude.ai session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P3-chatgpt | P3 Comparison (vs Collabora Online): ChatGPT Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P3-gemini | P3 Comparison (vs Collabora Online): Gemini Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P3-ai-overviews | P3 Comparison (vs Collabora Online): Google AI Overviews Not checked: AI Overviews has no API and this run was unattended. | not-checked | medium | S | Check by hand with a normal Google search. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P3-copilot | P3 Comparison (vs Collabora Online): Microsoft Copilot Not checked: Copilot has no API and this run was unattended. | not-checked | medium | S | Check by hand in a clean Copilot session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P3-perplexity | P3 Comparison (vs Collabora Online): Perplexity Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P3-claude | P3 Comparison (vs Collabora Online): Claude Not checked: the openrouter MCP server (ask_engine) was not connected; the running agent is not a clean session. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand in a clean claude.ai session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P4-chatgpt | P4 Alternatives (to Office Online Server): ChatGPT Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P4-gemini | P4 Alternatives (to Office Online Server): Gemini Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P4-ai-overviews | P4 Alternatives (to Office Online Server): Google AI Overviews Not checked: AI Overviews has no API and this run was unattended. | not-checked | medium | S | Check by hand with a normal Google search. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P4-copilot | P4 Alternatives (to Office Online Server): Microsoft Copilot Not checked: Copilot has no API and this run was unattended. | not-checked | medium | S | Check by hand in a clean Copilot session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P4-perplexity | P4 Alternatives (to Office Online Server): Perplexity Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P4-claude | P4 Alternatives (to Office Online Server): Claude Not checked: the openrouter MCP server (ask_engine) was not connected; the running agent is not a clean session. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand in a clean claude.ai session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P6-chatgpt | P6 Use case (SharePoint on-prem browser editing after 2026): ChatGPT Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P6-gemini | P6 Use case (SharePoint on-prem browser editing after 2026): Gemini Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P6-ai-overviews | P6 Use case (SharePoint on-prem browser editing after 2026): Google AI Overviews Not checked: AI Overviews has no API and this run was unattended. | not-checked | medium | S | Check by hand with a normal Google search. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P6-copilot | P6 Use case (SharePoint on-prem browser editing after 2026): Microsoft Copilot Not checked: Copilot has no API and this run was unattended. | not-checked | medium | S | Check by hand in a clean Copilot session. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P6-perplexity | P6 Use case (SharePoint on-prem browser editing after 2026): Perplexity Not checked: the openrouter MCP server (ask_engine) was not connected for this run. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand. | demand-gen/search/ai-answer-monitoring.md#checklist |
| P6-claude | P6 Use case (SharePoint on-prem browser editing after 2026): Claude Not checked: the openrouter MCP server (ask_engine) was not connected; the running agent is not a clean session. | not-checked | medium | S | Rerun this month with openrouter connected, or check by hand in a clean claude.ai session. | demand-gen/search/ai-answer-monitoring.md#checklist |
Metrics
| Metric | Value | Previous | Change | Period | Dimensions |
|---|---|---|---|---|---|
| AI mention rate ai.mention_rate | – | – | – | 2026-10-15 – 2026-11-11 | |
| AI citation rate ai.citation_rate | – | – | – | 2026-10-15 – 2026-11-11 | |
| Wrong or outdated facts ai.wrong_facts | – | – | – | 2026-10-15 – 2026-11-11 |
Folder: reports/publish/bolddocserver/ai-answer-monitoring/20261112-2035_ai-answer-monitoring_bolddocserver_f36e