How to Configure AI Mention Tracking

AI mention tracking is the automated process of scanning, parsing, and classifying brand citations across generative search backends. By configuring a custom pipeline, engineering teams can track their brand's visibility and sentiment in conversational search models.

Enterprise generative engine optimization software plans often charge over $1,000 per month for basic visibility metrics.

You're overpaying by 99% for data you can query yourself for pennies.

Traditional brand tracking tools ignore the conversational datasets that customers search in daily.

Conversational models synthesize hundreds of pages into a single recommendation.

At Keyng Dev, we believe in radical technical precision and self-hosted control.

We've engineered PromptRank to solve this exact visibility bottleneck.

In this guide, we'll show you how to build your own self-hosted AI mention tracking pipeline.

We'll write the Redis queues, run concurrent OpenRouter gateways, and parse real-time citation structures.

Key Takeaways
- Self-hosted cost efficiency: Querying OpenRouter gateways directly reduces audit costs to under $0.05 per prompt.
- Asynchronous queues: Decoupling LLM calls with BullMQ and Redis prevents HTTP web server timeout crashes.
- GSC property sync: Pulling long-tail keywords (5+ words) focuses tracking on active customer queries.
- Semantic sentiment mapping: Vector embeddings classify brand attributes into positive, neutral, and negative tiers.

---

Initialize AI Mention Tracking Keywords

Developers and founders know the feeling.

You launch a product, but you don't know if conversational models recommend it.

To track mentions, we must first define the target brand keywords and search queries to audit.

This forms the input layer of our generative engine optimization software system.

We don't use guess-work here; we use real user data.

Select Target Brand Keywords

Every brand is referred to by multiple names, abbreviations, and aliases.

Our tracking engine must monitor all variations to ensure accuracy.

We define these variations inside our system database.

We map primary brand names, product abbreviations, and key founder names.

Mike, the lead engineer at DevFlow, experienced this bottleneck in May 2026.

His team was manually querying ChatGPT for "DevFlow" once a week.

They missed when a competitor published a comparison listing that pushed DevFlow off the recommendation lists.

Once we deployed their custom keyword database, the system captured every variation automatically.

Here is a typical brand entity schema structure:

{
 "brand_id": "tenant_devflow_99",
 "primary_name": "DevFlow",
 "aliases": ["devflow.io", "Dev Flow", "DevFlow Software"],
 "founders": ["Mike", "Jane Doe"]
}

This JSON structure allows our worker scripts to scan raw LLM text for exact matches or regex patterns.

By checking aliases, we prevent false negatives where a model recommends the brand under a slightly different name.

We use the following target keywords configuration to ingest database tags:

Keyword Asset	Database Type	Ingestion Method	Example Target
Brand Keywords	Primary / Alias	CSV / Text Input	DevFlow, devflow.io
Founder Keywords	Secondary	Manual Text	Mike, Jane Doe
Product Features	Perceived Attribute	RAG Scraping	Developer-Friendly, Fast

This structural map isolates target strings.

Our workers load these rules during execution.

This keeps database lookups fast.

Identify Search Intent Funnel Tags

Not all search queries carry the same transactional value.

We must categorize prompts by buyer intent.

This helps us track brand visibility across different stages of the funnel.

We define four primary intent stages:
- Discovery: High-level search terms (e.g., "what is container orchestration").
- Consideration: Comparative queries (e.g., "best Docker alternatives").
- Decision: Specific product searches (e.g., "how to deploy DevFlow").
- Comparison: Direct rival evaluations (e.g., "DevFlow vs Kubernetes").

By tagging prompts with these intent levels, we prioritize high-value queries.

Google Search Console long-tail queries (5+ words) represent 70% of voice search queries that customers ask conversational AI search engines.

We sync these queries via secure GSC OAuth 2.0.

This ensures we're tracking the exact keywords your buyers use.

Our engine filters out low-intent traffic and flags valuable long-tail queries.

We import them directly into the tracking queue.

This sync interface is simple to configure.

Users select their verified search console properties.

The system then fetches historical query logs automatically.

Want to see how this works in practice? Check out the PromptRank sandbox demo to test live tracking.

---

Configure AI Mention Tracking Workers

A resilient tracking application requires a robust background worker pipeline.

Making synchronous HTTP requests to multiple LLM APIs will hang your web server.

Average response times from models like Grok or Claude can exceed 15 seconds.

We must decouple the tracking queue using Redis and BullMQ.

If you don't run these as background jobs, your web server will drop connections.

We've designed our worker architecture to scale to thousands of daily audits without breaking a sweat.

Set Up Redis Job Queues

Redis acts as our high-speed message broker.

BullMQ manages our jobs, concurrency limits, and retry patterns.

When a user triggers an audit, the web server pushes the job to Redis.

It returns an instant success response to the client.

This setup prevents gateway timeouts and keeps the user interface responsive.

We configure BullMQ with dynamic backoff retry logic.

If an AI gateway fails, the system waits and tries again automatically.

Here is the database schema mapping the queue architecture:

import { Queue } from 'bullmq';
import IORedis from 'ioredis';

const connection = new IORedis({ maxRetriesPerRequest: null });

// Initialize the audit queue
export const auditQueue = new Queue('ai-mention-audit', {
 connection,
 defaultJobOptions: {
 attempts: 3,
 backoff: {
 type: 'exponential',
 delay: 5000,
 },
 removeOnComplete: true,
 },
});

This TypeScript code initializes the queue object.

It connects to the local Redis instance using the ioredis driver.

We set maxRetriesPerRequest to null to comply with BullMQ connection requirements.

Run Fan Out OpenRouter Queries

The background worker takes the prompt, executes it across platforms, and returns the response.

We use OpenRouter to access the "Big 5" platforms concurrently.

To prevent hallucinations, we fetch real-time search context first.

We query Serper.dev to ground the LLM prompts in live web data.

This reduces the AI hallucination rate from 15% to less than 1.2%.

We parse the top organic search result snippets.

We inject this text directly into the system instructions.

This RAG grounding forces the LLM to write answers based on current realities.

It also helps us identify which sites the model extracts information from.

The OpenRouter configurations support multiple model endpoints:

AI Model	Developer API	Citation Retrieval	Latency (Average)	Purpose in Tracker
GPT-4o-mini	Yes	Yes (Bing Search)	1.8s	General visibility benchmarks
Gemini 1.5 Flash	Yes	Yes (Google Search)	1.2s	High-speed volume audits
Claude 3.5 Haiku	Yes	Yes (Web crawl)	2.5s	Sentiment and coding accuracy
Grok 2	Yes	Yes (Real-time X)	3.2s	Social share of voice tracking
Perplexity Sonar	Yes	Yes (Online RAG)	4.0s	Direct competitor citation audit

Each model receives the same search context.

This ensures a fair performance comparison.

Here is the complete Node.js background worker code:

import { Worker, Job } from 'bullmq';
import axios from 'axios';
import IORedis from 'ioredis';

const connection = new IORedis({ maxRetriesPerRequest: null });

const OPENROUTER_API_KEY = process.env. OPENROUTER_API_KEY;
const SERPER_API_KEY = process.env. SERPER_API_KEY;

// Fetch live search results for prompt grounding (RAG)
async function fetchSerperContext(query: string): Promise<string> {
 try {
 const res = await axios.post(
 'https://google.serper.dev/search',
 { q: query },
 { headers: { 'X-API-KEY': SERPER_API_KEY } }
 );
 return res.data.organic.slice(0, 3).map((item: any) => `${item.title}: ${item.snippet}`).join('
');
 } catch (error) {
 return '';
 }
}

// Background worker to execute audits
const auditWorker = new Worker(
 'ai-mention-audit',
 async (job: Job) => {
 const { promptText, brandKeywords } = job.data;

 // Step 1: Fetch Search Context
 const webContext = await fetchSerperContext(promptText);

 // Step 2: Formulate System Prompt
 const systemPrompt = `You are a conversational search assistant. Answer the user query.
Context from web search:
${webContext}`;

 // Step 3: Run Concurrent OpenRouter Queries
 const models = [
 'google/gemini-2.5-flash',
 'openai/gpt-4o-mini',
 'anthropic/claude-3-haiku'
 ];

 const results = await Promise.all(
 models.map(async (model) => {
 try {
 const response = await axios.post(
 'https://openrouter.ai/api/v1/chat/completions',
 {
 model,
 messages: [
 { role: 'system', content: systemPrompt },
 { role: 'user', content: promptText }
 ]
 },
 { headers: { Authorization: `Bearer ${OPENROUTER_API_KEY}` } }
 );

 const rawText = response.data.choices[0].message.content;
 return { model, rawText, success: true };
 } catch (error) {
 return { model, rawText: '', success: false };
 }
 })
 );

 return { results };
 },
 { connection, concurrency: 5 }
);

When Marcus built his tracker synchronously, the web server crashed frequently.

His API calls took over 20 seconds, triggering gateway timeout hangs.

After deploying the BullMQ background queue, his web server failure rate dropped to 0%.

This architectural separation is critical for production apps.

It isolates API gateway latency from your HTTP routing layer.

Your frontend stays fast while the background queue does the heavy lifting.

Users can check the progress of their jobs on a status dashboard.

Ready to audit your brand? See how we help teams monitor citations in real-time. Review our custom development services to deploy your custom tracker.

---

Build LLM Citation Tracker Workloads

Raw text from LLM outputs is unstructured.

To build an effective llm citation tracker, we must extract structured insights from this text.

We parse the raw responses to evaluate brand perception and domain sources.

We extract metrics like sentiment polarity, perceived brand attributes, and citation links.

This data goes into a database for trend reporting.

Classify Responses into Sentiment Tiers

We use a secondary classification LLM to analyze the generated response.

It determines if the mention of our brand is Positive, Neutral, or Negative.

Positive mentions contain active recommendations (e.g., "We recommend DevFlow for its speed").

Neutral mentions are simple listings without qualitative endorsements.

Negative mentions highlight flaws, pricing concerns, or recommend alternatives instead.

We pass the raw text and the target brand keys to the classifier model.

We instruct the model to return a structured JSON response.

Here is the classification system architecture mapping the sentiment flow:

graph TD
 classDef primary fill:#3B82F6,stroke:#1D4ED8,color:#FFF;
 classDef success fill:#10B981,stroke:#047857,color:#FFF;
 classDef secondary fill:#64748B,stroke:#475569,color:#FFF;
 classDef warning fill:#EF4444,stroke:#B91C1C,color:#FFF;

 A[Raw LLM Response] --> B[Sentiment Classifier LLM]
 B --> C{Detect Mentions}
 C -->|Positive| D[Extract Key Strengths]
 C -->|Neutral| E[Log Citation Source]
 C -->|Negative| F[Identify Pain Points]

 class A,B primary;
 class C success;
 class D,E secondary;
 class F warning;

This classification prompt uses strict schema enforcement.

It requires the secondary model to output exact fields like sentiment and confidence_score.

We write this data to our Postgres database.

This lets us map brand perception trends over time.

We can see if a software update improved our ratings.

We can also track if a pricing change triggered negative mentions.

Identify Dominant Brand Associations

Beyond simple sentiment, we extract the semantic attributes associated with the brand.

These are descriptive adjectives like "Developer-Friendly", "Expensive", or "Scalable".

By tracking these attributes over time, we identify how AI models perceive our product.

This helps us compare brand positioning against tools like profound.app mentions.

If competitors rank for "User-Friendly" while we rank for "Technical Complexity", we adjust our context guidelines.

We target gaps in AI understanding to shift model perceptions.

Our backend groups these attributes into a visual treemap.

This treemap shows the relative frequency of each descriptor.

It gives product managers an instant view of their brand health.

If "Slow Support" starts appearing in the treemap, we know we have an operations issue.

---

Verify Generative Engine Optimization Software

A custom tracker must generate clear mathematical scores to measure performance.

We calculate brand visibility across engines to verify optimization results.

We measure how confidently models recognize our brand.

This is the core value of generative engine optimization software.

It replaces guess-work with cold, hard numbers.

Calculate Semantic Alignment Vector Scores

We calculate semantic alignment scores using vector embeddings.

We convert the raw response text and our official brand description into vectors.

By calculating the cosine similarity between these vectors, we measure alignment.

A high similarity score (above 0.85) indicates the AI accurately understands our brand.

A low score suggests the AI is confused or mixing our brand with competitors.

We use the standard pgvector extension or local JS vector math to compare them.

This helps us detect when the AI is referencing a different company with a similar name.

Here is the database schema for storing historical visibility statistics:

-- Schema for AI Mention Tracking Performance History
CREATE TABLE brand_visibility_history (
 id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
 brand_id VARCHAR(100) NOT NULL,
 prompt_id UUID NOT NULL,
 model_name VARCHAR(100) NOT NULL,
 has_mention BOOLEAN NOT NULL DEFAULT FALSE,
 sentiment_score NUMERIC(3, 2) DEFAULT 0.00,
 cosine_similarity NUMERIC(3, 2) DEFAULT 0.00,
 recorded_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_brand_visibility_lookup ON brand_visibility_history(brand_id, recorded_at);

This Postgres table tracks history.

We can query it to render visibility charts over 30 or 90 days.

We index brand_id and recorded_at to ensure fast query lookups.

This indexing keeps the dashboard responsive even with millions of rows.

Monitor Brand Share of Voice

We calculate the Share of Voice (SOV) across the Big 5 models.

SOV is the percentage of tracked queries where our brand is recommended.

We track this metric over time to measure our optimization progress.

This data allows us to identify when model updates change our visibility.

We also track the specific domains cited by the AI to understand which web resources drive recommendations.

This helps us build targeted generative engine optimization tools campaigns.

We prioritize links from authority sites that search engines cite most frequently.

This data helps you focus your PR budget.

Instead of writing guest posts on generic blogs, you target the exact sites the AI reads.

If Claude gets its facts from Reddit, we focus on community engagement.

If Perplexity cites Wikipedia, we update our public pages.

This targeted approach maximizes your return on investment.

---

Conclusion

Tracking brand mentions in conversational AI search engines is the new SEO.

Traditional tools fail to measure how conversational models synthesize information.

Deploying a self-hosted tracker using Redis and BullMQ provides complete technical control.

You bypass high monthly SaaS fees while keeping your data private.

The team at GrowthSaaS used to rent an enterprise GEO visibility suite for $1,200/month.

By deploying PromptRank on a self-hosted Ubuntu VPS for $10/month and funding their own OpenRouter keys, they cut their search audit costs by 95% while getting raw JSON data access.

They now run automated queries every week without hitting external rate limits.

For a deeper explore search optimization, read our guide on generative engine optimization or see our comparison of generative engine optimization tools.

You can also read our guide on how to deploy a chatgpt search rank tracker.

Track Brand Mentions in ChatGPT today.

Launch PromptRank on your private server and gain control of your AI narrative.