Project Sentinel: AI Bot Detection Cuts Ad Waste by

The persistent challenge of bot traffic in app analytics continues to skew marketing performance metrics, costing businesses substantial resources and distorting strategic decisions. Our recent campaign, “Project Sentinel,” aimed to precisely quantify and mitigate this impact using advanced AI bot detection techniques, in the end enhancing app analytics accuracy and reinforcing overall data hygiene. This initiative revealed a stark reality: ignoring sophisticated bot activity is akin to operating blind. How can marketers truly understand their return on investment when a significant portion of their reported engagements are fraudulent?

Campaign Teardown: Project Sentinel

Project Sentinel was conceived as a direct response to a growing discrepancy between reported app installs and actual in-app engagement. We suspected a significant portion of our paid acquisition budget was being siphoned off by non-human traffic, but lacked the granular tools to prove it definitively and, more importantly, to act on it. This campaign ran for six weeks, from Q3 to Q4 2025, with a primary objective of improving the integrity of our mobile app marketing data.

Strategy and Objectives

Our core strategy was twofold: first, to deploy a sophisticated AI bot detection platform to identify and categorize suspicious traffic patterns in real-time. Second, to use this newfound intelligence to refine our media buying and optimize campaign performance. We focused on a popular casual gaming app, “Galaxy Quest,” which had a broad user base and a significant ad spend across various networks.

• Primary Objective: Reduce fraudulent clicks and installs by 25% across all paid mobile ad channels.
• Secondary Objective: Improve ROAS by 10% through more efficient ad spend allocation.
• Tertiary Objective: Establish a baseline for clean CPL and CPA metrics.

The AI Bot Detection Platform

For this campaign, we integrated a third-party AI bot detection solution, AdVerif AI, directly into our app analytics stack. This platform uses machine learning algorithms to analyze over 200 data points per user session, including IP addresses, device IDs, user agent strings, click-to-install times, and in-app behavioral anomalies. It flags suspicious activity with a confidence score, allowing us to filter out low-quality traffic before it even reaches our attribution models. The setup process involved a two-week integration phase, configuring webhooks and APIs to ensure smooth data flow between AdVerif AI, our mobile measurement partner (AppsFlyer), and our internal data warehouse.

Creative Approach and Targeting

The creatives remained consistent with our ongoing “Galaxy Quest” campaigns: lively, short-form video ads showing gameplay, and static image ads featuring key character art and in-game rewards. Our targeting was broad, encompassing various demographic segments on Meta Ads, Google App Campaigns, and several programmatic ad exchanges. We deliberately kept the creative and targeting unchanged during the initial phase to isolate the impact of the bot detection system itself. This allowed us to establish a clear “before” and “after” picture regarding traffic quality, rather than confounding the results with creative performance variations.

Initial Performance Metrics (Pre-AI Deployment)

Before activating AdVerif AI, our campaign metrics for “Galaxy Quest” over a two-week period looked like this:

• Budget: $150,000
• Impressions: 15,000,000
• Clicks: 450,000
• CTR: 3.0%
• Installs: 30,000
• CPL (Cost Per Install): $5.00
• In-App Purchase ROAS (Day 7): 65%
• Cost Per Conversion (Trial Subscription): $25.00

These numbers, on their face, seemed acceptable. However, the disconnect between reported installs and subsequent user retention or revenue generation was a persistent concern. Our internal data showed a significantly lower percentage of day-1 active users than the install numbers suggested, a classic red flag for bot activity.

What Worked: Identifying and Eliminating Bot Traffic

The immediate impact of activating the AI bot detection system was significant. Within the first 72 hours, AdVerif AI flagged approximately 32% of our incoming clicks as fraudulent or highly suspicious. These were not just simple click farms. The AI identified sophisticated patterns indicative of device farms, click injection, and even attribution fraud where bots attempted to claim credit for organic installs.

We began by blocking flagged IP ranges and device IDs at the ad network level. For programmatic buys, we worked with our DSPs to exclude specific publishers and inventory sources identified as high-risk. This proactive filtering was critical. The real-time nature of the AI allowed us to react swiftly, preventing further budget drain. One particular ad exchange, which previously accounted for 15% of our daily installs, was found to have a bot rate exceeding 70%. Shutting down spend there was a painful but necessary decision.

Data Comparison: Pre-AI vs. Post-AI (First 3 Weeks)

The initial drop in reported clicks and installs was initially concerning. However, the subsequent increase in ROAS and the decrease in the cost per actual conversion (trial subscription) painted a clearer picture. Our “clean” CPL increased, which is an important point: it reflected the true cost of acquiring a human user, not a blend of human and bot traffic. This transparency is invaluable for accurate budget forecasting.

What Didn’t Work: The Learning Curve

While the overall outcome was positive, the implementation wasn’t without its challenges. Initially, our aggressive blocking of flagged sources led to a temporary dip in overall impression volume and reach. Some ad networks pushed back, questioning our data and claiming their inventory was clean. This required extensive communication and data sharing, presenting AdVerif AI’s detailed reports to justify our actions. It became clear that managing relationships with ad partners through this transition requires patience and irrefutable evidence.

Another hurdle was the dynamic nature of botnets. As we blocked certain IP ranges, new ones would emerge, or bots would adapt their behavioral patterns to bypass detection. This highlighted the need for continuous monitoring and algorithm updates from our AI solution provider. Relying on a static blacklist is not enough. The system must constantly learn and evolve.

Optimization Steps Taken

Based on the initial findings and challenges, we implemented several optimization steps:

1. Granular Budget Reallocation: We systematically shifted budget away from ad networks and placements with high bot rates towards those demonstrating consistently clean traffic and higher engagement. This wasn’t a blanket ban but a data-driven, iterative reallocation. For instance, we increased spend by 20% on a specific gaming-focused ad network that showed less than 5% bot activity, while reducing spend by 30% on a general audience network with a 45% bot rate.
2. Whitelisting and Blacklisting Refinement: Instead of broad blocking, we refined our approach to create dynamic whitelists of high-performing, clean sources and more precise blacklists for specific fraudulent patterns. This involved setting up automated rules within AdVerif AI to adjust bidding strategies based on real-time fraud scores.
3. Enhanced Post-Install Analysis: We integrated AdVerif AI’s data with our internal business intelligence tools to correlate bot scores with in-app events beyond just installs (e.g., tutorial completion, first purchase, day 3 retention). This allowed us to see how bot traffic impacted downstream metrics, further solidifying the case for aggressive fraud prevention. We discovered that even some “low-risk” flagged installs had zero in-app activity, confirming their non-human nature.
4. Regular Performance Reviews: Weekly meetings were established with our ad network representatives, sharing anonymized fraud data and collaboratively identifying problematic inventory. This fostered a more transparent and productive relationship, in the end leading to better quality traffic from those partners.

Final Results and Impact

By the end of the six-week campaign, Project Sentinel achieved its objectives and provided invaluable insights into our app marketing ecosystem.

• Fraudulent Clicks Reduced: 38% reduction in fraudulent clicks, exceeding our 25% goal.
• ROAS Improvement: Achieved a 15% increase in Day 7 in-app purchase ROAS for the “clean” traffic segment, surpassing our 10% target. This was a direct result of spending money on real users.
• Clean CPL: Our true Cost Per Install for human users settled at $6.25, a significant adjustment from the initial $5.00, but a far more accurate and actionable metric for future planning.
• Budget Efficiency: An estimated $28,000 in ad spend was saved over the campaign duration by preventing fraudulent engagements.

The most deep impact was the clarity gained in our app analytics. We could now trust our conversion data, allowing for more confident decisions regarding creative optimization, targeting adjustments, and budget allocation. This improved data hygiene has ripple effects across all marketing efforts, from retargeting campaigns to product feature development, as we are now analyzing the behavior of actual users, not automated scripts.

It’s my opinion that neglecting AI bot detection in 2026 is a critical oversight. The sophistication of ad fraud means manual detection is largely ineffective. You are simply leaving money on the table, and worse, making decisions based on faulty intelligence.

The adoption of AI for bot detection is no longer a luxury. It’s a foundational element of effective digital marketing. The precision it brings to app analytics accuracy allows marketers to reclaim wasted spend and focus resources on genuine user acquisition, fundamentally improving overall data hygiene and campaign performance.