Datadog creates tests on its own and stops relying on scripts that break

Datadog entered technical preview with two features from its Digital Experience Monitoring suite. They are called Journey Monitoring and Bits Testing. The announcement came out on September 24.

However, access remained restricted to enterprise customers. It also requires the Real User Monitoring, Synthetic Monitoring, and Product Analytics modules.

The problem being addressed shows up in almost every company. Marketing teams track conversion drops while engineers watch infrastructure failures, each group in its own dashboard.

Journey Monitoring brings conversion and SLOs together in the same console

The idea is simple to understand. The tool maps real user activity alongside service level objectives.

In other words, journey start, completion time, and drop-off rate appear alongside RUM availability and synthetic uptime.

When a transaction fails, the map shows the extent of the damage. It also draws the upstream and downstream paths, illustrating how an authentication failure spreads to checkout and account management.

However, breached thresholds get visual emphasis. Color-coded markers point to every breached SLO on the experience map.

The detail drills down to the specific case. With an availability SLO breached on a purchase form, the team opens Session Replay or Session Explorer without leaving the journey record.

There is also a discovery feature. The system continuously evaluates production traffic and identifies emerging paths, allowing new flows to be recorded before complaints reach support.

Datadog: Bits Testing generates the suite from a URL

Here the approach gets bolder. The engineer provides a web address or natural language instructions.

After that, an agent crawls the application, maps viable transactional routes, and produces executable test collections. Coverage includes browser interactions, network calls, and APIs.

However, the reasoning behind this approach is well known. Script-based synthetic tests break whenever someone changes the layout or introduces non-deterministic elements.

The combination of deterministic and goal-oriented testing

The design mixes two strategies. Deterministic routines validate rigid sequences.

This category includes payment endpoints, authentication contracts, and fixed checkout logic. After all, these flows need exact verification.

Goal-oriented routines work differently. They store the goal in natural language instead of a static sequence of clicks.

During execution, the agent decides the path. The examples cited include completing a round-trip flight booking or getting a response from a conversational assistant.

If a button changes position, the agent adapts the route. Likewise, a recommendation engine with variable components no longer breaks the test.

What Datadog says about the strategy

Yanbing Li, the company's chief product officer, summed up the vision. According to her, AI should be present in infrastructure, security, and now in the user journey.

Li describes the two features as complementary fronts. One continuously tests journeys, while the other discovers and observes the customer's most critical paths.

What to evaluate before adding this to your workflow

First, think about the execution environment. An agent that navigates the application needs realistic test data and proper isolation.

Second, define where each approach fits. Payment flows call for deterministic verification, and exploratory journeys can work with natural language goals.

Third, handle false positives carefully. Adaptive testing raises many findings, and poor triage wears the team out quickly.

Also, review the execution cost. An exploring agent consumes much more than a simple script running on a fixed interval.

Finally, keep human review over the generated suite. A test created automatically still needs to make sense for the business.

Where this fits into the Datadog industry movement

The pattern is becoming clearer. The previous week, SmartBear brought the BearQ agent into Jira. Before that, PractiTest launched a release readiness score.

Quality tools are shifting from script execution to autonomous exploration. Meanwhile, it's worth watching for independent reliability data on these agents.

Follow our profile on Instagram!