With the rise of AI the potential for automation has never been greater. When applied correctly, it can save significant time and effort. In many data & ai teams, I see AI being used individually for coding tasks, while its potential to enhance and automate the DevOps process remains largely untapped. And that is a missed opportunity.
Engineers spend a significant amount of time on activities that are necessary, but don’t directly add value to the business. Microsoft Research (Kumar et al., 2025) concluded that developers spend substantial time on activities such as pr/code reviews, monitoring, and communication. All activities which are suitable for partial automation.
So the question isn’t: Can AI help developers write code? It should also be: How can we leverage AI into the DevOps to eliminate the work around the code?
So let’s try to embed AI in the DevOps Process
In this article, I demonstrate a proof of concept for using an AI agent to automate part of the Pull Request review process. The goal is not to replace an engineer performing the review. Instead, the agent acts as an initial reviewer that can:
- Evaluate pull requests against predefined coding standards
- Identify potential breaking changes
- Highlight issues before a human reviewer spends time on the PR
- Provide comments directly on the pull request
This creates an additional layer of automated quality control within the existing development workflow.
Foundry, Azure Databricks & Azure DevOps as a foundation
To mimic an analytics environment, I use Databricks on Azure. I created a pipeline inside of a Databricks Declarative Automation Bundle that ingests and processes sample data through a simple medallion architecture. Each catalog represents an environment (dev, uat, prd), while each schema represents a layer (bronze, silver, gold).
Azure DevOps is used to manage the development workflow. Azure Repos is used to store the Databricks Declarative Automation Bundle code in dev, uat, and prd branches.
The AI agent is created in Azure AI Foundry, and is given a set of rules for my medallion architecture, and access to the Azure DevOps repository to read the full code, and comment on PRs.
Together, this results in the following flow:
- A user makes a PR to the dev or to the uat branch.
- After the PR is created, an Azure DevOps pipeline is triggered which prompts the PR-Review-Agent to take a look at a specific pull request.
- The Pr-Review-Agent looks up the PR number, and checks the committed code based on the instructions, the provided rules, and the overall code repository.
- The agent comments on the PR, giving the developer an initial opinion based on the defined rules, without taking time from another engineer. Including issues, breaking changes, dependencies, or code rule violations.
Results of our PR review agent in four scenarios!
To test the agents I tried 4 scenario’s. One that doesn’t break any rules, and three that either introduce breaking changes, or don’t follow the rules I have set for my medallion architecture (attached to my agents knowledge base).
Scenario 1: Adding a column to my silver table that does not violate any of my rules. The PR review agent is comfortable with the change and sees no risks / dependencies.
Scenario 2: Breaking a rule in bronze. I specified that columns in bronze needs to be the same as the source. However, in this example I try to rename a column. That is not allowed, and would also break the silver & gold table. The PR review agent flags the breaking change successfully! The PR review agent flags the breaking change successfully!
Scenario 3: Breaking a rule in silver. The coding rules state that silver columns should be in CamelCase. However, in the silver table one column does not follow that convention which is incorrect and should also break gold. The agent correctly identifies this mistake as well!
Scenario 4: Introducing a non-existent column in gold. In gold I use a column called DiscountAmount which is not present in silver.
Again the agent highlights the issue successfully. It is flagged as a minor issue, which I don’t agree with since it could break the view for actual business consumers. But more instructions with the definition of minor and breaking changes could solve that in practice.
Conclusions & Learning
The agent performed well against the test scenarios. And based on the PoC I believe that a PR review agent can result in significant time savings for engineers. In addition, this use case can be extended to many other areas in the DevOps process to support engineers in their work.
As an example, a developer assistant agent can already prevent a lot of coding mistakes before the PR review. And a monitoring/ root cause analysis agent can make sure that bugs and incidents are resolved quickly which is important for continuity.
There are a few learnings….
- Context is key: The instructions, tools, and added documentation are most important for the end results. During the PoC the agent sometimes ran into errors due to the MCP server that could not find files in my code. Specific instructions on how to find files helped. In real-life scenarios a setup like this would also require constant updates to the documents that the agent uses in the decision making.
- Cost: In total I did 18 runs (excluding failed runs) which resulted in 489.1K tokens. Costs were less then 1 euro on Foundry models. Very cheap, but do keep in mind that the agent reviewed very simple PRs in a very basic setup. In an enterprise setup this cost will increase drastically.
- Scaling: For scaling this across a department additional work would be required on token rate limits, model selection, and availability of the agent (queueing).
- Using AI makes life easier: During this PoC I used Claude to support on pipeline creation in Databricks and DevOps. It significantly reduced the time to a working example. Although it was less strong on integration (required roles for service principals) and newer API standards.
- Human oversight remains important: This PoC flags clear violations, breaking changes, and changes opposed to guidelines. This can be a tool that helps senior engineers, but not replace their role. Human engineers have much more context on timelines, business objectives, and governance. Therefore, they should always be in the lead.
I published the full code on my GitHub account. This includes the Databricks code, the Azure Foundry files, documentation, and a step-by-step explanation of the setup. If you want more information about the use case or setup, feel free to reach out!