# How AI Agents Change the Work of an ML Engineer

[Watch on tuber](https://tuber.guzus.xyz/video/cqQi3-CuWL8/summary) · [YouTube](https://youtu.be/cqQi3-CuWL8)

**Score: 64/100 · skim**

Niels Bantilan is chief ML engineer at Union AI, maintains the Flyte orchestration platform (the transcript spells it "Flight") and created the Pandera data-validation library. He says coding agents now raise his team's velocity at least 3x on well-structured, linted, tested codebases. Humans still review PRs and still need to understand the code, and he sees Flyte's role shifting toward an "AI runtime" where agents compose compute-heavy tools instead of following hardcoded DAGs.

- **0:14-3:05** — **Guest and projects.** Host Dan introduces Niels: chief MLE at Union, core maintainer of Flyte, creator of Pandera, with a public-health informatics and biology background. Flyte grew up in MLOps orchestration, solving problems similar to Airflow's, but its core assumptions suit ML and AI workloads. It's being repositioned as an "AI runtime": the code and infrastructure that support an LLM inside a production system. Pandera validates many dataframe formats.

- **3:05-7:20** — **Open source vs. commercial, and how agents changed development.** Flyte is owned by the Linux Foundation and shepherded by Union, which gives it commercial pressure. Pandera is an uncommercialized "labor of love." Most Union developers now use agents for features and bug fixes. Niels says that as of mid-2026, agents work very well if the project already has linters, type checking, a style guide, consistent module layout and structured tests.
  - **His tools:** mainly Claude or Cursor. He's slowly moving to Pi and has tried OpenCode, partly to explore local "sovereign" AI.
  - **Local setup:** a DGX Spark running Qwen 80B Coder Next. It handles bug fixes and small scoped changes well, such as adding a CLI flag and propagating it through the codebase.
  - **Larger work:** for hour-long tasks he hands a sizable chunk of a PRD to Claude, which usually delivers.
  - **Caveats:** open-source harnesses need more work on prompting, skills and harness setup to match commercial tools. Agents also produce a lot of code to read, so PRs still need to stay small and reviewable.

- **7:20-10:30** — **Review practices and velocity.** The team still uses PRs and code review, but most reviewers now trust agents fairly highly and mainly look for code smells, security issues and performance problems. He estimates the speedup at "not 10x, but at least 3x." His debug loop is now the agent itself: he asks it to add tests for uncovered execution paths. He doesn't use a debugger directly anymore, and the team hasn't adopted Hypothesis property-based testing yet. For cluster work, he points the agent at the Flyte MCP config, which authenticates through the browser. The agent can then run workloads on Kubernetes and read logs and errors.

- **10:30-13:30** — **MCP vs. skills/CLI.** He could have written a skill describing kubectl commands. He chose MCP because Flyte already abstracts away Kubernetes, which he deliberately avoids touching ("I'm not a platform person"). The MCP exposes only the smaller, constrained API surface Flyte gives users, which is enough for an agent debugging loop. The agent can see an out-of-memory error, change the Flyte resource config, and even profile the dataset to estimate how much more memory it needs (e.g. about 4 GB). His point is that ML differs from software engineering mainly in compute: you can't easily unit-test or mock model training, because results only appear at scale after the loss drops for a while.

- **13:30-17:00** — **From data scientist to AI engineer.** Niels started in data science (visualization, modeling), then learned build systems, CI and testing while productionizing models at a startup. His analogy: classic ML was about training, and AI engineering is about what you do at inference time. Outputs grew from single scalars (regression or classification) to whole documents, images and video, so the space of inference applications exploded. Reinforcement learning is now bringing training back in a hybrid form, since rollouts need an inference server serving the latest policy. He's rereading Sutton & Barto because Union's customers are moving that way.

- **17:00-23:05** — **RL infrastructure and data movement.** Supporting RL in Flyte is about ergonomics. He argues agents and humans both still need good APIs. RL pieces have different compute needs:
  - environments may be CPU- or GPU-bound;
  - training and inference servers are GPU-bound.

  Flyte lets you write what feels like local code (for example, a loop over the environment) while each part runs in a separate pod, with data passed through a Rust-powered object store. Building for many companies, and having to serve both ML users and the platform engineers who maintain the system, means Union makes careful technology choices and leans on open source rather than custom GPU-stack work. Rust has been a big win. His sense is that agents are "okay but not as good" at Rust as at Python or JavaScript, and Dan notes they underuse the type system. Each Flyte task runs in its own container, with outputs written to S3-compatible storage. Using RustFS, internal benchmarks showed about 500 GB downloaded in roughly 4 seconds.

- **23:05-29:05** — **The bug that converted him, and the skill-atrophy worry.** Union built a system that loads model weights from S3 straight into GPU memory without writing them to disk first. An off-by-one error in the metadata index mapping tensor chunks to model architecture made the LLM output garbage symbols. He couldn't find it by hand; an agent given the logs found the missing "-1" in about 5 minutes. The downside: he now skims agent reports without fully understanding root causes, and feels he's losing the hard-earned skill of "finding a needle in a haystack." With so much context switching, he says he works more like a technical PM with a coding background. His advice is to hold on to deep understanding, which he calls the "last bastion" for humans. Dan adds that people still add value on structure and reusable abstractions, which agents don't prioritize because they "aren't lazy."

- **29:05-34:10** — **Agent quirks and internal agents.**
  - **A quirk:** rather than renaming a constant across the codebase, an agent kept an alias (old name assigned to the new value). The code worked, but it's a reason he doesn't fully trust agents.
  - **Review automation:** GitHub Copilot review is on in some repos and catches good things. There are no custom review or security agents, because they haven't felt a strong need.
  - **Node-pool agent:** modeled on Rick and Morty's single-purpose "butter robot," it only edits customer node-pool YAML in specific directories of one repo. It's triggered by a keyword in the customer's Slack thread or by a webhook from a node-change form. In about six months it has closed 100+ PRs, all reviewed by humans and checked by tests. Its mistakes, and the human edits fixing them, become context it can learn from through Git history.
  - **"Doxy":** watches SDK PRs, decides whether a change needs documenting, and opens a PR to the docs repo.

- **34:10-41:05** — **Agents beyond code, and when to move to Flyte.** Agents also help with a PRD repo, a go-to-market repo (blog posts and code examples), and a writing skill fed with his past writing to copy his voice. That draft beats writer's block. He then edits in Notion, sometimes rewriting large chunks, but the structure and narrative are usually right; he keeps the wording and taste for himself. Typical flow: start locally in Claude or another harness, where one-click OAuth connections to Notion, Linear, Slack and GitHub make proofs of concept fast. His rule for moving an agent to Flyte:
  - If a human must open a terminal and supply context for each task, keep it local.
  - If external programmatic events (Slack, GitHub, webhooks) trigger it and supply enough context, run it in Flyte.

  The friction is integration work (building Slack bots or GitHub apps, provisioning service accounts and API keys). He sees that as a gap Flyte should address.

- **41:05-47:30** — **The agent as workflow orchestrator, and code mode.** His estimate is that about 80% of agentic apps send most compute to an LLM provider (or a self-hosted open-weights model such as GLM, which was slow on Union's small instance), and the agent loop itself is light. Flyte stands out when the agent orchestrates heavy ETL, ML or hyperparameter-optimization pipelines: the old DAG's nodes become tools the agent combines freely, guided by Python type hints.
  - **Code mode:** Flyte uses Pydantic Monty, a Rust interpreter for a strict subset of Python that blocks network and IO. Flyte's file, directory and dataframe types (pointers to S3) let IO happen indirectly: control flow runs in the sandbox, while compute and IO run in separate pods managed by Flyte.
  - **Skills:** text that prescribes how tools should be sequenced.
  - **Range of uses:** from structured config-based HPO up to Karpathy-style "auto research," where an agent edits a training script until a metric hits a target.

- **47:30-52:35** — **Union's definition of an "AI runtime."** Dan stresses data locality (moving gigabytes across clouds is slow and expensive). Niels names three parts:
  1. **Durable workflows:** if step 98 of 100 fails, the run resumes there instead of recomputing the earlier steps.
  2. **Serving:** including spinning up inference servers at RL checkpoints for rollouts, as well as ordinary endpoints.
  3. **Multi-silicon:** multiple clouds, GPUs, TPUs and future accelerators, coordinated through Kubernetes.

  He expects agents to become compute-bound at the tool level too; Dan's example is compiling Rust. Flyte spun out of Lyft around 2019–2020 and Niels joined in 2021. Frontier-lab narratives gave him "existential moments," but practice convinced him human ML and AI engineers are still needed. An auto-research "Ralph loop" won't produce AGI, and labs still hire people. Dan points out they're also building large consulting arms.

- **52:35-1:01:10** — **Pandera roadmap, agent-friendly output, and close.** For Pandera, agents have been "a pure win": with a 2-year-old at home, maintenance no longer needs 3 a.m. sessions. He calls the project useful and "kind of boring," with no commercial pressure.
  - **Next direction:** validating quasi-tabular or multimodal data (vectors or images inside tables, Daft-style dataframes, vector DBs like LanceDB, RAG use cases) without bloating dependencies.
  - **Narwhals backend:** a contributor just shipped one, and a new release (spoken as "0320") adds validation for Ibis, Polars and PySpark through it. pandas support via Narwhals is still catching up, and LanceDB could be validated by exporting to Polars.
  - **Agent use:** Dan suggests agents running Pandera checks and fixing data from the errors. Niels says Pandera's concise plain-text errors are already agent-ready, unlike Pointblank's attractive HTML reports. He'll probably add HTML reports eventually, built with Jinja templates.
  - **Markdown vs. HTML:** both prefer Markdown for human–AI interfaces, since HTML clutters diffs; anything humans read needs to stay maintainable.

  The last ~20 seconds are the outro.

**Practical takeaways**
- Agents perform best on codebases that already have strict linting, type checking, a consistent layout and test conventions. Invest there before expecting large speedups, and keep agent PRs small.
- Give agents a narrow, purpose-built interface (like the Flyte MCP over raw Kubernetes) so they can run, observe logs and fix things such as OOM errors in a closed loop.
- Start agents locally to prove value. Move them to an orchestrator once they're triggered by events (Slack, GitHub, webhooks) and can gather their own context.
- Narrow, permission-scoped internal agents (one repo, specific directories, human-reviewed PRs, tests as gates) can safely handle repetitive customer-facing changes.
- Deliberately keep understanding root causes even when agents fix the bug; skimming agent reports wears down debugging skill.

## Audience signal

- No high-signal audience themes available.