Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Setting up Agentic AI

Slides

Agentic AI is the most significant change to software engineering ever. It is also still a fast-changing field; we’ll try to balance general advice and specific examples that might change.

Selecting a harness

The first thing you’ll interact with is the harness. These tend to come in three flavors:

Most harnesses come in all three flavors, you just pick the one you like interacting with best. Some harnesses are more feature rich in one flavor. For example, Copilot usually adds features to the editor plugin first. Claude Code and OpenCode favor the TUI.

There’s a second component to harnesses, and it’s really important: the system prompt. The reason most model providers have a harness is to provide a system prompt that is customized to that model. Model specific customizations are likely not really needed, especially for larger models, so feel free to explore if you can. But note that’s likely the biggest difference between the same model using different harnesses.

Some providers (like Anthropic) require you use their harness to use subscription coding, so you don’t get a choice. (Providers can cache the prompt tokens, reducing the cost and latency of the 10k-30k tokens of system prompt).

Harnesses provide tools, which are things the model can call to perform operations. Models are mostly smart enough to figure out how to use tools, and most harnesses provide a small set of useful tools, so this isn’t usually an issue except maybe for small local models.

This is what makes Agentic AI special, and more useful than a simple chat; the model can call tools, read output, and loop, fixing mistakes, just like a human would. Many other things in this course - tests, formatters, linters, type checking, and CI all feed into the agentic loop, letting the agent correct mistakes and produce high quality output.

Most harnesses have these features:

Selecting a model

There are different levels of models, with different costs / subscription usages associated with them. Small open source models can even run locally. Some models are faster than others, too. In general, if you are just learning about agentic AI, you should use a fairly powerful model, so you don’t hit model limitations. Once you’ve used it for a bit, then you can start matching model strength to the problem description. You often have an “effort level” toggle as well. Using a strong model with a high effort level on a simple task can actually overengineer sometimes.

Here’s a current breakdown of some current models:

Here are some suggested task breakdowns. You can always use a stronger model, this is just a recommended minimum:

Effort level

Many models (most commercial models and some newer open source models) provide an effort level selector. This controls how much a model “thinks” (the budget for the thinking trace) before answering. In general, a smarter model at a lower effort is more cost effective and writes better code. High effort explores a larger space, making it good for searching for things or fixing a hard bug.

Tips:

Next steps

Once you have a harness installed and signed in, the rest of this week covers what to do with it: