LLM Engineering: Part II Summary

We reach the end of this part with 12 chapters that covered very different ground — from the limits of a base model to how 16 Anthropic agents built a compiler without any LLM orchestrating anything. Before we move to Part III, it is worth naming something that kept repeating without us saying it out loud: every technique we covered gives the model more capability, and every time we gave it more capability, we had to add more verification.

The tour, chapter by chapter

Chapter	What we learned	The verification that showed up
1. Base-model limits	A base model does not know your domain, is not up to date, is hard to control, and loses precision with long context	Recognizing that the model is not trustworthy by default
2. Basic prompt engineering	The cheapest way to improve behavior, without touching weights	Still no formal verification — we trust the prompt to guide well
3. LLM judges and evals	Before trusting an improvement, we need to measure it systematically	First real piece of verification: the LLM judge and the test cases
4. Why (usually) not to fine-tune	When prompting is not enough, you touch the weights — with a risk of overfitting	Evals to detect overfitting and validate that the improvement is real
4b. LoRA vs. QLoRA	Parameter-efficient fine-tuning, lower risk and lower cost	The same evals, but the cost of iterating drops
5. RAG	Access to external knowledge via vectorized documents	Verification that retrieval is relevant (RAGAS)
5b. Vector databases	How to index and search among millions of chunks	Metadata filters, access control, audit
5c. MCP	Hands to act on the real world	Mandatory human confirmation before actions with real effects
6. Agentic flows	The LLM decides the order of the tools	Autonomy control (low/medium/high), explicit decision points
6b. Agentic architecture patterns	Most of the most-used patterns (sequential, parallel, loop) deliberately do not give the LLM the orchestration	Code controls the flow; the LLM only executes point tasks
7. Multi-agent systems (home automation)	More specialized agents working together	New risk — implicit trust; solution — confidence thresholds and cross-verification

The pattern is unmistakable: at every step, the system’s capability grows — more context, more knowledge, more action, more autonomy. And at every step, verification not only grows, it becomes more sophisticated and more critical.

The fundamental tension: autonomy and verification grow together

This is the most important lesson of all of Part II: every time you give a system more freedom, you have to add more controls — not fewer.

Autonomy level	Verification required	Example
Text only (prompt)	Low — you trust the prompt to guide well	Summarize a document
+ RAG (external query)	Medium — you verify that retrieval is relevant	Ask about policies
+ MCP (action)	High — human confirmation for actions with real effects	Submit a vacation request
+ Flow autonomy (coordinator)	Very high — confidence thresholds, audit, checkpoints	The coordinator that routes sensitive complaints
+ Multi-agent (implicit trust)	Critical — cross-verification between agents	The Security Agent that does not blindly trust the Movement Agent

The most common error in AI projects: giving the system more autonomy without adding the corresponding verification — like giving an employee access to the safe without establishing dual-signature controls.

The counterexample that confirms the rule: Anthropic’s compiler

In Agentic architecture patterns we saw the experiment where 16 agents built a C compiler in Rust without an LLM orchestrating the flow — coordination by git, verification against GCC. Why did it work without an LLM orchestrating? Because the task was deterministically verifiable — GCC acts as a reference oracle, with no ambiguity and no subjective interpretation.

What happens when the task is NOT deterministically verifiable? “Get the house ready, I’m on my way” in our home-automation system (Multi-agent systems) — there is no “GCC of home automation”. The task is open-ended and subjective; there you do need an LLM orchestrating, and with it, all the verification controls we saw.

The practical lesson: the right question is not “use more or less AI?” — it is “does this task need reasoning over ambiguous language, or can it be solved with deterministic verification?” If it is technical and verifiable (compile code, validate a form), use the minimum AI necessary — cheaper, faster, more predictable. If it is open-ended and ambiguous, AI is necessary — but you have to design the verification with the same dedication as the agent itself.

A decision framework: what level of autonomy do I need?

Question	If YES…	If NO…
Is the task predictable, solvable with fixed steps?	Use sequential or parallel (code controls the flow)	Consider ReAct or coordinator (the LLM decides the flow)
Does it require access to external information?	Use RAG	Prompting alone may be enough
Does it require executing actions with real effects?	Use MCP + human-in-the-loop	It only needs to generate text
Does it handle sensitive data or high-risk decisions?	Demand high confidence thresholds, audit, human control	You can allow more automation
Can it be verified deterministically?	Use traditional code — you do not need an LLM orchestrating	You need an LLM to interpret the ambiguity

What is worth doing, in practice: always start with the lowest level of autonomy that solves the problem, and only add more when the data —evals, logs, metrics— show that the current solution is insufficient.

Why we built the compendium this way

The concepts that run through the 12 pieces of this part —how to evaluate systematically, how to think about autonomy trade-offs, how to design verification points— do not depend on which framework or protocol is in fashion. MCP was mentioned here as a relatively new standard, but the question it solves (how to standardize a model acting on the world, and what controls to put around it) is the same underlying question that existed before it had that name, and will keep existing under whatever name it takes later. The same applies to RAG, to embeddings, to agent patterns: the technologies change, the design principles remain.

Looking toward Part III: from architecture to infrastructure

What follows is infrastructure: how to serve these models in production, at what cost, with what latency guarantees, and how to monitor them to detect drift or failures. The question we take from this part into the next: every infrastructure decision —which model to use, how much batch, when to self-host— is, at bottom, the same tension we just named here, applied to resources instead of to behavior: how much capability we give the system, and how much control we keep over it.

In Part III we will see: inference optimization (reducing latency and cost without sacrificing quality), observability (monitoring agents in production, detecting drift, auditing decisions), agentic security (protecting systems against manipulation and unwanted behaviors — the Finnbot case in depth), and Evaluation-Driven Development (EDD) — turning evaluation into a continuous engineering practice, not an occasional manual review.

Part II summary, in one paragraph

A base model is a plausible text predictor, but limited. With prompting, RAG, and MCP, we give it knowledge and action. With agents and patterns, we give it autonomy to orchestrate its own flow. But every layer of capability demands an additional layer of verification — evals, confidence thresholds, human confirmation, audit, checkpoints. The art of designing AI systems is not in giving the model maximum autonomy, but in balancing autonomy and verification so the system is powerful without being dangerous. And remember the compiler counterexample: sometimes the best decision is not to use AI to orchestrate, because the task is deterministically verifiable. The key question is always: how verifiable is this task, and how much control do I need to give up to solve it?

If you remember only one thing from all of Part II, let it be this: capability without control is not power — it is risk. Every time you add capability —more tools, more autonomy, more agents— you are adding risk. Verification is not an obstacle; it is the price you pay for having a system that actually works in the real world.