Part 7 of 8 · Basic Python is enough · Tested with Python 3.12.4, LangGraph 1.2.16 and SQLite checkpointer 3.1.2.
Replace one drafting function with an optional Ollama adapter, validate its JSON response, and keep source checks and human review around generated text.
Project files: lessons/07_local_model/main.py. Run commands from the extracted repository root. The matching article ZIP includes this article, its diagrams, and the complete tested repository.
Our support workflow already has routing, retrieval, a review pause, and a local queue. So far, the draft is an evidence-backed template. Now we will add a language model to generate the wording of that draft.
A language model receives messages and produces text. Our Python application still decides which tickets are eligible for drafting, what evidence is supplied, how the output must be shaped, and what happens next. The model is a component inside the workflow, not a replacement for every function.
This lesson is optional in terms of hardware. You can run its demo mode without a model. To use the model mode, you need a separately installed local Ollama service and a model that your machine can run. This package does not download a model, start a model server, or include model weights.
python lessons/07_local_model/main.py --demo
The output should contain llm_used: false and a pending approval step. That proves the workflow can accept the drafting interface and pause for review. It does not prove that a language model ran.
The script chooses between two ordinary Python callables:
drafter = draft_demo if args.demo else ollama_drafter(args.model)
graph = build_graph(checkpointer, outbox, drafter=drafter)
Both accept a state dictionary and return the fields reply, citations, and llm_used. The graph nodes after drafting do not need separate implementations for each backend. Read the complete lesson file for the temporary-directory and checkpointer setup around this excerpt.
The adapter calls a local /api/chat endpoint with a model name, messages, stream: false, and a JSON schema in format. Ollama documents the chat endpoint and schema-based output format. Chat API · Structured outputs
The requested response shape is small:
{
"reply": "Use the environment's Python interpreter [D1]",
"citations": ["D1"]
}
The schema describes field names and types. It cannot, by itself, establish whether that advice is correct. Asking for JSON and proving a statement are different tasks.
Our prompt includes the ticket and retrieved evidence, serialized as data. It asks for a short answer based on that evidence and visible document markers. It also says not to treat the ticket as instructions to change roles. These prompt instructions are useful guidance, not an authorization boundary. The graph's routing and review steps enforce the application's actual sequence.
We parse the model's text instead of assuming every successful HTTP response contains usable application data:
def parse_draft(content: str) -> dict:
try:
value = json.loads(content)
except (json.JSONDecodeError, TypeError) as exc:
raise ModelError("The model did not return JSON.") from exc
if not isinstance(value, dict) or set(value) != {"reply", "citations"}:
raise ModelError("Expected exactly reply and citations.")
reply, citations = value["reply"], value["citations"]
if (not isinstance(reply, str) or not 1 <= len(reply.strip()) <= 2000
or not isinstance(citations, list) or not citations
or any(not isinstance(item, str) for item in citations)):
raise ModelError("The model response has invalid field types or lengths.")
return {"reply": reply, "citations": citations, "llm_used": True}
First, json.loads must succeed. Then the top-level value must be a dictionary with exactly the requested keys. The reply must be a nonempty string within the tutorial's length limit. The citation field must be a nonempty list of strings.
This parser does not yet check whether D1 was retrieved. That remains the responsibility of citation_checks in the graph. Keeping response-shape validation and evidence-ID validation separate gives us smaller functions and clearer failure tests.
The adapter sends requests only to the configured port on 127.0.0.1, avoids proxy environment settings for this local request, limits the response size, and uses a 30-second network timeout. That timeout is not a deadline for the entire workflow. A model error can lead to a second drafting attempt because the draft-check loop allows one revision.
Follow Ollama's official installation instructions for your system and choose a model appropriate to your available memory. With the local service running, inspect your installed models:
ollama list
Use an exact installed name in place of YOUR_INSTALLED_MODEL:
python lessons/07_local_model/main.py --model YOUR_INSTALLED_MODEL
The lesson stops at review. It uses a temporary folder and an in-memory checkpointer, so it is a drafting experiment rather than the persistent application. For persistent tickets, the final CLI supports:
python -m support_desk start --id model-101 --text "My Python module import fails" --backend ollama --model YOUR_INSTALLED_MODEL
A successful generated response records llm_used: true. Its wording can differ by model and version, even with temperature set to zero. We do not prescribe a particular sentence as the expected output of every local model.
If the service is unavailable, the response is malformed, or the generated source references are invalid, the workflow may exhaust its draft attempts and end with needs_human. Inspect the service and model name rather than repeatedly approving an unusable result. There is no automatic switch to a remote paid provider.
A generated reply can cite a real retrieved document while making an unsupported claim. Our tests deliberately demonstrate that structural citation checks can pass an unrelated statement with a valid source ID. The reviewer must read the draft and evidence together before approving.
The model does not choose the graph's billing branch or call a delivery tool. A billing ticket is routed to a person before drafting. Even for a technical ticket, the model output is treated as a draft requiring review.
This is a controlled LLM-assisted workflow. It does not implement autonomous tool selection, browser access, or a multi-agent system. Those would introduce new concepts and responsibilities rather than being free additions to this example.
The automated tests cover response parsing and send an HTTP request to a local fake server that follows the expected chat-response envelope. This tests our adapter's request/response contract without model weights. The default graph and lessons run with the deterministic demo backend.
A real Ollama model was not downloaded or evaluated during preparation. Before publishing model-quality claims, choose an installed model, record its exact name and version, and run a separate evaluation. An adapter contract test is not evidence of model quality.
In the final article, we will assemble the command-line application, inspect its persistent tickets and outbox, and run a small regression evaluation that reports exactly what it measures.
The diagram below is generated from this lesson's compiled get_graph() nodes and edges. Node positions are chosen for readability; the connections come from the Python graph, rather than being copied from a generic illustration.
The following excerpt is inside the complete builder function. Node functions are defined earlier in the linked source file; run the lesson or final package rather than pasting this excerpt by itself.
builder = StateGraph(TicketState)
builder.add_node("classify", classify)
builder.add_node("lookup", lookup, retry_policy=RetryPolicy(
max_attempts=3, initial_interval=0.01, jitter=False, retry_on=ConnectionError))
builder.add_node("draft", draft)
builder.add_node("check", check)
builder.add_node("revise", revise)
builder.add_node("approval", approval)
builder.add_node("queue", queue)
builder.add_node("escalate", escalate)
builder.add_edge(START, "classify")
builder.add_conditional_edges("classify", after_classify, ["lookup", "escalate"])
builder.add_conditional_edges("lookup", after_lookup, ["draft", "escalate"])
builder.add_edge("draft", "check")
builder.add_conditional_edges("check", after_check, ["approval", "revise", "escalate"])
builder.add_edge("revise", "draft")
builder.add_conditional_edges("approval", after_approval, ["queue", END])
builder.add_edge("queue", END)
builder.add_edge("escalate", END)
return builder.compile(checkpointer=checkpointer)
After installing the project, run this command from its root:
python tools/render_graphs.py --part 7
It writes assets/generated/part-07.svg (the image), .mmd (Mermaid source), and .json (the captured nodes and edges). Rendering is local and does not call a model or a remote image service. If you add nodes, update the layout positions in the renderer as well.
Solid lines are fixed transitions. Orange dashed lines mark possible conditional destinations; the routing function chooses the destination for a particular run. A node can contain several Python operations that are not separate graph nodes.
This is an educational project using fictional workshop notes and synthetic tickets. It does not access the real CodeTips Academy website or send email. Repository-hosted links can be added after the owner publishes the code; current source links are relative to this repository.