THE CIRCLE / PRACTICE NOTES
This feed gathers field notes, open questions, and ideas from the guides. Every thread returns to a concrete job instead of talking about AI in the abstract.
Our support-summary prompt kept drifting, so I mapped instructions, ticket history, policy excerpts, and output rules separately. The missing piece was not wording: the model never received the latest refund exception. That map saved us another round of prompt polishing.
I tested my release-note assistant on six ordinary cases, two messy inputs, one empty change log, and one security-sensitive case. The tenth case stopped the launch, but it also gave me an exact guardrail to add. A small deliberate set was more useful than fifty random examples.
Writing the rejection conditions first changed our brief. We now reject a product description if it invents a feature, omits the source SKU, or hides uncertainty. Reviewers stopped debating whether the tone merely ‘felt right’ and started checking the same contract.
LEARNING ROUTE / STAGE 05 · COMMUNITY
A useful discussion names the task, conditions, observation, and next move. That lets another learner judge whether the experience transfers and continue testing from the linked guide.
A REPRODUCIBLE FIELD NOTE
A note worth discussing names the task, conditions, exact change, observed result, and next test. It can be brief, but it must let another practitioner judge whether the lesson transfers.
HOW TO RESPOND
Begin by confirming the conditions under which the lesson holds, then identify an evidence gap or counterexample. Do not infer a model, industry, or all users from one result, and do not present preference as a universal rule.
Delivery cases became steadier, but intense language was still mislabeled as refunds.
The observation covers one input slice and does not yet show stable classification. Next, hold intent constant while changing tone, then hold tone constant while changing intent. If the label follows tone rather than intent, expand cases and acceptance rules before lengthening the prompt.
Most of the 40 historical messages were classified correctly.
That result can support a pilot for these four queues, but it does not transfer to automated replies, which introduce customer commitments and new authority. Keep automatic writes disabled and add mixed intent, sensitive data, and unable-to-decide stop cases.
Our support-summary prompt kept drifting, so I mapped instructions, ticket history, policy excerpts, and output rules separately. The missing piece was not wording: the model never received the latest refund exception. That map saved us another round of prompt polishing.
I tested my release-note assistant on six ordinary cases, two messy inputs, one empty change log, and one security-sensitive case. The tenth case stopped the launch, but it also gave me an exact guardrail to add. A small deliberate set was more useful than fifty random examples.
Writing the rejection conditions first changed our brief. We now reject a product description if it invents a feature, omits the source SKU, or hides uncertainty. Reviewers stopped debating whether the tone merely ‘felt right’ and started checking the same contract.
A useful next exercise would be a grader-calibration pack: three answers that look similar but should receive different decisions. It would make the lesson easier to use when two reviewers disagree on what counts as supported.
The one-page brief exposed a disagreement before our client workshop. Operations wanted a recommendation; legal expected a source-backed summary with no recommendation. Naming the acceptable output made that conflict visible while it was still cheap to resolve.
The fit matrix ruled out AI for a fixed scholarship deadline lookup, but not for comparing eligibility language across programs. Would you split those into two steps—deterministic retrieval first, then an AI-assisted comparison with citations?
Separating source material from conversation history was the useful move for me. Our team had been calling both ‘context,’ which hid the fact that approved brand claims and a user’s earlier guesses were entering the model through different trust paths.
The matrix would be even easier to run in a product review with one counterexample per risk level. Teams often agree an error is possible but disagree on whether it is reversible; a filled comparison could make that trade-off concrete.
I used the runbook on an image-labeling workflow and discovered our ‘stop’ action prevented new jobs but did not preserve the failed batch. Adding a saved-state checkpoint made the recovery test feel like a real operating rehearsal rather than a document exercise.
We are placing review before customer-facing actions, but the reviewer still sees only the drafted message. Should the checkpoint also show the source evidence, model confidence notes, and the exact action that will run after approval?
There are no discussions under this filter yet.
POST TO THE CIRCLE
Linking a guide shows where your judgment began and makes its working conditions easier to reproduce.