AI, LLMs & agents

Identify the actual function before evaluating an “AI-powered” product. Does it classify defects, estimate demand, generate text or choose an action? Each task needs different inputs and tests. Useful intelligence in one narrow task does not establish competence elsewhere. For a practical comparison, write down what enters the system, what result is expected and what a wrong result costs. This makes AI claims easier to evaluate without relying on vague descriptions such as “human-like understanding.”

References

• S032 · Google Developers — Machine Learning Crash Course
• S061 · Google Developers — Neural networks
• S033 · Vaswani et al. — Attention Is All You Need

Explore next

• What is an algorithm, and is every algorithm AI?
• How do machine learning, deep learning and neural networks differ?
• What is an LLM, and how is it different from a chatbot?

A rule such as “reject a part when its measured diameter exceeds the limit” is an explicit decision procedure. A learned classifier instead estimates a decision from patterns in examples. Both require correct inputs, tested assumptions and defined failure behaviour. Calling a process an algorithm says little about whether it learns, how complex it is or whether it is trustworthy. Ask which decisions are fixed in code and which depend on trained parameters.

References

• S032 · Google Developers — Machine Learning Crash Course
• S060 · scikit-learn maintainers — Metrics and scoring: quantifying the quality of predictions

Explore next

• How do machine learning, deep learning and neural networks differ?
• What is the difference between training and inference?
• What is machine vision, and when does it need AI?

Model choice should follow the task and available evidence. A small structured dataset may be well served by a simpler statistical model; image or language tasks may justify more complex architectures. Complexity adds training, evaluation and deployment requirements rather than guaranteeing better results. Compare against a baseline using held-out examples, relevant error costs and the hardware that will actually run the system. Do not infer biological awareness from the word “neural.”

References

• S032 · Google Developers — Machine Learning Crash Course
• S061 · Google Developers — Neural networks

Explore next

• What is the difference between training and inference?
• How do I know whether an AI model is good enough for my task?
• What is a vision model, and does it need language?

Keep training data, validation data and final test data appropriately separated. Otherwise, a system can appear accurate because it has already encountered information from the test. Record the model version, data preparation and decision thresholds with each evaluation. For production, consider processing time, memory, monitoring and rollback as well as accuracy. An application that stores a conversation for later retrieval has not necessarily updated the underlying model's parameters.

References

• S032 · Google Developers — Machine Learning Crash Course
• S063 · Hugging Face and contributors — LeRobot documentation
• S067 · OpenAI — Evaluation best practices

Explore next

• How do I know whether an AI model is good enough for my task?
• When should I use RAG, fine-tuning or ordinary rules?
• How do imitation learning and reinforcement learning train robots?

For defect detection, measure missed defects and unnecessary rejects separately. For document extraction, check each critical field and whether the cited evidence supports it. Split results by product variant, language, lighting or other conditions that can expose weaknesses. Compare confidence scores with actual correctness, and monitor whether production data changes. Acceptance should specify thresholds and escalation rules before results are inspected, rather than selecting a flattering metric afterwards.

References

• S060 · scikit-learn maintainers — Metrics and scoring: quantifying the quality of predictions
• S064 · scikit-learn maintainers — Probability calibration
• S067 · OpenAI — Evaluation best practices

Explore next

• How should I validate a vision system before production?
• Why can an LLM give a confident but incorrect answer?

When comparing assistants, separate the underlying model from the surrounding application. Internet access, file handling, code execution, conversation storage and integration permissions are product features, not automatic properties of every LLM. Conversely, a model can be used behind the scenes for extraction or classification without any chat interface. Specify the task and required tools before choosing a product; otherwise, a model comparison can accidentally become a comparison of unrelated application features.

References

• S033 · Vaswani et al. — Attention Is All You Need
• S065 · OpenAI — Key concepts — models, embeddings and tokens
• S068 · Anthropic — Building effective agents

Explore next

• What are tokens and context windows?
• Should I run a language model locally or use a cloud service?
• What is the difference between an AI assistant, an agent and a workflow?

Match the checking method to the claim. Verify a legal date against the operative text, a part number against the correct manual and a calculation with explicit inputs. Require accessible sources where evidence matters, then inspect whether they support the statement. A citation can also be irrelevant or wrong. For business automation, use deterministic validation for required fields and rules, with human review for consequential uncertainty. Do not equate an asserted confidence score with calibrated reliability.

References

• S066 · OpenAI — Structured model outputs
• S067 · OpenAI — Evaluation best practices
• S034 · Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Explore next

• How do I know whether an AI model is good enough for my task?
• Is private browsing, a VPN or local AI enough to make my activity private?

Count the instructions, conversation, retrieved passages, tool responses and requested output budget, not just the latest message. Token counts differ across languages and content, so use the relevant tokenizer or provider usage data rather than a universal words-to-tokens rule. Keep authoritative facts in concise, structured context and remove stale contradictions. Long-term application memory is a separate mechanism from the current context window; a previous conversation is not automatically available to every request.

References

• S065 · OpenAI — Key concepts — models, embeddings and tokens
• S033 · Vaswani et al. — Attention Is All You Need

Explore next

• What is an LLM, and how is it different from a chatbot?
• When should I use RAG, fine-tuning or ordinary rules?
• How can I reduce AI coding costs without weakening review and testing?

For a support assistant, retrieving current manuals is different from teaching a preferred response format. Start with a tested prompt and retrieval baseline before funding fine-tuning. Evaluate retrieval quality, source versions and unsupported claims separately from writing quality. Fine-tuning is not a reliable substitute for an up-to-date document store. A hybrid can work well: retrieve the relevant policy, extract structured fields and apply fixed eligibility rules outside the model. Validate the complete pipeline on real examples.

References

• S034 · Lewis et al. — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• S032 · Google Developers — Machine Learning Crash Course
• S066 · OpenAI — Structured model outputs

Explore next

• Why can an LLM give a confident but incorrect answer?
• What is the difference between training and inference?
• How are Jev's typed decisions different from an LLM returning JSON?

Test the same workload on both options, including long inputs and peak concurrency. Account for memory, model loading, updates, monitoring, electricity and staff time locally; account for usage charges, rate limits and outages remotely. Check exactly what leaves the device and what is retained. “Runs locally” does not establish that telemetry, model downloads or connected tools are offline. For sensitive workflows, document permissions and information flows rather than relying on a marketing label.

References

• S065 · OpenAI — Key concepts — models, embeddings and tokens
• S071 · NVIDIA Developer — Jetson modules
• S067 · OpenAI — Evaluation best practices

Explore next

• Can I run the whole NVIDIA robotics stack on a Jetson or Raspberry Pi?
• Does open-weight mean open-source, free and unrestricted?
• Is private browsing, a VPN or local AI enough to make my activity private?

Inspect the licence attached to the exact checkpoint and version, together with code, dependencies and any usage restrictions. Record attribution, redistribution and modification requirements. Also assess whether the release includes enough documentation to reproduce its stated capabilities. Compare hosted and self-hosted options on supportability and evidence, not simply the word “open.” This FAQ does not grant a licence or replace a review of the actual terms governing the planned use.

A licence claim for one component must not be applied to an entire AI stack.

References

• S040 · Convai Innovations — Laya model card
• S043 · NVIDIA — Isaac GR00T N1.7 repository
• S042 · NVIDIA — NVIDIA Cosmos

Explore next

• Can Laya run locally, and what licence applies?
• Are NVIDIA robotics models and tools all open-source and commercially interchangeable?
• Should I run a language model locally or use a cloud service?

Start with a bounded task, such as classifying incoming requests and proposing a response. A fixed workflow may be simpler to test than an agent that chooses arbitrary actions. Add model-directed planning only where it solves a real variability problem, and define permissions, stopping conditions and escalation. Evaluate completed tasks and unwanted actions, not merely whether the conversation sounds capable. A system that drafts an email is materially different from one authorised to send it or alter a customer record.

References

• S068 · Anthropic — Building effective agents
• S065 · OpenAI — Key concepts — models, embeddings and tokens

Explore next

• How should I limit an AI agent's access to files, accounts and tools?
• How can I reduce AI coding costs without weakening review and testing?
• What is an LLM, and how is it different from a chatbot?

A practical handoff can state: current behaviour; desired change; affected paths and data contracts; edge cases; validation commands; prohibited production actions; and expected diff, test results and remaining risks. Distinguish facts already established from assumptions the agent must verify. Avoid dumping unrelated repository history into every task. When the required design is still uncertain, resolve that uncertainty before requesting a broad implementation. This is an editorial workflow template, not a vendor-prescribed prompt or a guarantee that any agent will follow every instruction.

References

• S068 · Anthropic — Building effective agents
• S067 · OpenAI — Evaluation best practices

Explore next

• How can I reduce AI coding costs without weakening review and testing?
• Can an AI code review replace tests or a human reviewer?
• What should a useful technical handover or repeatable procedure contain?

Define an allowlist of tools and destinations, keep secrets outside ordinary prompts, and require approval for sensitive actions such as publishing, sending messages, spending money or modifying production data. Use a test environment and reversible changes where practical. Log actions without unnecessarily copying confidential content. Evaluate malicious or misleading inputs alongside normal tasks. Model-level instructions help describe boundaries, but technical permission controls should enforce them. These controls are a risk-reduction design, not proof that prompt injection or erroneous actions are impossible.

References

• S068 · Anthropic — Building effective agents
• S018 · NIST — Guide to Operational Technology (OT) Security, SP 800-82 Rev. 3
• S084 · UK National Cyber Security Centre — Top tips for staying secure online

Explore next

• How can I recognise phishing, fake support and malicious copy-paste instructions?
• Is private browsing, a VPN or local AI enough to make my activity private?

Record a compact project state with the current revision, unresolved decisions and next task. Use targeted searches instead of repeatedly reading the entire repository, and preserve useful test evidence between milestones. Stop repeated repair attempts when the failure needs a new diagnosis. Compare representative completed tasks, including human review time and regression risk. No provider prices or savings percentages are assumed here; measure your actual usage and confirm current billing terms before setting a budget.

References

• S068 · Anthropic — Building effective agents
• S065 · OpenAI — Key concepts — models, embeddings and tokens
• S067 · OpenAI — Evaluation best practices

Explore next

• What should a useful prompt for a coding agent contain?
• Can an AI code review replace tests or a human reviewer?
• What are tokens and context windows?

Review the actual diff against the requested behaviour, then run relevant checks on the same revision. Verify interface changes, migrations, error handling, permissions and negative cases. Ask the agent to distinguish tests it executed from tests it merely recommends. A passing suite does not prove that an untested requirement works; a second model agreeing with the first is not an independent runtime test. For sensitive changes, retain an accountable reviewer and a rollback or recovery plan.

References

• S067 · OpenAI — Evaluation best practices
• S068 · Anthropic — Building effective agents
• S066 · OpenAI — Structured model outputs

Explore next

• What should a useful prompt for a coding agent contain?
• How can I check whether a spreadsheet or small automation is trustworthy?
• Why can an LLM give a confident but incorrect answer?