I use three questions before a team commits to ML. The order matters because each question rules out a cheaper and more controllable approach.

Can the team write the rules?​

If the logic can be stated, write it in software.

Tax calculations, eligibility checks, workflow states, thresholds, and deterministic validations usually belong in code. A rules-based implementation is easier to test, explain, and change. It also tells an operator exactly why the system acted.

This choice can feel less ambitious than an ML project. That is irrelevant. The job is to produce a dependable outcome with the least unnecessary machinery.

A useful test is to ask the domain expert to work through several cases. If the expert can name the inputs, decision steps, and exceptions with consistency, the first version should probably be deterministic.

Can a pretrained model recognize the pattern?​

Some rules are hard to enumerate because they live in language, images, or a large number of small variations. A pretrained model may already handle the pattern well enough.

Common examples include:

• classifying the intent of a message
• extracting fields from familiar documents
• summarizing text for a person to review
• answering a bounded question from supplied context
• producing a draft that a person owns

The model still needs evaluation. "The model can do this" is not an acceptance criterion. Build a set of representative inputs, define correct behavior, and inspect the failure cases that could cause harm.

Retrieval and tool calls do not increase a model's judgment. Retrieval supplies information. Tools give the model a way to act. The surrounding system still needs to decide what the model may do, which result is acceptable, and when to ask a person.

Is the useful pattern specific to your data?​

Custom machine learning becomes a serious option when the rules are unknown, the pattern is absent from general training data, and the organization owns enough relevant examples.

Fraud detection, demand forecasting, equipment failure prediction, recommendation systems, and domain-specific risk scoring often fit this shape. The signal appears across many examples and is difficult to encode by hand.

Before training, check whether the data describes the decision you need to make. Volume cannot repair the wrong label, a biased collection process, or a target that arrives after the decision point. A smaller dataset with a sound definition can be more useful than a large historical dump.

Custom ML also creates an operating commitment. Someone must monitor drift, retrain when the environment changes, investigate regressions, and decide when the model should stop acting.

The decision in one sequence​

Use this order:

1. Write deterministic rules when the decision can be specified.
2. Evaluate a pretrained model when the pattern is common but difficult to enumerate.
3. Train a custom model when the useful signal is specific to your data and simpler approaches fail.

Hybrid systems are common. A model may extract intent while deterministic code applies policy. A custom score may prioritize work while a person makes the final decision. The boundaries matter more than the label attached to the system.

What engineers still own​

AI can produce a large share of implementation code. Engineers still own the choice of method, the test set, the failure policy, and the result in production.

Good applied AI work begins by refusing unnecessary uncertainty. Add a model after the system has earned the complexity.

These are the principles I return to while building products and teams.

Plan while the direction is uncertain​

Planning should reduce uncertainty. Once the direction is clear, another planning cycle often delays the evidence the team needs.

I plan when the team disagrees about the goal, the cost of reversal is high, or an untested assumption sits under the work. I push for delivery when the decision is reversible and customer use will teach us more than another meeting.

The leader's job is to tell those conditions apart and make the change of pace explicit.

Adapt to the person and hold the outcome steady​

People need different kinds of support. One engineer may need a clear boundary and room to work. Another may need faster feedback or help turning a broad problem into a first step.

Adapting the support does not require lowering the standard. The expected outcome, owner, and review point should remain clear. Fair leadership gives people what they need to meet a shared bar.

AI can produce code; people own the result​

Generated code makes implementation faster. It does not own intent, judgment, coordination, or the production incident.

Engineering leadership now places more weight on problem selection and review. Teams need to understand why a change exists, which behavior matters, how the system can fail, and who will make the call when the evidence is mixed.

Code volume is a weak measure of progress. A team that deletes an unnecessary service or replaces a model with a rule may have made the stronger engineering decision.

Put company goals into the work​

Company goals become useful when teams can connect them to product behavior and customer outcomes. A slogan at an all-hands does not create alignment.

I want a team to know which customer problem a project addresses, what evidence will change our view, and which tradeoff we accepted. That context helps people make local decisions without waiting for approval.

Personal growth should happen through meaningful responsibility. The company gets better work, and the person gains judgment that survives the current project.

Reward care and ownership​

Technical skill can grow through practice. Care is harder to create through process.

I look for people who notice when a result is incomplete, ask about the effect on customers, and stay with a problem through the unglamorous last mile. Leaders should protect that behavior from environments that reward visible activity over durable outcomes.

Ownership also needs boundaries. A person cannot own an outcome without authority, context, and time to act. Assigning responsibility without those conditions is delegation in name only.

Do the homework before asking for trust​

Teams read a leader's clarity as evidence of preparation. A vague answer may be honest, but repeated vagueness tells people that the decision has no owner.

When I do not have an answer, I say what I need to learn and when I will return with a position. That is more useful than filling the space with confidence.

The quality of a team's decisions cannot rise far above the quality of the context its leaders provide. Clear goals, named tradeoffs, and honest uncertainty give people something solid to work with.

These mental models help application engineers read a model card and turn architecture details into production questions.

Dense and mixture-of-experts models​

A dense model uses the same parameter set for each token. A mixture-of-experts model routes tokens through a subset of specialized feed-forward blocks.

The distinction changes how to read size. Total parameters describe stored capacity. Activated parameters give a better signal for compute per token. A large MoE model can activate far fewer parameters than its headline number suggests, though it may still require substantial memory and more complex serving.

Ask:

• How many parameters are active for each token?
• Can the serving stack place and route the experts efficiently?
• Does routing behavior remain stable on the target workload?
• Is the latency distribution acceptable, not only the average?

Dense models often offer simpler deployment behavior. MoE models can offer a useful quality-to-compute tradeoff when the serving system supports them well.

Context is capacity, not memory​

A context window states how many tokens a model can accept. It does not promise equal attention to every part of that input, reliable retrieval, or durable memory between calls.

Long context affects cost and latency through attention computation and the key-value cache. It also changes evaluation. A model can accept a long document and still miss a fact in the middle, combine unrelated passages, or follow an instruction from the wrong source.

Ask:

• How much context does the real task need?
• Which information must remain exact?
• Can retrieval reduce the input before generation?
• Does the model find and use evidence at the positions your application creates?

Treat persistent memory as a separate system. It needs rules for what to store, how to retrieve it, when to expire it, and how a user can correct it.

Attention design shapes long-context cost​

Architectures use techniques such as grouped-query attention, multi-query attention, sliding windows, and optimized attention kernels to reduce the cost of long inputs and decoding.

The model specification alone does not determine application performance. Hardware, serving framework, batch behavior, prompt length, output length, and cache reuse all affect the result.

Measure the full request path with representative traffic. A theoretical context limit has little value if the target latency or concurrency collapses long before that limit.

Tokenization is part of the bill​

Models do not read characters or words directly. Their tokenizer can represent the same source text with different token counts.

This matters for code, identifiers, multilingual content, structured records, and specialized terminology. More tokens increase input cost and consume context capacity. Token boundaries can also affect extraction and constrained output.

Run the tokenizer against real inputs before estimating cost. Averages from general English text will not describe every product.

Quantization trades precision for operating cost​

Quantization stores or computes model values with fewer bits. It can reduce memory use and increase throughput, but the effect on quality depends on the model, method, hardware, and task.

Small regressions can matter in tool arguments, structured output, numerical work, or rare edge cases even when general text still reads well.

Evaluate the quantized build against the same acceptance set as the original model. Compare failure categories and latency rather than relying on a single aggregate score.

Post-training defines behavior​

Two models with similar base architecture can behave differently after instruction tuning, preference optimization, safety training, and tool-use training.

Application engineers experience this as behavior: whether the model follows a schema, calls the right tool, asks for missing information, cites supplied evidence, and refuses outside the intended boundary.

Prompt work cannot repair every post-training mismatch. A smaller model trained for the required interaction may outperform a larger general model in a bounded workflow.

Failure behavior beats the leaderboard​

Average benchmark performance hides the cases that determine production trust.

Build an evaluation set around the system's decisions. Include ambiguous requests, missing data, conflicting context, long inputs, invalid tool results, and requests that should reach a person. Record how the model fails and whether the surrounding system can detect the failure.

The best model for an application is often the one whose errors the team can recognize and contain.

Keep a model decision record​

For each candidate, record:

1. The workload and representative inputs
2. Required output behavior
3. Context and tokenizer measurements
4. Latency, throughput, and cost under expected traffic
5. Failure categories and escalation policy
6. Security, privacy, and deployment constraints

That record turns model selection into an engineering decision. It also makes the next evaluation faster when models, prices, and product requirements change.