What Business School Taught Me About AI Engineering

AI engineering.

What Business School Taught Me About AI Engineering

Years ago, at Pepperdine, I studied management science.

At the time, I learned about things like hypotheses, experiments, evidence, optimization, decision-making, and measurable outcomes.

Then I spent decades in engineering.

And somewhere along the way, a lot of that thinking got buried under delivery.

Build the feature.

Close the ticket.

Ship the release.

Fix the defect.

Move to the next sprint.

All useful work.

But AI is bringing me back to something I learned a long time ago:

Engineering should be experimental.

Not experimental in the sense of being reckless.

Experimental in the scientific sense.

You start with a hypothesis.

You design the smallest experiment that can test it.

You collect evidence.

Then you decide whether the hypothesis was right, wrong, or incomplete.

That sounds obvious.

But we don’t always work that way.

A Simple Example: ModelGate

I recently built a small experiment called ModelGate.

The hypothesis was simple:

Not every engineering task needs the most powerful and expensive AI model.

That sounds reasonable, but reasonable is not evidence.

So instead of just arguing about it, I built something that could test the idea.

ModelGate looks at an engineering intent and evaluates factors such as complexity, context, consequence, and capability.

Then it recommends the smallest model tier that should be sufficient for the work.

Now I can run experiments.

Does the recommendation stay consistent?

Can the lower-cost model actually complete the task?

Does the output still meet the success criteria?

How much money do we save?

Where does the approach fail?

That is the interesting part.

The goal is not to prove that my original idea was right.

The goal is to discover what is actually true.

AI Engineering Has a Lot of Untested Beliefs

AI engineering is full of assumptions right now.

More context is better.

More agents are better.

Bigger models are better.

More detailed specifications are better.

More RAG is better.

More orchestration is better.

Maybe.

But each of those is a hypothesis.

And hypotheses should be tested.

That is where I think engineering needs to change.

Instead of asking only:

Can we build this?

We should also ask:

What do we believe about how this should work?

What is the smallest experiment that could prove us wrong?

What evidence would make us change direction?

That is a very different engineering culture.

Evidence Over Opinion

I have seen the same thing in my work with Intent-Driven Engineering.

One question I had was whether highly detailed intents produced better software.

So I tested it.

Heavy intent.

Lean intent.

Same basic outcome.

Different levels of constraint.

The result surprised me.

In many cases, the leaner intent produced a better result, faster and with less wasted effort.

That changed how I work.

Instead of treating a detailed specification as automatically better, I now start with the minimum sufficient constraint and add detail only when evidence says it is necessary.

That is not philosophy anymore.

It came from experimentation.

The Workbench Is Also a Laboratory

I have been talking a lot lately about engineering workbenches.

A good workbench brings the right tools, evidence, process, and decisions into one organized place.

But I am starting to think there is another important idea inside that metaphor.

A good engineering workbench is also a laboratory.

It is where we test assumptions.

Where we compare approaches.

Where we measure results.

Where we decide what should become a reusable practice and what should be discarded.

That is especially important with AI because the technology is changing too quickly for us to build everything around fixed beliefs.

The best practices of today may be the bad habits of tomorrow.

Maybe Engineering Needs More Science

Software engineering has always had rigor.

Testing.

Architecture.

Code review.

Performance measurement.

Reliability.

But we often test the software more rigorously than we test the assumptions behind how we build the software.

AI gives us an opportunity to change that.

The cycle can be very simple:

Hypothesis → Experiment → Evidence → Decision → Improvement

And then repeat.

Looking back, I guess my management science degree finally caught up with my engineering career.

It just took AI to remind me why it mattered.

The future of AI engineering may not belong to the teams with the most tools.

It may belong to the teams that are best at asking:

What do we believe — and how can we prove it?