Building My Own Tool Scanner — Part 2: When Looking Right Isn't the Same as Being Right

At the end of Part 1, I had a camera mounted above the scanning area and a fairly simple goal:

Turn pixels into millimetres.

I didn't need the camera to understand what a ratchet was.

I didn't need AI or object recognition.

I just needed to be able to point at a position in the image and reliably say:

That point is here in the real world.

My first attempt at doing that was considerably simpler than the system I'm using now.

Two L-shaped markers.

Surely that would be enough.

The first calibration target

I created an A4 calibration sheet with two known L-shaped references positioned apart from each other.

Each L had known dimensions, and because I knew where those references should be in the real world, I could locate them in the camera image and use them to work out how the scanning area had been distorted by perspective.

The theory was fairly straightforward.

The camera was mounted above the scanning surface, but there's no realistic way I was going to get it perfectly perpendicular.

Even a small amount of tilt means a rectangular workspace won't necessarily appear perfectly rectangular in the captured image.

One side might appear slightly wider than the other.

Parallel lines can begin to converge.

Distances measured in pixels can change depending on where you are in the image.

For a photograph, none of that really matters.

For something I'm eventually going to turn into a physical part?

It matters quite a lot.

Correcting the perspective

Using the known marker positions, I could calculate a perspective transformation and effectively straighten the scanning area.

The distorted camera view could be transformed into a nice rectangular workspace.

And initially...

It looked great.

Straight lines looked straight.

The workspace looked square.

The L-shaped references ended up where I expected them to be.

Measurements close to those reference points weren't bad either.

At this point I thought I'd basically solved the calibration problem.

Which, looking back, was optimistic. 😂

Then I started measuring things elsewhere

Once I started testing measurements across more of the scanning area, something didn't quite add up.

An object could measure correctly in one part of the image...

Move it somewhere else...

And the measurement would change.

Not massively.

But enough.

And that was a problem.

If I scan a tool on the left-hand side of the workspace and then move exactly the same tool to the right-hand side, I shouldn't get two different answers.

The physical tool hasn't changed.

My calibration has.

That immediately told me the perspective correction wasn't describing everything the camera was doing.

The problem I'd ignored

I'd accounted for the relationship between the camera and the scanning surface.

What I hadn't properly accounted for was the relationship between the lens and the image itself.

A camera lens doesn't necessarily map the real world onto an image perfectly.

Depending on the lens, straight lines and distances can become distorted differently as you move away from the optical centre.

And my two L-shaped markers simply didn't contain enough information to describe that behaviour across the entire workspace.

They could tell me:

“These known points should be over here.”

What they couldn't reliably tell me was:

“Here's exactly how the image behaves everywhere between and around them.”

That distinction became rather important.

Visually accurate isn't dimensionally accurate

This was probably the first big lesson from the project.

I'd been looking at the corrected image and thinking:

That looks right.

But the scanner doesn't care whether something looks right.

If a 100 mm object looks beautifully square but measures 101.5 mm, the calibration is useless for what I'm trying to do.

Likewise, being accurate around my calibration markers wasn't enough.

I wanted to be able to put a tool anywhere inside the usable scanning area and get essentially the same measurement.

That meant I needed to stop thinking about calibration as simply straightening an image.

I needed to actually map the behaviour of the camera across the workspace.

So the L-shaped markers were retired

The basic idea hadn't been useless.

Far from it.

The L markers proved that I could identify known geometry in the image and use it to correct spatial errors.

They also proved that the overall scanner concept was viable.

They just weren't giving me enough information.

I had a handful of known reference positions trying to describe an entire image.

What I really needed was known reference geometry everywhere.

So I went slightly overboard.

Instead of two L-shaped markers, I created a calibration target containing a 5 mm grid covering virtually the entire usable scanning area.

Now, instead of asking the camera where a couple of known points were...

I could ask it where thousands of them were.

And if I knew exactly where every one of those points should be in millimetres, I could start modelling what the camera was actually doing to the image.

The final system would eventually detect:

2,337 individual grid intersections.

Two L-shaped markers had just become 2,337 calibration points.

Because apparently there was no sensible middle ground. 😂

Next: Part 3 — Calibrating the Bloody Thing

Having thousands of reference points was one thing.

Proving that the resulting calibration was actually accurate was another.

So the next stage became a six-stage calibration and validation process involving known reference bars, hold-out testing, checkerboard testing, physical test pieces and final calibration verification.

Because if I'm eventually going to trust a camera measurement enough to manufacture something from it...

“Looks about right” isn't going to cut it.