Building My Own Tool Scanner — Part 3: Calibrating the Bloody Thing

So I'd gone from two calibration markers to 2,337 known points.

Slight escalation.

The 5 mm grid gave me enough information to model how the camera was behaving across virtually the entire scanning area rather than trying to correct everything from a couple of reference points.

But that created another problem.

How do I know the calibration is actually right?

It's easy enough to create a mathematical model that fits the data you give it.

That doesn't necessarily mean it'll accurately predict positions it wasn't given.

And ultimately, that's what I needed.

I'm not scanning the calibration grid.

I'm scanning tools placed somewhere inside the area that the grid calibrated.

So I didn't just need a calibration process.

I needed a way of testing the calibration itself.

That eventually turned into a six-stage calibration and validation process.

If one of the critical stages fails, the calibration doesn't continue.

No “close enough.”

It gets rejected.

Stage 1 — Find the grid

First, the system has to identify the calibration grid itself.

The grid uses 5 mm spacing, giving me known physical positions distributed across the scanning area.

Once the intersections have been detected, I effectively have thousands of pairs of coordinates:

Camera position → Real-world position

For example, the camera might see a particular intersection at some arbitrary pixel coordinate.

But I already know exactly where that intersection exists on the physical calibration sheet.

Do that thousands of times across the image and I now have enough information to start describing how the camera maps pixels onto the real world.

The final grid contains 57 × 41 intersections.

That's:

2,337 known calibration points.

Slightly more information than my two L-shaped markers. 😂

Building the correction model

Those points are then used to build a mathematical mapping between the camera image and the physical workspace.

I eventually settled on a cubic polynomial model.

Rather than assuming that one simple scale factor applies everywhere, the model can account for the fact that the relationship between pixels and millimetres changes slightly depending on where you are in the image.

Once that's built, I can take a position from the camera and ask:

Where is this point in millimetres?

That's the bit the scanner ultimately cares about.

But before I trust that answer, the calibration has to prove itself.

Stage 2 — Horizontal reference bars

The calibration sheet contains reference bars of known physical lengths.

The system locates the horizontal bars and measures them using the newly generated calibration.

These give me an immediate real-world dimensional check.

The permitted error is:

±0.5 mm

Anything outside that tolerance and the calibration fails.

It doesn't continue to the next stage.

Because if the scanner can't correctly measure geometry printed on the calibration target itself, there's not much point asking it to measure a ratchet.

Stage 3 — Vertical reference bars

Then I do essentially the same thing in the other direction.

Known vertical reference bars are detected and measured.

Again:

±0.5 mm maximum permitted error.

Again:

Outside tolerance = fail.

This is important because a calibration that behaves nicely across one axis isn't necessarily behaving equally well across the other.

At this point I've established that the generated mapping can reproduce known dimensions in both X and Y.

But there's still a fairly obvious problem.

I'm testing a model using the same grid that helped create it.

That's not enough.

Stage 4 — Don't let the model see the answers

This is probably my favourite part of the calibration process.

Instead of testing whether the model can reproduce data it already knows, I deliberately start taking information away from it.

The first test is a random hold-out test.

I remove 20% of the detected grid intersections at random.

The cubic model is then rebuilt using only the remaining 80%.

Now I give it the missing points back and ask:

Where do you think these should have been?

Because the model wasn't allowed to use those points when it was created, they provide a much better test of whether it can actually interpolate across the scanning area.

But I didn't stop there.

The checkerboard test

Randomly removing points is useful, but they're still scattered throughout the grid.

So I added another test.

This time, alternating sections of the calibration data are deliberately withheld in a checkerboard pattern.

Again, the model has to be constructed without them.

It then has to predict the missing areas using the calibration information surrounding them.

This gives me a much better idea of whether I've created a genuine mapping of the camera's behaviour or simply a model that's very good at reproducing its own calibration data.

Both tests have to remain inside the permitted tolerance.

If they don't:

Calibration failed.

Start again.

Stage 5 — Remove the calibration sheet

Up until this point, everything has involved the printed calibration target.

Eventually I wanted a completely separate sanity check.

So the calibration sheet comes out.

Five physical squares measuring 25.6 × 25.6 mm are placed at different locations around the calibrated workspace.

The scanner then measures them.

This stage is slightly different from the others because I deliberately don't use it as an automatic pass/fail gate.

It's there for me.

It's a physical sanity check that says:

You've done all the maths. Now measure an actual object.

The squares can also be checked independently with a vernier, so I'm no longer relying entirely on dimensions printed on the calibration sheet.

If a nominal 25.6 mm square physically measures around 25.6 mm and the scanner also reports around 25.6 mm...

We're getting somewhere.

Stage 6 — Build the final calibration

Once everything else has been completed, the results are brought together into the final calibration data used by the scanner.

That includes the mapping required to convert camera positions into physical coordinates and rectify the usable workspace.

My final rectified scanning area ended up at:

280 × 200 mm

with the image represented at:

4 pixels per millimetre.

That gives me a rectified workspace of approximately:

1121 × 801 pixels.

More importantly, the scanner doesn't just blindly assume that calibration remains valid forever.

During operation, the calibration data can be cross-referenced against the generated calibration files so that I'm not quietly using something that has unexpectedly changed.

Because moving the camera, changing the physical setup or altering the relationship between the camera and scanning surface potentially invalidates the thing everything else depends on.

So... how accurate was it?

Once the calibration and validation stages were complete, I could finally put some actual numbers against it.

The final calibration produced a mean residual of approximately:

0.178 mm

The 95th percentile was approximately:

0.411 mm

And the independent hold-out testing came back at roughly:

0.43 mm P95.

In other words, 95% of the tested calibration errors were below roughly half a millimetre.

Is that metrology-grade?

Absolutely not.

That was never the goal.

I'm using a normal webcam looking through a normal lens at a home-built scanning platform.

What I wanted to know was whether it was accurate enough to manufacture 3D-printed tool pockets.

And for that?

More than good enough.

Accuracy versus repeatability

There was another thing I cared about just as much as the absolute numbers.

Consistency.

If I scan something and get 100 mm, then move it somewhere else in the scanning area and suddenly get 102 mm, I don't have a useful scanner.

Even if I can find one magical spot where it measures perfectly.

The whole reason for abandoning the original L-shaped markers was that I wanted dimensional accuracy across the workspace, not just near a couple of convenient reference points.

The full-grid approach finally gave me that.

I could put an object in different areas of the scanning surface and get measurements that agreed closely enough for the manufacturing tolerance I actually needed.

That was the important bit.

From “looks right” to “prove it”

This entire stage changed how I approached the rest of the project.

Originally, calibration basically meant:

Correct the image until it looks square.

Now it meant:

Generate the model. Test it against known dimensions. Hide data from it. Make it predict that data. Test physical objects. Then decide whether I trust it.

Considerably more work.

Also considerably more useful. 😂

And after all of that, I finally had a camera system capable of telling me where something was in real-world millimetres with enough accuracy to actually use the result.

Which meant I could finally concentrate on the tool itself.

Except...

There was another problem.

The scanner needed to reliably identify the edge of the tool.

And apparently nobody had informed my collection of polished chrome tools that they were supposed to cooperate with a computer-vision system.

Next problem: illumination.

Next: Part 4 — Shiny Tools Are a Pain in the Arse

Calibration could tell me exactly where a detected point was.

Now I needed to make sure I was detecting the right point in the first place.

That meant dealing with shadows, reflections, chrome surfaces, changing ambient light and trying to create one clean silhouette that the software couldn't misunderstand.

Sometimes the answer isn't better image processing.

Sometimes you just need to give the software a better image.