← Back to the notebookTinyML
A working note

Before I blame the model, what did the microphone send it?

The microphone test has a smaller job than the classifier. Its code sets 16 kHz, 32-bit left-channel I²S, with word select on GPIO 15, data on 16 and clock on 14. It reads blocks into an int32 buffer, then prints a value called SoundLevel. Before treating that number as evidence about a word, I want to understand the operations that produced it.

The conversion is specific: shift each sample right by 14, narrow it to int16, take its magnitude and average those magnitudes. That gives me a precise question to carry into the rest of the pipeline: are the test and the inference application giving the same meaning to the microphone numbers?

The line that makes me stop is the conversion from the 32-bit values. The code shifts them by 14 bits. It’s a small-looking operation with a large effect on the scale of the numbers. If I copy it without understanding why it is there, I can pass along audio values that don't mean what I think they mean.

A 32-bit variable describes the space available to store a number. It doesn't automatically mean every one of those bits is useful microphone signal in exactly the arrangement I expect. The microphone format and the way the controller reads it both matter. That's why I want the component notes and the code together rather than an explanation based only on the type name.

As a basic arithmetic example, shifting a positive integer right by one bit divides it by two while discarding anything that cannot remain as an integer. Fourteen positions correspond to a factor of 16,384. It shows why a shift isn't a harmless formatting change. The position and range of the signal need to justify it.

Signed values deserve care too, because audio varies on both sides of its centre. I don't want a conversion that makes a waveform look as if it only moves one way, or a cast that loses the information I meant to keep. I'm reading the order of the operations instead of assuming that any combination of shifts and types will give the same samples.

The sketch prints an average amplitude indication. That’s a useful smaller target before involving classification. If the value changes, it tells me something about the data path. It still doesn't tell me whether the captured audio has the right shape, whether it is clipped, or whether its scale matches the recordings used for training.

The code's mean of magnitudes avoids the cancellation that an ordinary signed average can produce. It’s still different from RMS and from keeping the waveform itself. That feels familiar from the meter project: a number has to stay attached to the operation that produced it. SoundLevel tells me which summary was calculated, rather than describing every detail of the audio.

If a signal is pushed beyond the range a representation can hold, increasing the sound level no longer produces the expected shape. A graph or recording can help reveal that in a way a single average value may not. The original waveform contains information the summary has thrown away.

The 16 kHz sample rate gives another check I can do on paper. One second corresponds to 16,000 samples at that rate. If I expect a one-second window but collect a different number or label the rate incorrectly, the next stage may interpret the timing differently from how the audio was recorded. A mismatch can begin before the classifier has done any work.

I want to inspect the recording itself, ideally in a form that lets me listen and look at it, rather than only celebrate changing values on the serial monitor. The actual way to capture and transfer it needs to fit the board and program. That's the specific question I want my mentor's help with: how can we check what the application really supplies to feature processing?

The important comparison after this is with the training input. Which microphone produced those recordings? How were they scaled and processed? I can't assume a laptop recording and this live stream will arrive at the model in equivalent form just because both contain the word on.

I also want any plotted values labelled with their stage: raw input or converted sample. Otherwise a scale change could look like a change in the sound itself.

The inference sketch makes one concrete comparison possible. It also shifts the int32 microphone value right by 14, but casts the result directly to float for its audio buffer. The microphone test first narrows it to int16. Those expressions are not identical, so I should not assume the printed test summary and the inference input have passed through exactly the same representation. That’s a source-level difference to understand, not proof that it caused the poorer live classification.

The useful result is a more precise way to read the test and a smaller set of questions. Before asking why the model chose the wrong label, I want to be able to say what sound, in what numerical form, I actually gave it.

Leaving this here for next time.All the notes ↗