← Back to the notebookTinyML
A working note

A hundred recordings become one minute and forty seconds

The dataset view contains 100 one-second clips: one minute and forty seconds of sound. The split is 82 training examples and 18 test examples. That’s a concrete scale I can think about, rather than simply calling the dataset small or large. Each second is one of the examples behind the patterns I’m asking the model to recognise.

The training configuration also has a separate 20% validation allocation. That’s not another name for the 18 held-out test clips. I want those roles to remain clear when I read a result. Training changes the model, validation helps assess choices during development, and a test split has a different job. A percentage is useful only when I know which collection of examples it describes.

The screenshot shows 82 training samples and 18 test samples. Those are different jobs. Training examples help the model learn its patterns. Test examples are meant to check behaviour on data held aside from that learning. There’s also a validation allocation during training, shown separately in the configuration. I need to keep all three roles straight when I talk about a result.

It would be easy to see one score and call it accuracy without mentioning which examples were used. But a result on the training data, a validation result and a held-out test result don't answer the same question. A validation percentage cannot be substituted for a separate held-out test result just because both describe correct predictions.

Dataset screenshot showing 82 training and 18 test samples.
Dataset screenshot showing 82 training and 18 test samples.

The split is also more than a choice of counts. Imagine a situation where several clips are pieces of almost the same recording, made by the same person in the same conditions. If closely related clips appear on both sides of a split, the evaluation might be easier than the intended real use. That isn't something I've established about this dataset. It’s a reason to ask how the examples were collected and separated.

Who spoke? Which microphone was used? Were the words recorded at a similar distance and level? What was happening in the background? These details might sound like extra notes, but they help define the task the model has actually seen. A filename list can't tell me the amount of variety by itself.

I also want to examine the labelled recordings without letting the label explain the sound for me. ON and OFF sound obvious when written in capitals. A clip might start late, cut off a sound, contain extra speech or include a pause that dominates the one-second window. The useful step is to check the examples rather than assume the labels make them clean.

Noise is the most open-ended class. It can contain quiet background, movement, other speech or many different sounds, depending on what was recorded. If the class contains only a narrow set, the model hasn't somehow learnt everything else that can happen in a room. The label is a name we supplied; the actual clips are the evidence it learns from.

This is why adding more data needs a reason. If I add many near-copies of an example already well represented, I might make the count larger without addressing a missing situation. If I collect examples connected to a particular error, I have a clearer question. Does the model still confuse that kind of sound after seeing a better range of it?

The choice of a test also depends on what I want the device to do. Recognising one familiar person's commands in similar conditions is different from recognising many speakers in different surroundings. Neither description should be quietly substituted for the other. I need the intended use clear before deciding how broad a claim the evaluation could support.

Small datasets make individual examples worth looking at. One unusual clip can affect the picture more than I expect from a headline percentage. A confusion matrix can show which classes are being mixed up, but I still have to return to the recordings to understand what those mistakes sound like. The table points me towards questions; it doesn't hear the audio for me.

I'm keeping the dataset and model configuration together in the notes. If another training run looks better later, I want to know whether I changed the examples, the split, the feature processing or the classifier. Otherwise I could attribute an improvement to the wrong part. The version of the data belongs to the experiment too.

This is what I want to discuss with my mentor before collecting clips simply to reach a larger number. Which conditions are missing? What should remain held out? What result would tell us something about the way the device is meant to be used? Those are better starting questions than asking how many recordings a model needs in general.

The training page also fixes the context around its result: 50 cycles, learning rate 0.005 and batch size 32. Those values do not describe how diverse the speakers or recording conditions are, but they do distinguish one training configuration from another. The model's 16,000-sample input window gives a second useful connection: a one-second clip at 16 kHz supplies the duration the application expects. Counts, timing and evaluation roles can be concrete even while broader real-world coverage remains a separate question.

I'm still interested in the model itself, but the sample list has become less boring now that I see what decisions it contains. One minute and forty seconds is not just a small amount of audio. It's the set of examples behind every pattern I’m asking the device to recognise. I want to know that set much better before being impressed by a score.

Leaving this here for next time.All the notes ↗