Getting Started with Edge Impulse

Read the TinyML by Edge Impulse sectionTinyML by Edge Impulse

Imagine wanting to sense when a certain gesture occurs on a microcontroller. Normally, this would be accomplished by gathering training data from a sensor, labeling it, and then using a script to train a model and then deploy it. This workflow takes a lot of time and can be very daunting to beginners.

TinyML is a service by Edge impulse that allows for the easy creation of these models by simply connecting a microcontroller to their service, clicking a button, and then deploying it after their servers do the heavy lifting. Their drag-and-drop interface makes it a breeze to add more sensors or learning capabilities.

Read the Potential Applications sectionPotential Applications

As alluded to before, Edge Impulse's TinyML is great for quickly creating projects that rely on large amounts of data to detect and/or predict when events will occur or are occurring. A great example of this is hotword recognition, where a device can be set up that listens for a specific phrase and then performs an action based on it, such as "open blinds" or "lights on." Another great use for the platform is motion recognition, such as a way to count steps or check if an item is currently being used.

Read the Setting Up an Account sectionSetting Up an Account

To begin, first create an account by going to https://studio.edgeimpulse.com/signup.

After entering in your information and verifying your email, you will be greeted by a getting started page. This will walk you through the process of connecting a device, gathering data, and finally deploying a model. I named my first project PhoneTest-1, but it can be anything you like.

Read the Connecting a Mobile Phone sectionConnecting a Mobile Phone

TinyML supports many devices, including the ESP32, many ST ARM Cortex-M3 boards, and several Arduino WiFi-enabled kits. However, many of the same tasks can be accomplished simply by using a smartphone via a web browser, as it contains a microphone and accelerometer. To connect your phone, simply click on the "Use your mobile phone" button which opens up a QR code.

After scanning it, you will be taken to their website and automatically connected to their API via an API key. Make sure to keep your phone on and the browser window open throughout the rest of the guide.

Read the Gathering Data sectionGathering Data

Now it's time to really dive in and create a model. But first, there has to be data to train it on. Make sure you have your phone handy because you'll be using its sensors to capture the data. To begin, go to the data acquisition tab and make sure your phone is selected. Choose the accelerometer sensor and frequency and then click "Start sampling." After you are done moving your phone, you can view the collected data in a graph.

Read the Training a Model sectionTraining a Model

Since you have some data recorded, it's time to train a model from it. Go ahead and navigate to the "Create Impulse" page and select the recommended Spectral Analysis processing block and a Keras Neural Network learning block. Then go ahead and save the impulse.

Next, set up your data's scaling, filter, and FFT settings. These will control how your data gets pre-processed before being sent into the NN. After doing that, view and generate the features.

On the NN settings page, I decided to change the default confidence threshold from 80% up to 91%. After training the model, I was able to view a graph of what the model came up with. Then I went to the "Classification" page and gathered a bit more data from my phone and saw what the model was able to detect.

Read the Deploying sectionDeploying

To deploy the model, I exported the model as a WebASM file and then unzipped it. Then I created a new js file called run-impulse.js and placed it into the same folder as the model (but the file is attached to this project page). To run it, I entered the node command into the command prompt followed by run-impulse.js and then pasted the "Raw features" array in quotes as the second argument for the node command.

Read the Code sectionCode

// Load the inferencing WebAssembly module
const Module = require('./edge-impulse-standalone');

// Classifier module
let classifierInitialized = false;
Module.onRuntimeInitialized = function() {
	classifierInitialized = true;
};

class EdgeImpulseClassifier {
	//_initialized = false;

	init() {
		if (classifierInitialized === true) return Promise.resolve();
		return new Promise((resolve) => {
			Module.onRuntimeInitialized = () => {
				resolve();
				classifierInitialized = true;
			};
		});
	}
	
	classify(rawData, debug = false) {
		if (!classifierInitialized) throw new Error('Module is not initialized');
		
		const obj = this._arrayToHeap(rawData);
		let ret = Module.run_classifier(obj.buffer.byteOffset, rawData.length, debug);
		Module._free(obj.ptr);
		if (ret.result !== 0) {
			throw new Error('Classification failed (err code: ' + ret.result + ')');
		}
		
		let jsResult = {
			anomaly: ret.anomaly,
			results: []
		};
		for (let cx = 0; cx < ret.classification.size(); cx++) {
			let c = ret.classification.get(cx);
			jsResult.results.push({ label: c.label, value: c.value });
		}
		
		return jsResult;
	}
	
	_arrayToHeap(data) {
		let typedArray = new Float32Array(data);
		let numBytes = typedArray.length * typedArray.BYTES_PER_ELEMENT;
		let ptr = Module._malloc(numBytes);
		let heapBytes = new Uint8Array(Module.HEAPU8.buffer, ptr, numBytes);
		heapBytes.set(new Uint8Array(typedArray.buffer));
		
		return { ptr: ptr, buffer: heapBytes };
	}
}

if (!process.argv[2]) {
	return console.error('Requires one parameter (a comma-separated list of raw features)');
}

// Initialize the classifier, and invoke with the argument passed in
let classifier = new EdgeImpulseClassifier();
classifier
	.init()
	.then(() => {
		let result = classifier.classify(process.argv[2].trim().split(',').map(n => Number(n)));
		console.log(result);
	})
	.catch(err => {
		console.error('Failed to initialize classifier', err);
	});