Play the Dino Game… Via Mind?

You know the feeling when the internet cuts out? Ya, I’ll say it for you: it sucks.

Except seconds later, the Chrome dinosaur game appears. Suddenly, all your anger, hatred, annoyance, you name it, is out the window. For the next couple of minutes, you find yourself distracted, aimlessly tapping the space bar.

By default, the game was designed to take your mind off the annoying situation but after a while, it becomes boring. All you can do is press one key. But what if you could play without pressing any keys?

What if you could play it with your mind?

At this point, I probably lost you, but hang on. Let me explain. Through brain computer interface technology, a Muse headband, and some code, I’ve figured out a way to control the little 🦖 by mind.

Still don’t believe me? Well, in this article, I’ll explain exactly how it works so next time you see the dino game, you can play it with your mind 🧠

Getting Started

Before any of the magic can happen, the two items required include a laptop and a Muse headband. The Muse headband is a commercially available piece of EEG technology for meditation which has the ability to record brain activity through electrodes or little pieces of conductive metal. If you want to understand the science behind the magic, check out this article here.

Muse 2 Headband which I used

With a computer and code, data collected from the headband can be retrieved and used. Code can detect variance in brain activity which is holds the power to control the dino game.

Since the Muse headband’s main purpose is for meditation and not for programming, a small setup process is required to connect it to the computer. Luckily, I’ve put together a step by step guide here outlining how to connect the headband to the computer.

Feel free to check out my guide or follow the steps below which is directly taken from my guide. Everything was optimized for macOS so if you use another operating system, you may have to slightly adjust the steps.

Notice

If you’ve already followed my previous guide on detecting blinks with the Muse headband, congrats! You can skip the next two sessions as you have already completed them. Pick up reading from the Playing the Dino Game session!

Setting Up The Code

Before we touch the Muse headband, we must get the code part of things sorted out. Follow the 5 steps below to install all the coding dependencies for Python and this project.

Step 1

Since all the code is in Python, the first step is to download Python if you haven’t yet. Visit their website here and download the latest version.

Step 2

After installing Python, you want to open the Terminal on your laptop. You can do this by finding the Terminal app or pressing (⌘ + space), then typing in the word “Terminal” for the app to pop up. If you have trouble opening Terminal, check this out.

Terminal is a text-based interface that allows a user to interact with a computer by entering commands as text. It is commonly used to execute commands, run programs, and perform various tasks on a computer.

This is what Terminal should look like once its open

Step 3

Now that Terminal is up and going, you want to verify your python installation. To do so, simply copy and paste the code below into your terminal and press “enter”

python --version

If that gave you an error along the lines of “command not found: python”, you may have the newer version of python so instead, enter the following.

python3 --version

Either one of the commands above should have printed the version number of your Python software, if not, you may want to return back to step 1 and try it again.

Step 4

At this point, your Python should be up and running. Before we find a code editor, we must check if the Python dependency pip is installed. Pip is a Python package that allows the user to install more commands from the Python Package Index. With pip in place, you can install tools to use later on with just one line.

To check if pip is installed, enter one of the two commands below. If you had to add “3” after python in step 3, you will want to add “3” after pip in this step.

pip --version

or:

pip3 --version

Step 5

Once you’ve confirmed that pip is installed, now is the time to find a code editor where you can edit your code. I recommend Visual Studio Code or VS code developed by Microsoft. The great thing about VS code is the fact it’s easy to use and available on Windows, Linux, and macOS.

Installation is also super easy. Click here and download it from their official website following their instructions.

Once installed, open it up and leave it on the opening page.

At this point, you’ve successfully installed all the code side of things! 🥳 However, things aren’t done yet. The next set of instructions will show you how to connect your headband to the code but I promise, it’ll be more hands and fun!

Connecting the Muse Headband

Now that you’ve set up the computer side of things, we can finally connect the Muse headband! Prior to 2016, Muse designed a platform specifically for this purpose but it has since been disabled. 😢

However, after a tedious trial and error process, I have found a workaround. Follow the steps below to install and connect to your Muse.

Step 1

First, in order for the Muse headband to speak with the laptop, an outlet must be created. The outlet basically allows the Muse bluetooth signals to be “plugged in” or connected to the laptop.

There are several apps that help create this outlet. My personal favourite is Petal Metrics which can be installed here following the website instructions. It’s super user friendly with a great user interface. Petal Metrics works on Windows, Linux, and macOS.

If Petal Metrics doesn’t work for whatever reason, BlueMuse is also an option for Windows users and Windows users only. However, the setup is slightly more complicated requiring Github instead of a simple download.

Step 2

Now that an outlet is created, you want to connect your Muse headband to it. Under type, select LSL (lab streaming layer). This is done in the image below. LSL signals are the best for the type of development we’re doing because it provides the best time-synchronization.

To connect your Muse, simply power it on, turn on your laptop’s bluetooth, and click the big green stream button.

It will say “connecting” or “streaming Muse ####” if you scroll but you won’t see anything else. Don’t worry, you’ve done everything right.

Step 3

Now you’re officially streaming your headband! Go you! At this step, you’re about halfway done the entire project. To actually view the data, there has to be code which turns the signals into a visual. Thankfully, Alexandre Barachant created a wonderful program called Muselsl which achieves this. All we have to do is install Muselsl to both view and code with the Muse data.

Muselsl is a Github file, meaning it is more complicated to download but I will show you how in just a second.

First, you want to open up the Muselsl Github repository. You can do so here. A GitHub repository is a fancy way of a folder for a coding project. Then, you want to clone the repository. On the page, click the big green “code” button and then copy the link you see.

With the link copied, let’s move on to the next step.

Step 4

At this point, we have to head back to VS code we installed at the beginning. Open VS code if you haven’t already and click the first button on the left menu bar. (Refer to image below)

Now, click the “Clone Repository” blue button. A search bar thing should pop up at the top. Paste the link you copied from step 3 into the search bar. Open the repository according to the instructions and trust the author if prompted.

Step 5

You will see a bunch of files on the left but for now, leave them and open the VS code terminal as seen in the screenshot below.

Once open, the terminal should look something like this at the bottom:

Step 6

With the terminal open, you want to use pip to install muselsl. As a refresher, pip was the command installed at the very beginning to install other python dependencies.

Installing muselsl is super easy. Paste the following code into the terminal and press “enter”. Once again, the code you paste depends on which version (3 or without the 3) of python your laptop recognizes.

pip install muselsl

or:

pip3 install muselsl

Once the command is in, let it load and work it’s magic ✨

One error I ran into was the fact that my computer didn’t recognize pygatt. If you run into the same error, simply enter the following code and then try to reinstall muselsl again following the code above.

pip install pygatt

or:

pip3 install pygatt

Step 7

Finally! It’s time to view and stream the data. First, open up Petal Metrics from earlier and start streaming.

Then, go to the terminal in VS code and type in the command below

muselsl view

Your code will find the Petal Metrics outlet and stream the data if you’re correctly wearing your headband. It should look something like this:

Using the commands below, you can alter the way the graph looks for a clearer representation.

Yay! You finally established a connection and get to view your brainwaves! The next step is to modify the code and get it to recognize your blinks 👁️

Playing the Dino Game

Now here is what you’ve been waiting for: playing the actual dino game!

If you skipped the previous two sections, this is the section you should be reading.

First, congrats on setting up the previous two steps and getting everything to run properly — it is far from an easy task but going forward, things are going to be much simpler.

Through the previous steps, we’ve been able to view brain activity. If you paid close enough attention, you might have noticed each time you blink, the graph shows a spike or dip.

One thing we can do with this “dip” is to get the computer to recognize it. When the computer recognizes it, blinking can turn into a signal for the computer to press space/have the dino jump!

The steps below will show you how to get your computer to recognize your blinks.

Step 1

In the Muselsl GitHub repository you cloned earlier, click
examples > neurofeedback.py

neurofeedback.py is the file where the signals are gathered and printed.

After finding the file, you may be prompted to install Python for VS Code. If not, go to the left menu bar, click extensions, search Python, and install Python. The following image explains how to install Python in VS Code.

Step 2

With Python installed, go back to the neurofeedback.py file and click the run button (the little triangle) in the top right. Clicking run should bring up the terminal and there may be lots of errors showing in the terminal. Don’t worry, those errors are supposed to be there.

The errors are a cause of the imports at the top:

Your computer does not recognize those commands so type the following into terminal, one at a time followed by “enter” to install each of those unrecognized files. Once again, depending on the type of python you have, type “pip3” or “pip” to install.

pip3 install numpy

or

pip install numpy

pip3 install matplotlib

or

pip install matplotlib

pip3 install utils

or

pip install utils

pip3 install sklearn

or

pip install sklearn

At this point, you should be able to run your code with no problem. Now let’s move on to the next step.

Step 3

Your code is now running but it’s missing an EEG stream. To find that stream, turn on Petal Metrics from before and connect your Muse. Now, if you run the code, you should see a lot of numbers. These numbers represent the Alpha waves.

You can learn more about these waves in my previous article.

In short, alpha waves are your relaxed state waves. They are largely unaffected by muscle movement like blinks. To make the numbers sensitive to blinks, we must print the delta waves.

To detect delta waves, add the following code

deltaWaves = band_powers[Band.Delta]
print("Delta: {}".format(deltaWaves)) 

Place it where it’s placed in the screenshot.

Basically, the code creates a variable called deltaWaves and sets it to whatever value band_powers[Band.Delta] is. By creating a variable, it becomes much easier to refer to it later on.

If you completed all the instructions above correctly, hit run again and you should be seeing the delta waves.

Step 4

Delta waves can easily detect blinks and muscle movements. If you take a close look at the numbers or the screenshot above, you’ll see that the second you “blink”, the numbers become larger than 1.

Now we know greater than 1 = blink so what can we do with this info?

Well, in code, if we want something to happen given a condition, we use an if statement. In the case of blink detection, the computer should print “blink” if the delta wave is greater than 1. This can be done through the following lines:

if deltaWaves >= 1: 
      print("Blink!")        

The line should be placed below the line where you print the delta waves.

Upon running the code, you may notice that it only slightly works. This is because each time you blink, there is enough muscle movement to make the computer think you blinked several times. Instead of recognizing one blink, the computer will recognize several and in the case of the dino game, the dino will jump more than once.

How do we fix this? By making the code more precise.

Step 5

Let’s fine tune our code so the computer only recognizes one blink as ONE blink. This part can be a little bit tricky, especially if you don’t have any prior coding language but I’m going to do my best to keep it simple.

First, we’re going to make a list at the top of the code called allValues and store the variable 0 inside. Copy the code below.

allValues = [0]

Place it before the try and while loop as indicated by the screenshot below.

The list allValue is going to serve as a list where we add in all the delta values. Late on, we will use the stored values to help the computer determine whether there was one blink or if the detected blink is muscle movement from the same blink.

Step 6

After setting up the list at the start, it is time to actually make use of the list. Return back to the place with the if statement and right below the place where we created a variable called deltaWaves, add the following line:

allValues.append(deltaWaves)

What this does is add the most recent delta wave number to the list called allValues we created earlier. This is required so we can compare numbers in the next step.

Step 7

With the delta wave added into the allValues list, we can adjust our if statement to increase its accuracy. Replace the previous if statement you had with the following code.

if deltaWaves >= 1 and allValues[-2] <= 1: 
    print("Blink!")            

This code basically tells the computer that if the delta wave is greater than one and if the previous delta wave recorded to the list allValues is smaller than one, there is a blink.

The boxed value is the actual blink, everything else is the remaining muscle movement

Why does this work? Well because we only want the computer to detect the first delta wave greater than one and consider it a blink. All the numbers greater than one following will be the remains of the first blink. allValues[-2] in the code tells the computer to look back at the last value of the allValues list which is the last value of the delta wave recorded. If it was smaller than one, that means the current value is not remaining muscle movement and actually a blink.

Step 8

One step closer to success! We’ve fine tuned the code to detect just one blink but we’re not completely done yet…

Our code won’t detect blinks that happen right after another. Since our code only detects blink when the number before is smaller than one, if we quickly blink after our first blink, the computer will treat it like lingering muscle movement from the first blink. Luckily, a few more lines of code will fix this.

elif deltaWaves >= 1.7 and allValues[-2] >= 1:
    print("Blink!")          

Place this code right under the previous if statement.

In the lines above, the computer will detect a blink if the delta wave is greater than 1.7 and if the previous delta wave is greater than 1. How does this work? Well after one blink, the delta waves going to be above one but after testing, I realized they never exceed 1.7. However, if you blink right after the first blink, the number will exceed 1.7.

Using this information, when the delta wave exceeds 1.7 and the previous delta wave is greater than 1, we know that there was a previous blink but the current delta wave is another blink.

Step 9

We are all done with the blink detection part! Now we can actually make the dino jump and I promise, this is easy.

Go back to the top of your code, the very top where a lot of random commands are imported. This time, we are going to import something called pyautogui.

Pyautogui allows the code to control parts of the computer like the keyboard movement and mouse. Since the dino game requires the pressing of the space button, we will use pyautogui to control the space bar.

To import pyautogui, simply add the following line:

import pyautogui

When you run the code, you will get an error but it is only because you didn’t install pyautogui. To fix it, all you have to do is install pyautogui with pip. Simply try one of the following commands in your terminal.

pip3 install pyautogui

or

pip install pyautogui

Step 10

Pyautogui officially installed!

Since our goal is to make the dino jump once we blink, we can acheive this by calling pyautogui every time a blink is detected.

To do so, you want to go back to your if/elif statements, and add the line right after you print “Blink!”

pyautogui.press('space')

Your final code should look a little something like this. Every time a blink is detected, the word “Blink!” will be printed and simultaneously, pyautogui will press your space key for you!

Step 11: Last step!!

Congrats on making it so far. It really takes effort to follow the endless steps above.

We can finally play the game! Open up a chrome tab and turn off the internet to reveal the dino game or simply visit this site.

Run your code, connect your Muse, and go to your game. Start blinking and admire the wonders!

Conclusion

All set! You can actually play the dino game with your mind! How crazy is that 🤯

Check out this video of me playing the dino game with my mind!

If you have any problems, feel free to reach out to me — all my contact methods can be found here, and once again, thanks for reading!

GitHub repo: https://github.com/carolrongg/Muse-Blink-Detection.git

References

• ChatGPT for saving me when I ran into terminal issues.
• Controlling A Video Game With My Mind
• Muse 101 — How to start Developing with the Muse 2 right now
• 6 simple ways to open Terminal on Mac
• Petal
• Muselsl
• Detecting blinks with Muse