NeuroSky and Unity

April 25, 2008 in Community

One of the coolest things about working at a tool company is the chance to see the interesting, innovative and exciting things people do with that tool. This week I got to take advantage of that benefit by visiting the San Jose office of NeuroSky. These folks have developed a group of products based around bio-sensors that pick up on your brain waves and then translate those for use as input in a variety of applications. The idea of “thought based input” appeals to a variety of industries and use cases, whether for medical analysis, development of assistive devices or of course use in next generation game content.

The story of how I found out about these people covers a rather indirect path. I first heard of their technology through a friend and Unity user Barry while he was working with SFSU’s Institute for the Next Generation Internet, but I didn’t actually see it myself until GDC 08 in San Francisco. I was cruising the expo floor and passed by a booth with a rather large crowd of people gawking at some demo on screen, and that demo was clearly (to me) using what appeared to be Unity’s default GUI skin. Upon closer inspection and asking a few questions it turns out that they were (and still are) using Unity to help showcase their technology.

So just what is it that they do or make? It’s a headset that you wear, it has a few sensors that lightly touch your skin and it plugs in to your computer as an input device. They then load their demo, created in Unity, and you move around the world using WASD/Arrow Keys. There are various objects in the demo world, you walk up to them and can move (push/pull/lift) or burn (set fire/explode) them by simply focusing your concentration – no keyboard, no mouse, just thoughts! It took a bit of getting used to as you have to either concentrate (focused attention, push/pull or burn) or clear your mind and relax (lift). Of course while you’re doing that your brain is thinking of a million other things distracting you, but once you get the hang of things it gets really cool.

NeuroSky videos on YouTube.com

In the videos found on YouTube you’ll notice that not only are they using brain activity readings but their also using partner technologies to track head and eye movement! They’re actually able to track eye movement and use that movement, including blinking, as input and control mechanisms! I’ve seen various reports of this sort of technology but this was definitely the first time I’ve seen it in person and I was suitably impressed with what they had to show.

You might be wondering about why they’re using Unity, I was too at first. Given that their core business focus is on an input technology and not application or game engine development, they needed a tool that would allow them to quickly test and showcase their content through actual demos, and Unity is the ticket. As proof of that they first adopted Unity as a demo development tool last November and they were able to write their own custom plugin and develop their Unity demo in time for GDC in February! That’s three months to learn the tool, code up a custom plugin to accept their device input and to develop demo content to showcase its abilities. Another clear case of how Unity’s ease of use, intuitive UI and powerful development abilities are meeting the needs of many, including an increasing number of people not directly involved in games development themselves. Cool.

I’d like to thank Johhny Liu from NeuroSky for playing a bit of phone and email tag with me in the weeks since GDC and for helping arrange the meeting. Of course thanks go out to everyone at NeuroSky for helping develop some cool technology and for using Unity as part of your demo and showcase toolset!

Comments (11)

Subscribe to comments
  1. Jake Martin

    October 10, 2011 at 4:08 pm / 

    So i am a student and i am working on a project to use the NeuroSky Mindset in Unity, can somebody give me a link to a small template? Thank you

  2. Mark From Biofeedback Pain

    May 14, 2011 at 7:21 pm / 

    I’m primarily interested in biofeedback machines and the way they can be used for rehabilitation and healing, but the application of this technology for gaming is also quite appealing! What really excites me is that EEG technology is becoming cheaper and cheaper. I really want this tech to hit the mainstream crowd, and I think gaming could be the channel to get ball rolling. What we need is noninvasive sensor gear and some great software titles.

  3. Tom Higgins

    May 4, 2010 at 6:58 pm / 

    @B: unfortunately no, we cannot really assist you with 3rd party tech directly. I would suggest that you make a post on the forums and see what help turns up (ping me there if necessary by sending me a PM, user name is HiggyB).

  4. B

    May 1, 2010 at 12:56 pm / 

    Hello,

    I currently have the Neurosky Mindset and Unity Pro however I cannot manage to connect them.

    I’ve read through the documentation provided by Neurosky (but didn’t help me yet):
    http://developer.neurosky.com/docs/lib/exe/fetch.php?media=app_notes:using_thinkgear_with_unity.pdf

    Possible to get some assistance / feedback what to do specifically ?

    Thanks in advance :)

  5. Jeff Murray

    July 17, 2008 at 3:52 am / 

    I would *love* to make games for this. I tried the demo at GDC and it was freaking awesome – much to my surprise, it actually works. My buddy and I came away from the stand just brimming with game ideas. Oh, and seeing that the demo was done in Unity was pretty rockin’ too. It was that GUI skin that gave it away, for sure ;)

  6. Tom Higgins

    July 3, 2008 at 8:28 pm / 

    Yeah, what Aras said. :) You can download our free 30-day trial from the following URL:

    http://unity3d.com/unity/download

  7. Aras Pranckevičius

    July 3, 2008 at 2:11 pm / 

    We do have a 30 day fully functional free trial. That is quite enough to “start working with the software”, I think.

  8. jonhy

    July 3, 2008 at 10:49 am / 

    hi,

    I just saw you website software and it looks very good. Im not a genious, but why dont you create a Free Version so people could start working with your software, and create licenses for comercial use ou something like thar ?

    Thanks.

  9. Tom Higgins

    May 8, 2008 at 5:55 pm / 

    Steve: it will be interesting to see how Wii Fit (that pressure-pad device) shapes up, interesting moves out of Nintendo for sure. And yes, Doonesbury is easily one of my all-time favorite cartoons (right there with Bloom County). Nice reference!

    Christopher: sure, that’s essentially the same product as NeuroSky is offering, although that one looks like a monstrosity compared to NeuroSky’s (which is only a bit larger than a normal set of headphones). But I sense that both companies will be working hard on a smaller device so it’s more appealing to use. But really, all it takes is writing a simple plug-in to accept the device’s input and they’d be all set.

  10. christopher

    May 8, 2008 at 4:29 pm / 

    so what about this device:

    http://kotaku.com/358459/pc-psychic-controller-hits-this-year

    i wonder if this could be unity compatible too…

  11. Steve Weintz

    May 8, 2008 at 3:50 pm / 

    …Mmm, yummy!

    The Sublime: The Army is actively investigating alternative therapies for their huge load of brain-injury and PTSD cases, and both biofeedback and virtual-worlds therapies hold great promise. Psychologists could take advantage of input devices such as NeuroSky’s and Unity’s rapid-prototyping capacity to quickly experiment with treatments. At GDC I was taken by the new Wii pressure-pad input device — the cartoon ski-jumping was fun, but it was the yoga feedbackgame that really intrigued me.

    The Ridiculous: Reminds me of a Doonesbury cartoon from the early ’90′s, during the first giddiness about VR. BD’s wife Boopsie tries out a virtual online store using a head-mounted display, and blinks while looking around the database: “Oops, I think I just bought a forklift.”

Comments are closed.