Serial communication with Arduino + P5.js

So far I have been developing projects for Computational Media and Physical Computing separately. What I created in P5.js was developed and stayed in the browser, in the computer screen. Likewise, the inputs and outputs happening in my Arduino projects such as the LEDs, sensors and buttons in the breadboard, were physically interacting only outside any screen. But what if I want to make a physical button that triggers something in my P5.js sketch? How do I make it work?

For two devices to communicate, whether they are desktop computers, microcontrollers, or any other form of computer, we need a method of communication and an agreed-upon language. Serial communication, in this case, is one of the most common forms of communication between computers. Thus, this is the method I will be using to make a potentiometer and a button interact with the following sketch.

Check the code here.

As you can check this sketch is very similar to my previous Computational Media post. Indeed I used its code as a base. I replaced the images for a png cut taken out from a photo of my eye. I also took a photo of my eye closed and added an interaction so when the mouse is clicked the crazy random eyes can blink at you ;).
You can check also this version that where I replaced the images to pngs of my mouth so when the mouse is clicked the sketch sends you a kiss :*.

Check the code here.

Anyways, as you noticed I played enough with the p5.js sketch so now it’s time to integrate it with my Arduino code. First I established that I wanted to use, instead of the slider, a potentiometer to rotate the image. Second instead of the mouse Click for the blinking, I would replace this interaction for a physical button.

As a total beginner in serial communication, I followed this tutorial and watched the those videos of Professors Tom Igoe and Jeff Feddersen.

With that in mind, I created my circuit.

I added the code to my Arduino, keeping in mind that the way the data was sent was how my p5.js serial code would read, understand it, and then be able to make it interact properly with my sketch. Therefore, since I was using two inputs (the potentiometer and the button), I needed to make my Arduino code send it to my p5.js in a way it would get this data and understand it as two different variables. The way I did it was by parsing a “,” (comma) between those different numbers. You can check my Arduino code below.

Then, also following the tutorial linked before, i downloaded the p5.serialcontrol port and added the required information to make the effective serial communication with my Arduino circuit and my code.

And it worked!!!

Check the Serial P5.js code here.


Singing Hula Hoop #1 Prototype

In my Physical Computing class, we were starting to work with analog input data. Therefore, we were assigned to create a musical instrument that played with these different inputs and generated sound.

I am really excited with the idea of integrating dancing with physical computing as dance has always been a great way to express myself and let go. I think that further on it opens possibilities to develop art performances and mainly to create toys and learning tools for kids. But for now, the main idea was to prototype and have fun trying to make a hula hoop more interactive, transforming it into a musical instrument as proposed in the assignment.

Trial #1 —> Fail

Having in mind that my goal was to collect different inputs from the hooping (spinning) movement, and trying to avoid attaching many things to the hoop, my first idea was to work with infrared light. This would work in the following way: I would attach a battery and a series of infrared LED’s in the hoop, while the Arduino, with an infrared light sensor and a speaker, would be placed in a table and connected to a power source. Every time the distance from the infrared LED’s in the hoop and the infrared sensor in the table would vary, I would assign a specific note for it to play. This was a very practical and nice idea in theory, but it didn’t work in real life. In the following video you can check out why.

As you can see in this prototype version in the bread board, I was simulating the hoop movement to try to understand the variables I would be able to get from this sensor. Probably you noticed that actually the opposite from what I expected happened: the sensor data numbers, instead of increasing as I brought the LED’s light closer, decreased to a constant “0” (zero).  With this input I was not gonna be able to generate a proper sound and integrate as I wanted. Besides, I needed to bring the infrared LED’s super close to my Arduino and this wasn’t functional to my idea.

Trial #2 —> Success 

So I had to change my approach. I decided to work with an accelerometer. As I didn’t have the time to learn how to work to send bluetooth sensor data to the Arduino I had to embrace doing a very prototype-ish version of my idea and have the speaker and the Arduino placed in the Hula Hoop. My solution to minimize that was to get a smaller Arduino, so I ended up getting an Arduino MKR 1000. I also purchased the Adafruit_BNO055 accelerometer sensor.

To be able to work with  Adafruit_BNO055 accelerometer I had to download its documentation library. I only used its basic functions that enable be to get X,Y,Z variables. I chose to proceed with the X variable for now, as it sensed the 360 degrees movement.

So i had the sensor working (check!) and I identified the variables (check!). Now it was time to use the pitch.h library to attach the different variable inputs to notes (through if/else statements). You can check the code below:

#include <utility/imumaths.h>
#include "pitches.h"

Adafruit_BNO055 bno = Adafruit_BNO055(55);

int analogInput;

void setup(void)
Serial.println("Orientation Sensor Test"); Serial.println("");

/* Initialise the sensor */
if (!bno.begin())
/* There was a problem detecting the BNO055 ... check your connections */
Serial.print("Ooops, no BNO055 detected ... Check your wiring or I2C ADDR!");
while (1);




void loop(void) {
/* Get a new sensor event */
sensors_event_t event;

/* Display the floating point data */
Serial.print("X: ");
Serial.print(event.orientation.x, 4);
Serial.print("\tY: ");
Serial.print(event.orientation.y, 4);
Serial.print("\tZ: ");
Serial.print(event.orientation.z, 4);


int frequency = map(event.orientation.x, 0, 360, 100, 880);

if (event.orientation.x > 2 && event.orientation.x <= 5) { tone(8, NOTE_C4); } else if (event.orientation.x > 5 && event.orientation.x <= 10) { tone(8, NOTE_F6); } else if (event.orientation.x > 10 && event.orientation.x <= 20) { tone(8, NOTE_G3); } else if (event.orientation.x > 20 && event.orientation.x <= 30) { tone(8, NOTE_A3); } else if (event.orientation.x > 30 && event.orientation.x <= 40) { tone(8, NOTE_B3); } else if (event.orientation.x > 40 && event.orientation.x <= 50) { tone(8, NOTE_C4); } else if (event.orientation.x > 50 && event.orientation.x <= 60) { tone(8, NOTE_G3); } else if (event.orientation.x > 60 && event.orientation.x <= 70) { tone(8, NOTE_A3); } else if (event.orientation.x > 70 && event.orientation.x <= 80) { tone(8, NOTE_B3); } else if (event.orientation.x > 80 && event.orientation.x <= 90) { tone(8, NOTE_C4); } else if (event.orientation.x > 90 && event.orientation.x <= 100) { tone(8, NOTE_G3); } else if (event.orientation.x > 100 && event.orientation.x <= 110) { tone(8, NOTE_A3); } else if (event.orientation.x > 110 && event.orientation.x <= 120) { tone(8, NOTE_B3); } else if (event.orientation.x > 120 && event.orientation.x <= 130) { tone(8, NOTE_C4); } else if (event.orientation.x > 130 && event.orientation.x <= 140) { tone(8, NOTE_G3); } else if (event.orientation.x > 140 && event.orientation.x <= 150) { tone(8, NOTE_A3); } else if (event.orientation.x > 150 && event.orientation.x <= 160) { tone(8, NOTE_B3); } else if (event.orientation.x > 160 && event.orientation.x <= 170) { tone(8, NOTE_G3); } else if (event.orientation.x > 170 && event.orientation.x <= 180) { tone(8, NOTE_A3); } else if (event.orientation.x > 180 && event.orientation.x <= 190) { tone(8, NOTE_B3); } else if (event.orientation.x > 190 && event.orientation.x <= 200) { tone(8, NOTE_C4); } else if (event.orientation.x > 200 && event.orientation.x <= 210) { tone(8, NOTE_G3); } else if (event.orientation.x > 210 && event.orientation.x <= 220) { tone(8, NOTE_A3); } else if (event.orientation.x > 220 && event.orientation.x <= 230) { tone(8, NOTE_B3); } else if (event.orientation.x > 230 && event.orientation.x <= 240) { tone(8, NOTE_C4); } else if (event.orientation.x > 240 && event.orientation.x <= 250) { tone(8, NOTE_G3); } else if (event.orientation.x > 250 && event.orientation.x <= 260) { tone(8, NOTE_A3); } else if (event.orientation.x > 260 && event.orientation.x <= 270) { tone(8, NOTE_B3); } else if (event.orientation.x > 270 && event.orientation.x <= 280) { tone(8, NOTE_C4); } else if (event.orientation.x > 280 && event.orientation.x <= 290) { tone(8, NOTE_G3); } else if (event.orientation.x > 290 && event.orientation.x <= 300) { tone(8, NOTE_A3); } else if (event.orientation.x > 300 && event.orientation.x <= 310) { tone(8, NOTE_B3); } else if (event.orientation.x > 310 && event.orientation.x <= 320) { tone(8, NOTE_G3); } else if (event.orientation.x > 320 && event.orientation.x <= 330) { tone(8, NOTE_A3); } else if (event.orientation.x > 330 && event.orientation.x <= 340) { tone(8, NOTE_B3); } else if (event.orientation.x > 340 && event.orientation.x <= 345) { tone(8, NOTE_A5); } else if (event.orientation.x > 345 && event.orientation.x <= 350) { tone(8, NOTE_C4); } else if (event.orientation.x > 350) {
tone(8, NOTE_G3);


With that, the only remaining challenge was to attach the battery, Arduino, accelerometer, and speaker to the Hula Hoop in a way people could still hoop with it. So I imagined it would be better to balance out placing the 9V battery with the on/off button in a side and attach the rest in the other side of the Hula-Hoop. As it was a really prototype-ish version, i attached the Arduino/Accelerometer/Speaker with paper, and soldered it together. I also put the wires throughout the hoop as you can check in the images below.

* as you can see in the last image I added a resistor near the battery to avoid short circuit.

So as I turned on, now my Hulla-Hoop could emit noises with motion.

It is still not the best sound. Next step would be definitely adding a bluetooth integration from the accelerator to the Arduino. With that, I will place it inside the hoop and make it way better designed. Also, I have to work a lot on the code, to enable more control over the tunes when the user is hooping. It was a fun project for a first trial, but in order to be able to create performances I need to take it to the next level.



Trump + KimJong P5.js Collage

Playing with P5.js has been a bit hard for since most of P5.js sketches draw geometrical shapes. With that tou can create beautiful and interesting forms or animations, still, is hard for me to have fun with this type of graphic representation. That’s why I got super excited when we started adding images to our work. Physically, I enjoy making collages, so why not bring it in a coded, optimized version to my P5.js drawings in the browser?

What I like most about collages is getting this ready made, mainstream images of things in society I would like to change, and playing with it. Such as the ideal of female body image, consumerism, politics…or in this case the international agenda. Collage is this little vodoo of mine. And I love the opportunity that collages with p5.js invite the user to play with it too with mouseover, clicks, reloading the page and so on. I loved making this funny sketches, taking simple .png files of Trump’s and Kim Jong’s available in a simple google search, and enabling the users to also interact with their faces.

Check the code here.

To create this sketch I played with loops, objects and arrays. You can notice that every time you reload the browser the way the faces are displayed change, creating different drawings.

I also developed other 2 simpler sketches with this approach.

  1. Create little worms from trump and kimjong’s faces by clicking in the canvas. Check the code here.
  2. Just see trump and kimjong faces appear randomly in the canvas. Check the code here.

These p5.js collage sketches were developed based on watching this Objects and Images video from my teacher’s  – Daniel Shiffman – youtube channel.

Hope you have fun,