During my Hello Computer class, I developed a bunch of experiments such as the emoji story creator, invisible map and the dada poem reciter. Those were more fun and simple weekly assignments projects, where I learned a lot and helped me explore the use of voice interfaces.

Therefore, for my final for this class I decided to create a bot using dialogflow in order to create a functional voice interface tool to help people complete a specific task. My idea was to user test and also think about the UX behind voice interfaces as well as develop a platform that could eventually be used in parallel (or even more used, who knows!?) than the webpage designed to fulfill the same functionality.

So what do I – really – need help with?

My main inspiration for this project was the current Google Assistant and that amazing – but also scary – video released in this years Google I/O where the digital assistant calls a hairdresser to book a hair saloon appointment for you.

As most impressive as that is, still,  for my daily tasks the best use that i make of my Google Home/Assistant is mostly to ask for the temperature, to play a song, and eventually to help decide what should I wear according to the weather. Which is nice, but I would imagine that this tool should have way more potential to be functional in my life. So what is something that I make often and would like to get done by voice command in at least a weekly basis?

Differently from the Persona in the Google’s I/O example, that wants have her haircut done and need to book hours for it, as a student something that I schedule often is Office Hours with Residents and Professors. And ITP has a website for that.

 

 

The website collects all links from the residents/professors calendars and display their Bookable Events, which you can click and book as Office Hours. Despite some professors that do a double check with an internal NYU account, as long as you have any google @gmail account you are enabled to do that.

The website does its job, but  it can be annoying as you have to go back to check if the person with the skill you want is available for the day you are also available. If the person is not, you have to go to the home and to the steps again to find another resident or teacher that could help you… and so on.

So, what if I could create a voice assistant that could book ITP office hours for you?

Would I succeed on developing it? If I did, would it indeed be useful? Would people be willing to switch from using only the website and find a voice interface actually more helpful?

 

Setting up DialogFlow and Firebase for a first prototype

I followed Nicole’s instructions in class and set up a simple working prototype from the DialogFlow/Firebase side for creating the ITP Assistant.

My idea for now was to make the simplest UX possible that could fulfill the task of booking office hours, even if that woudn’t be good enough for the final interface, but could work as a proof of concept and then grow from there. This is the interface I aimed to create:

So that part was working,  as you can check in the video below.

My code at this point was very simple.

I added a JSON file with the names of the residents (office hours with teachers too would have to wait a bit!), with their skills, and, once i had received the string inside my skills Intent, I would parse it and try to match it with the existing skills on the JSON. If it found a matching skill I would push the Resident’s name into an array, than then would be randomized, and one of the matching resident’s selected. Once selected, while the DialogFlow would be saying to ‘hold for a sec while I book office hours”, the code would run a function to  get that Resident’s URL and actually book the office hours. – that then would be said back by the Dialogflow with the date and time scheduled. You can check the code here.

So that’s when my problems begun…

Finding out that Google’s Calendsr API couldn’t work for me

(I’ll finish writing by tomorrow Oct 26th)

Working with puppeteer (fun!)  and Booking office hours through my server

(I’ll finish writing by tomorrow Oct 26th)

Trying to make firebase and pupeteer work together (not successful)

(I’ll finish writing by tomorrow Oct 26th)

Next Steps

(I’ll finish writing by tomorrow Oct 26th)

Share this Post

Leave a Reply

Your email address will not be published. Required fields are marked *