Alexa on Rails – how to develop and test Alexa skills using Rails


Introduction

Alexa is awesome and I think that conversational software is the future. This post documents what I set myself as a technical learning challenge:

  • Host the skill locally, to allow a fast development feedback cycle prior to pushing code.
  • To find a way to automated tests (unit, functional and end-to-end), as most demos refer to manual testing.
  • To use something other than JS (like most of the demos do)
  • To write an Alexa skill that’s backed by a data store
  • To be able to handle conversations.

The way Alexa services interact with apps is the following:

User->Echo: “Alexa, …”
Note right of Echo: Wakes on ‘Alexa’
Echo->Amazon: Streams data spoken
Amazon->Rails: OfficeIntent
Rails->SkillsController: POST
SkillsController->Amazon: reply (text)
Amazon->Echo: reply (voice)
Echo->User: Speaks

The skill

The skill is a data retrieval one, giving information about the company’s offices and the workers there.

Alexa, Rails, git, ngrok and an Amazon account

I bought a dot and set up an Amazon account to register the skill on.

Install Rails and git for your OS. You’ll also need a data-store, easily using sqlite, or mysql gems.

ngrok is a nifty tool that will tunnel Alexa calls in to our local server.

Get the code

Fork or clone the repo for a head-start, or read along taking only pieces you need from this post.

Set up the app

  • Setting some environment variables

The database connection use the following environment variables:

export ALEXA_DB_USERNAME=
export ALEXA_DB_PASSWORD=
  • Setting up the database
bundle
rake db:create db:migrate db:seed spec

This will create and setup the database tables, seed the development tables and run the unit and integration tests.

  • Running tests
rake

Will run all tests excluding the audio tests, which I’ll describe below. Make sure all tests pass.

Connecting to the real thing

When a user invokes your skill, Amazon will route requests to an endpoint listed on the Alexa site. In order for this to function, you must first configure the skill there. It’s straightforward, but must be manually uploaded to the skill’s configuration page on Amazon’s site.

Intent schema

This is where you define the intents the user can express to your skill. I think of ‘intents’ as the skill’s ‘methods’, if you think of the skill as an object.

Utterances

Permutations on the intent’s syntax. For example:

Bookit for vacant rooms between {StartDate} and {EndDate}
OfficeWorkers who the {Staff} from {Office} are

Slot types

Here are the slot types for our skill, defining synonyms for our slots, being the parameters for intents. If you think this is complex, please remember that I am only the messenger here…

slots

Now that you have configured the skill’s interfaces, we now need to route communications from Amazon to our local server running Rails as we develop and debug. This is easily done using ngrok, explained below.

ngrok

ngrok is a service, with a free tier, that will redirect traffic from outside your home/office’s firewall into your network. Once configured, it will route traffic from Amazon to our http://localhost:3000, essential for our aspired fast development cycle.

Run it using:

ngrok http -hostname=endpoint.ngrok.io 3000

Your configuration may vary, depending on whether you are paying customer or not, so change ‘endpoint’ accordingly.

You’ll see something like this once you run it:

1495923802.png

Add your endpoint to Amazon’s skill page under configuration:

endpoint

Generating a certificate

Once you’ve settled on the endpoint URL, you’ll need to create or reuse a certificate for Amazon to use when communicating with your server process.

genrsa 2048 > private-key.pem
openssl req -new -key private-key.pem -out csr.pem
openssl req -new -x509 -days 365 -key private-key.pem -config cert.cnf -out certificate.pem

Copy the the contents of ‘certificate.pem’ to the skill’s page on Amazon:

cert

Toggle the test switch to ‘on’, otherwise Amazon will think you’re trying to publish the skill on their Skills store:

testing

Last but not least, enable the skill on your iPhone or Android by launching the Alexa app and verifying that the skill exists in ‘Your skills’ tab.

Amazon recap

We uploaded the skill info, including:

  • The Interaction model, uploading the ‘intent schema’, ‘Custom slot types’, and ‘Sample utterances’.
  • Configured the end-point
  • Uploaded the SSL cert
  • Enabled the test flag
  • Verified that the skill is enabled by using your Alexa app on your mobile device

The moment we’ve been waiting for

Run your rails app:

rails s

Run ngrok in another terminal window:

ngrok http -hostname=alexa01.ngrok.io 3000

Say something to Alexa:

Alexa, tell Buildit to list the offices

If all goes well, you should:

  • See the request being logged in the ngrok terminal (telling you that Amazon connected and passed the request to it)
  • See that the rails controller got the request by looking at the logs
  • Hear the response from your Alexa device

If there was a problem at this stage, please contact me so I can improve the instructions.

Code walkthrough

Route to a single skills controller:

 Rails.application.routes.draw do
   # Amazon comes in with a post request
   post '/' => 'skills#root', :as => :root
 end

Set up that controller:

class SkillsController < ApplicationController
  skip_before_action :verify_authenticity_token

  def root
    case params['request']['type']
      when 'LaunchRequest'
        response = LaunchRequest.new.respond
      when 'IntentRequest'
        response = IntentRequest.new.respond(params['request']['intent'])
     end
     render json: response
  end
end

Handle the requests:

def respond intent_request
  intent_name = intent_request['name']

  Rails.logger.debug { "IntentRequest: #{intent_request.to_json}" }

  case intent_name
    when 'ListOffice'
      speech = prepare_list_office_request
    when 'OfficeWorkers'
      speech = prepare_office_workers_request(intent_request)
    when 'OfficeQuery'
      speech = prepare_office_query_request(intent_request)
    when 'Bookit'
      speech = prepare_bookit_request(intent_request)
    when 'AMAZON.StopIntent'
      speech = 'Peace, out.'
    else
      speech = 'I am going to ignore that.'
  end

  output = AlexaRubykit::Response.new
  output.add_speech(speech)
  output.build_response(true)
end

Test walkthrough

Unit tests

Really fast, not touching any Alexa or controller code, just making sure that the methods create the correct responses:

 

require 'rails_helper'

RSpec.describe 'Office' do
  before :all do
    @intent_request = IntentRequest.new
  end
  describe 'Intents' do
    it 'handles no offices' do
      expect(@intent_request.handle_list_office_request([])).to match /We don't have any offices/
    end

    it 'handles a single office' do
      expect(@intent_request.handle_list_office_request(['NY'])).to match /NY is the only office./
    end

    it 'handles multiple offices' do
      expect(@intent_request.handle_list_office_request(['NY', 'London'])).to match /Our offices are in NY, and last but not least is the office in London./
    end
  end
end

Integration tests

Mocking out Alexa calls, ensure that the JSON coming in and out is correct:

describe 'Intents' do
  describe 'Office IntentRequest' do
    it 'reports no offices' do
      request = JSON.parse(File.read('spec/fixtures/list_offices.json'))
      post :root, params: request, format: :json
      expect(response.body).to match /We don't have any offices/
    end

    it 'reports a single office' do
      request = JSON.parse(File.read('spec/fixtures/list_offices.json'))
      Office.create name:'London'
      post :root, params: request, format: :json
      expect(response.body).to match /London is the only office/
    end

    it 'reports multiple offices' do
      request = JSON.parse(File.read('spec/fixtures/list_offices.json'))
      Office.create [{name: 'London'}, {name: 'Tel Aviv'}]
      post :root, params: request, format: :json
      expect(response.body).to match /Our offices are in London, and last but not least is the office in Tel Aviv./
    end
  end
end

Audio tests

I was keen on finding a way to simulate what would otherwise be an end-to-end user-acceptance test, like a Selenium session for a web-based app.

The audio test I came up with has the following flow:

describe 'audio tests', :audio do
  it 'responds to ListOffice intent' do
    london = 'Paris'
    aviv = 'Tel Aviv'

    Office.create [{ name: london }, { name: aviv }]

    pid = play_audio 'spec/fixtures/list-office.m4a'

    client, data = start_server

    post :root, params: JSON.parse(data), format: :json
    result = (response.body =~ /(?=#{london})(?=.*#{aviv})/) > 0

    reply client, 'The list offices intent test ' + (result ? 'passed' : 'failed')
    expect(result).to be true
  end

end

Line 6: Creates some offices.
Line 8: Plays an audio file that asks Alexa to list the offices
Line 10: Starts an HTTP server listening on port 80\. Make sure that rails is not running, but keep ngrok up to direct traffic to the test.
Line 12: Will direct the intent request from Alexa to the controller
Line 13: Makes sure that both office names are present in the response
Line 15: Replaces the response that would have been sent back to Alexa with a curt message about the test passing or not.
Line 16: Relays the test status back to RSpec for auditing.

This is as close as I got to an end-to-end test (audio and controller). Please let me know if you have other ways of achieving the same!

Conclusion

What was technically done here?

  • We registered an Alexa skill
  • We have a mechanism to direct traffic to our server
  • We have a mechanism to unit-test, integration-test and acceptance-test our skill
  • We have a mechanism that allows for a fast development cycle, running the skill locally till we’re ready to deploy it publicly.

My main learning, however, was not a technical one (despite my thinking that the audio test is nifty!). Being an advocate for TDD and BDD, I realise that now there’s a new way of thinking about intents, whether the app is a voice-enabled one or not.

We may call it CDD, being Conversation Driven Development.

The classic “As a..”, “I want to…”, “So that…” manner of describing intent seems so static compared to imagining a conversation with your product, whether it’s voice-enabled or not. In our case, try to imagine what a conversation with an office application would be like?

“Alexa, walk me through onboarding”. Through booking time, booking conference rooms, asking where office-mates are, what everyone is working on etc.

If the app happens to be a voice-enabled one, just make audio recordings of the prompts, and employ TDD using them. If it’s a classic app, use those conversations to create BDD scripts to help you implement the intents.

 

Arduino programming using Ruby, Cucumber & rSpec


The project

This project serves as a sanity check that all is in order with the hardware, without the need to write on-board code using the IDE nor use the avr toolchain. What better tool than Ruby to do so?

The first thing we’ll do is to assure that the board and its built-in LED are responsive. Let’s define the behviour we would like, and implement it using Cucumber, in true BDD fashion:

Feature:
  Assure board led is responsive

  Background:
    Given the board is connected

  Scenario: Turn led on
    When I issue the led "On" command
    Then the led is "On"

  Scenario: Turn led off
    When I issue the led "Off" command
    Then the led is "Off"

The step implementation follows:

require 'driver'

Given(/^the board is connected$/) do
  @driver ||= Driver.new
end

When(/^I issue the led "([^"]*)" command$/) do |command|
  value = string_to_val command
  expect(@driver.set_led_state value).to be value
end

Then(/^the led is "([^"]*)"$/) do |state|
  expect(@driver.get_led_state).to eq string_to_val state
end

def string_to_val state
  case state.downcase
    when 'on'
      my_state = ON
    when 'off'
      my_state = OFF
  end
end

Some things to note:

  • We don’t have an assertion on @driver ||= Driver.new because the driver will simulate a connection in case the phyical board is disconnected or unavailable due to disrupted communications.
  • The user communicates using the words “on” and “off”, which are translated to ON and OFF for internal use.

This test will fail, of course, as we have yet to define the Driver class and we drop to rSpec, in TDD fashion:

require 'driver'

describe "led functions" do
  before(:each) do
    @driver = Driver.new
  end

  it "turns the led on" do
    expect(@driver.set_led_state ON).to eq ON
  end

  it "turns the led off" do
    expect(@driver.set_led_state OFF).to eq OFF
  end

  it "blinks" do
    @driver.blink 3
  end
end

This too fails, of course, and we implement Driver thus:

class Driver
  def initialize 
    @arduino ||= ArduinoFirmata.connect nil, :bps =&gt; 57600 
  rescue Exception =&gt; ex 
    puts "Simulating. #{ex.message}" if @arduino.nil?
  end 
  def set_led_state state 
    result = @arduino.digital_write(LED_PIN, state)
  rescue Exception =&gt; ex 
    @state = state 
    state 
  end 

  def get_led_state 
    @arduino.output_digital_read(LED_PIN)
  rescue Exception =&gt; ex 
    @state 
  end 

  def blink num 
    (0..num).each do 
      set_led_state ON 
      sleep 0.5 
      set_led_state OFF 
      sleep 0.5 
    end 
  end 
end

 

Some things to note:

  • I am using the arduino_firmata gem, please see the Gemfile for details.
  • The initialize method catches the exception thrown when the Arduino is not connected, as the other methods do, in order to simulate the board in such circumstances. The simulation is always succeeds, by the way, and was coded to allow development without the board connected.
  • arduino.output_digital_read is a monkey-patch to the gem, as I could not find a way to query the board if an output pin was on or off:
module ArduinoFirmata
  class Arduino
    def output_digital_read(pin)
      raise ArgumentError, "invalid pin number (#{pin})" if pin.class != Fixnum or pin &lt; 0
      (@digital_output_data[pin &gt;&gt; 3] &gt;&gt; (pin &amp; 0x07)) &amp; 0x01 &gt; 0 ? ON : OFF
    end
  end
end

All green

Having implemented the code, the tests should now pass and running rake again will run both Cucumber and rSpec, yielding:

~/Documents/projects/arduino (master)$ rake
/Users/ThoughtWorks/.rvm/rubies/ruby-2.2.1/bin/ruby -I/Users/ThoughtWorks/.rvm/gems/ruby-2.2.1/gems/rspec-support-3.3.0/lib:/Users/ThoughtWorks/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib /Users/ThoughtWorks/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb
...

Finished in 7.56 seconds (files took 0.27749 seconds to load)
3 examples, 0 failures

/Users/ThoughtWorks/.rvm/rubies/ruby-2.2.1/bin/ruby -S bundle exec cucumber 
Feature: 
  Assure board led is responsive

  Background:                    # features/initial.feature:4
    Given the board is connected # features/step_definitions/initial_steps.rb:3

  Scenario: Turn led on               # features/initial.feature:7
    When I issue the led "On" command # features/step_definitions/initial_steps.rb:7
    Then the led is "On"              # features/step_definitions/initial_steps.rb:12

  Scenario: Turn led off               # features/initial.feature:11
    When I issue the led "Off" command # features/step_definitions/initial_steps.rb:7
    Then the led is "Off"              # features/step_definitions/initial_steps.rb:12

2 scenarios (2 passed)
6 steps (6 passed)
0m4.579s

 

Make this better!

The project is here. Please feel free to fork and contribute.

Conclusion

How much is “good enough”? If you notice, the assertions are implemented using the data structure exposed by arduino_firmata, not with a call to the board itself. This is always a tradeoff in testing. How far should we go? For this project, testing via data structure is “good enough”. For a medical application, or something that flies a plane, it’s obviously not good enough and we would have to assert on an electric current flowing to the LED. And again, who is to assure us that the LED is actually emitting light?

There’s not much else we can do with a standalone Arduino without any periferals connected, but it’s enough to make sure that everything is set up correctly for future development.

Disclaimer

This installment was to show a quick-and-dirty sanity check without bothering to flash the device.

Afterword

The testing and writing of this installment were made while flying to Barcelona, hoping that fellow passengers would not freak out seeing wires and blinking lights mid-flight.

Happy Arduinoing!

How to reconnect to a database when its connection was lost


One of my projects has a long-running task that constantly needs information from the database. I needed a mechanism to assure that the task will automatically reconnect to the database if and when that connection was broken.

I came up with this scheme using a trick with rescue blocks (code abbreviated for clarity) in this gist.

def my_task

    while(true) do
      begin
        database_access_here
      rescue Exception => ex
        begin
          ActiveRecord::Base.connection.reconnect!
        rescue
          sleep 10
          retry # will retry the reconnect
        else
          retry # will retry the database_access_here call
        end
      end
    end
  end

Here’s a line-by-line explanation:

Line 4: This is where your application’s database access logic would be.

Line 5: Catch a database access exception here

Here is where it gets interesting:

Line 7: Open a new block and retry the connection.

Line 10: This retry will retry the reconnect method and will loop as long as the database connection is still down.

Line 11: The else clause will execute if _no_ exception happened in line 10, and will retry the original database call in line 4.

In my case and example, I am not counting retries because I don’t care that I’ve failed – I must continue to retry. You may want to use “retry if retries < 3” as a break mechanism.

I have also removed some mailer code that notifies me when the reconnect fails so I can (manually) see what happened to the connection. The moment the connection is re-established, life goes on as normal within the infinite while loop.

Weekend warrior – MacRuby and rSpec, Mac OS X Lion, Xcode V4.3.2


Inspired by the recent buzz over RubyMotion, of which I am a proud licensee, I wanted to play a little with MacRuby just to get into the swing of things.

After deciding that doing so was more worthwhile than to mow the lawn, I set out to see what it took to start a project in MacRuby with rSpec support as a basis to start work.

MacRuby’s article got me started, but did not work because the test target could not find the framework that I wanted tested. I don’t know why, since I (sort of) follwed the instructions there. I say “sort of” since the article shows screen-shots of an older Xcode, and even though I thought I set things correctly in my version (Xcode V4.3.2), it still would not build. Also, I am on Mac OS X Lion and that may have had something to do with it.

After realising that if I did not continue trying, a certain member of the household would make me mow that lawn, Google found another article here by Steve Madsen.

It too looked promising, but again, needed tweaking to get working in my environment. It’s thanks to Steve’s post that I managed to get it working.

Here were my steps:
a. Create a new project in Xcode (or use an existing one that you want to rSpec)
b. Install MacRuby
c. Follow Steve Madsen’s instructions

At that stage it still did not work for me, but that was because of a misunderstaning that was clarified quickly enough:

Steve’s screen-shot for the scheme settings on the Specs framework is cut off and does not show the “Expand Variables Based On” setting, so $(SRCROOT) was never expanded for me. I replaced it with an absolute path (ugh) and it worked, so I knew something was not picking up that macro. The solution was to give a value to that drop-down, as shown in the screen-shot below.

If, like me, you’re on Xcode V4.3.2, you might find the following screen-shots useful (just refer to them as you follow Steve’s post):

a. Build settings:
Image

b. Scheme settings:

Image

You cannot imagine the joy of seeing Ruby code drive an Objective-C framework testing session using rSpec in Xcode.

Now to that mower…

Oh, the places you’ll go…


Inspired from the Practicing Ruby entry, I somewhat clarified the code a little (for my taste) and learned that the call stack in Ruby is:

0) Undefined method resolution
1) Methods defined in the object’s singleton class (i.e. the object itself)
2) Modules mixed into the singleton class in reverse order of inclusion
3) Methods defined by the object’s class
4) Modules included into the object’s class in reverse order of inclusion
5) Methods defined by the object’s superclass, i.e. inherited methods

module ModuleA
 def foo
   "- Mixed in method defined by ModuleA\n" + super
 end
end  
module ModuleB
  def foo
   "- Mixed in method defined by ModuleB\n" + super
  end
end  
module ModuleC
  def foo
   "- Extended in method defined by ModuleC\n" + super
 end
end  
module ModuleD
  def foo
   "- Extended in method defined by ModuleD\n" + super
 end
end  
class A
 def foo
   "- Instance method defined by A\n"
 end
end  
class B < A
 include ModuleA
 include ModuleB
 def foo
    "- Instance method defined by B\n" + super
 end  
  def method_missing(method)
   puts "- method_missing (#{method}) on b. Redeirecting to b.foo\n"
   foo
 end
end  
b = B.new
b.extend(ModuleC)
b.extend(ModuleD)
def b.foo
 "- Method defined directly on an instance of B\n" + super
end
def b.method_missing(method)
 "- method_missing (#{method}) on b. Calling super\n" + super
end
puts "Calling 'bar' on b of type #{b.class}:\n"
puts b.bar

Which gives:

~/projects/ita/ruby$ ruby test.rb

Calling ‘bar’ on b of type B:

– method_missing (bar) on b. Redeirecting to b.foo
– method_missing (bar) on b. Calling super
– Method defined directly on an instance of B
– Extended in method defined by ModuleD
– Extended in method defined by ModuleC
– Instance method defined by B
– Mixed in method defined by ModuleB
– Mixed in method defined by ModuleA
– Instance method defined by A

Follow the conversation on Stack Overflow.