ELE 403:

Senior Project

Voice Activated Robot Programming

By Nate Lentz

Submitted to

Prof. Julie Ellis

Prof. Carlos Lück

Prof. Jim Masi

May 12^th, 2007

Table of Contents

Voice Activated Robot Programming

Abstract: 3

Introduction: Voice Activation Programming.. 4

Experimental Setup: 5

Procedure: 5

AnALysis: 8

Resources: 9

conlusion: 9

Abstract:

The goal of this project was to write a voice activation interface for the Microbot TeachMover robot. This was accomplished using Visual Studio C++ and the Microsoft Speech SDK (software developer’s kit) Win32 Speech API (SAPI, speech application program interface). A standard PC headset with microphone was used to interface with the soundcard. The DSP (digital sound processing) came from existing standards established in most sound cards used in the Windows Operating Systems. The SAPI SDK made the interfacing the speech recognition engine with the robot control software less complex. This project consisted of three distinct blocks of code; a speech interface, pattern-matching speech with robot commands and kinematic math applications used to calculate steps sent to individual motors on the robot. All of this software was designed in Visual Studio.

A pre-existing serial communications package was used to send steps to the Microbot. This package was created in Visual C++ and consisted of files that handle communications between the computer’s db9 serial port (COM1) and the Microbot. Dr. Carlos Lück provided the base code for these files, which also defined the structure for the Microbot class and constructor prototypes for potential kinematic functions. The code that was created for this project did not use any of the existing function prototypes. This project integrated new research in voice activation techniques and built code from the Microsoft SAPI. A high level kinematic realization for the Microbot was developed, coupled with the voice activation software.

The voice activation interface was built as a 32bit Windows application and is designed to deal with errors that typically occur in command recognition. The project was designed so the speaker must determine if the appropriate recognition sequence has been realized. Once the correct syntax is realized it is sent to robot. A console widow receives the spoken command, if the recognition engine doesn’t replicate the command properly (i.e. a homonym or misspoken word) the command will not be sent. The application of these concepts will be demonstrated by moving a queen around on a chessboard. The C++ software stores a series of movements that can be referenced by using three basic commands; find, drop and basic chess movements, for example “Bishop Two” will move the queen to the “Bishop Two” position. Developing heuristics was beyond the scope of this project. The program will not know if the movement is a correct chess movement, it will just move the piece, using the basic pick and place operations.

Introduction: Voice Activation Programming

The objective of this project was to become familiar with voice activation programming. There are many potential avenues where this technology may be employed, from assisting the handicapped, to creating convenience for the general population.

Applications using voice activation are very prolific. They are used in the medical profession, data retrieval services, smart home implementations and device manipulation; such as voice commands using a GPS (global positioning systems) in a car. In November 2004, Microsoft announced that it’s voice command software would let gadgets users, running the Pocket PC operating system, issue spoken commands to get calendar and contact information, make phone calls, access applications, and perform other basic functions. The software could also be used with the handheld version of Microsoft's Media Player to allow voice-activated control of digital music playback[1]. There is a lot of research and development currently being performed on voice activated web searching. “Searching the Internet on handhelds can be maddening because of tiny keyboards and unfriendly designs, which require, at best, a stylus or, at worst, a series of clicks through numerous menu screens. Speech recognition software may provide a better way”[2] Google has spent a lot of resources tackling this issue. The biggest problem with voice activation and voice recognition technology is the ability for the DSP chip (digital signal processor) to distinguish speech from other noise in the environment. The programs must also recognize all the variances of the human speech, from accents to dialects to speech impediments.

Figure 1 Pocket PC/ Phone/ PDA

Microsoft has provided a developers programming kit specifically for speech applications, Microsoft SDK 5.1 that uses SAPI. Function calls from within the SAPI dramatically reduce the amount of code required for an application to use speech recognition techniques. SAPI provides a high-level interface between an application and speech engines. SAPI implements all the low-level details needed to control and manage the real-time operations of various speech engines.

Experimental Setup:

The application environment was Microsoft’s Visual Studio C++. Microsoft also provided the SDK (software developer’s kit) for writing speech driven applications, the SAPI (sound application program interface). Microsoft’s Speech SDK 5.1 was used and is a free download from their website. There was a lot of documentation and tutorials that were read and interpreted for this project. The application that was created interfaced a voice recognition engine with a microphone to send complex kinematic step commands to the TeachMover Microbot.

There were three distinct portions of code that made up this project; the first was the speech recognition, the second was interfacing the recognized speech with the Robot and the third applied the actions to the movement of a chess piece. The SAPI was implemented as a 32bit Windows application with a console to monitor the recognition objects. Once the correct object was recognized it was written to a text file. This was required to properly affect pattern-matching techniques. The correct command pattern is then matched with specific functions using a co-ordinate system to effect the correct movement of the Microbot. Complex co-ordinate conversions required a working knowledge of kinematics. The inverse kinematic functions were utilized to find angle displacements from co-ordinate inputs.

Position and Orientation data was gathered from a chessboard. A precise system of calibration was developed due to frequent slippage of the Microbot, and the truncation of values. This occurs when float values are converted to integers in the step calculations of the Microbot joint motors. A queen chess piece was selected as the primary means to exhibit the voice activation environment. Half of the chessboard grid was mapped from queen to castle. This should provide a sufficient area to exhibit recognition examples.

Procedure:

The Microsoft Speech SDK uses many predefined powerful functions, classes and commands. The Speech API commands are interspersed with Windows API commands. Once the project was defined it consumed a lengthy amount of time for the analysis and development. There was a huge learning curve trying to implement the SAPI. Object calls from within the SAPI class are extremely memory intensive; allocating resources became paramount in the code design. Preprocessor directives were employed to lessen the load of the complier. Using the #ifdef, #endif and #else preprocessor directives allowed the active or true sections of the code to be compiled, leaving the false/inactive functions out of memory. The process for building the 32bit Window’s application was defined in the resource.h file using standard Visual studio wizards.

The speech recognition logic used by this SAPI implementation was taken from different examples in the SDK. The first step was to initialize the COM, component object model, by creating an instance of the object. By initializing the “COM one make’s sure it is present and active. Use the COM command CoInitialize() and later CoUninitialize(). To ensure COM is active throughout the application’s session, these commands are usually nested around the main message loop.” [3] The WinMain function was the call to use a 32bit Windows application. The next step was to create the Recognizer object in order to recognize that speech was being invoked. The Recognizer object, (ISpRecognizer), was the interface that set a recognition notification sequence. This object provided access to the recognition engine. If an event was recognized the speech record interest was set into recognition status, the default audio input was created and the input to the speech engine was set. Once the recognizer alerted the speech engine that it was active, a grammar tool was specified and loaded. This project utilized a dictation grammar. The third step is the event recognition. A recognition context is set for the speech engine. “A context is any single area of the application needing to process speech“ [4].

Once the engine was deemed active and was actively processing speech one of three things happened; an event, notification or interest. Examples of events in the code are included, “when a sound was first detected on the microphone, (SPEI_SOUND_START), when it ends (SPEI_SOUND_END), or when it successfully completes word recognition (SPEI_RECOGNITION)” [5]. Notifications occur concurrently with events, they alert when a SAPI event occurs. Notifications can be used in error checking to make sure the event occurred properly. By default SAPI will send about thirty different events and notifications back to the application. An interest functions as a filter and is set to only use specific events and discard the rest. In the case of this project the only event that was filtered as an interest was the recognition event (SPEI_RECOGNITION ). When this event was realized and there was a successful word match between the recognized event and the loaded grammar, the notification would consider the event to be a successful recognition.

The next step was to output the recognition event to the 32bit Window’s console. The 32bit application window looped the speech engine looking for additional recognition events. Once the program user determined that the correct 2-word phrase (the command or specific chess piece movement) has occurred the 32bit Window closes and the Robot code starts to execute. After the exit status in the 32bit application has occurred the exact 2-word command phrase is written to a text file.

The Robot code was written in two distinct parts. The first part was the pattern recognition section. This code interprets the outputted text file from the voice recognition sequence, and breaks it down into individual characters placed into an array. The pattern matching becomes a simple character search upon the elements within the array. There were two commands used, find and drop, and there were four positional elements used as well; queen, bishop, horse and castle. Knight was not used because of the homonym problem with night. These six words have unique first letter so the pattern match ended there. When a positional selection was made there was an accompanying number. was also in the array from one to eight. The numbers two and three and four and five, have the same first letter, but the second letters are unique which ends the pattern matching in those instances. The positional co-ordinates gathered from the chessboard were assigned to the appropriate pattern matching instances.

The next portion of the code applied the logic used in the pick and place operations for the Microbot. The Microbot’s home position was calibrated using a glass block. The endefactor’s position and orientation were described with three-dimensional positional co-ordinates, x, y and z and three orientation descriptors, pitch, roll and gripper (P, R and G). The Microbot has 5 degrees of freedom and a gripper value. The home position values were stored in a text file, which were read into the system. The system also needed to keep track of the queen chess piece. These co-ordinates were also stored in a text file and read into the system.

Once the appropriate command was realized a system of logical steps were sent to the Microbot. For instance when any command was realized the logic employed to send a specific change to the Microbot was; load the home file (endefactor location), calculate joint angles of the Microbot from the positional data, load the piece location co-ordinates, calculate the joint angles, compute angle difference, calculate steps to effect angle displacement, store and write all new calculations of position and orientation for further action, send steps to the joint motors. The Microbot received steps to individual motors and then moved to the appropriate co-ordinates. The command logic put together a series of these moves to affect pick and place operations. The find piece logic hovered the endefactor over the piece, opened the gripper, dropped down on top the piece, closed the gripper and then picked the piece up, totaling five distinct series of steps sent to the Microbot. When the command sequence recognized a movement command, it read the file for the current position, checked itself and read the file for the new position and then sent the appropriate steps to hover over the new position. The drop command lowered the endefactor to the board, opened the gripper, raised the robot arm, closed the gripper and then moved back to the home position.

There were three functions used to compute the kinematics of the Microbot. The first was the inverse kinematic function, which took the position and orientation of the endefactor and computed the joint angles of the Microbot that realized the same co-ordinates. The second function was the conversion function that found the angular displacement values between the existing position and the desired position, and then calculated integer steps to be sent to the stepper motors in the Microbot joints. The forward kinematic function was only called once; it takes joint angle data and calculates the position and orientation of the endefactor.

Analysis:

The application that was developed for this project accomplishes the goals that were set out in the proposal. A voice-activation interface was created to move the Microbot. The entire code is listed in the Appendix. Some of the processes were extremely redundant so similar type comments were removed. The development cycle was split into three distinct phases; the Voice recognition phase, the Pattern recognition phase and the Microbot logic phase. Writing the voice activation portion of the code proved to be a lengthy and difficult process. The tutorials in the SDK were extremely helpful yet complex. The code that was previously developed for Microbot serial communications used a DOS console window. The SAPI implementation that was developed for this project used a 32bit Windows application. Attempts were made to meld the two together, but there was some difficulty with the compiler.

A secondary goal in the original proposal for this project discussed potentially using the voice activation interface with a web interface, to move the Microbot remotely with voice commands. I offer this task to someone else. I have written the interface in PERL for Dr. Lück and loaded an Apache server in the Microbot Lab to move the robot remotely. Someone could tie these two projects together easily. The course requirements were met for this project with the amount of time, new research and new code that was written.

Resources:

The faculty advisors that assisted with this project were Dr. Carlos Lück and Dr. Jim Masi. The software resources that were used in this project were Microsoft Speech SDK and Microsoft Visual C++. The Questech TeachMover Microbot was the robot used. There was no money spent on this project, just a lot of time researching, reading and writing code.

Conclusion:

This project created a program that uses speech recognition to move a robotic arm. The Microsoft Speech application interface made it possible to integrate voice recognition technology with any type of programming. This project integrated a kinematic mathematic code project with the speech recognition interface.

There are many potential applications of using speech recognition with robotics. Robot arms are used in many industrial settings and specific voice command sequences could be used as fail safe shutdowns. The applications for the disabled and handicapped population are endless. A paralyzed person without control of their arms could activate a robotic arm with speech recognition. This could provide a more self-sufficient existence.

In the future robots will become more prevalent within general society. Their value has already been proven in industry. A number of novelty toys and Mall attractions already exist. There is a robotic bartender that will mix drinks in a hermetically sealed environment, and serve them through a window to the buyer. There are toys that can be programmed with voice activation sequences. The ROBO-Sapien line of toys can be programmed with C++ to receive voice commands and additional code. As our society becomes more techno-centric voice activation will become commonplace. A typical use will be accessing devices like phones, PDAs and handheld PCs. The fields of robotics and voice recognition programming will continue to expand and grow as society uses more and more technology.

APPENDIX – C++ Code

#include "stdafx.h"

#include "resource.h"

#include <sphelper.h>

#include "simpledict.h"

#include <fstream>

#include <iomanip>

#include <iostream>

#include <math.h>

#include <cmath>

#include <string>

#include "kinematics.h"

//Function Prototypes

bool inverse (float ,float,float,float,float,float, float&, float&, float&, float&, float&, float&, float&,float&,float&,float&,float&);

int forward (float,float,float,float,float,float, float&, float&, float&, float&, float&, float&);

bool conversion (float&,float&,float&,float&,float&,float&,float&,float&,float&,float&,float&,float&,int&,int&,int&,int&,int&, int& );

bool

LRESULT CALLBACK SimpleDictDlgProc( HWND, UINT, WPARAM, LPARAM );

//Global Variables

float Ht = 195;

float L1 = 177.8;

float LL1 = 96.5;

float pi = 3.14159265359;

float home[8];

float store[8];

float store2[8];

int store3[8];

float theta[8];

bool test, notnum;

Microbot robot; // Local variable of the microbot class

Registerspace delta; // Local variable for input of motor steps

int spe=240; // Motor speed; should not be higher than 240

int R10=1;

float p;

float x, y, z, P, R, G;

using namespace std;

// Win API Entry point

int APIENTRY WinMain(HINSTANCE hInstance,

HINSTANCE hPrevInstance,

LPSTR lpCmdLine,

int nCmdShow)

{

#ifdef _WIN32_WCE

if (SUCCEEDED(::CoInitializeEx(NULL,COINIT_MULTITHREADED)))

#else

if (SUCCEEDED(::CoInitialize(NULL)))

#endif

{

{

CSimpleDict SimpleDictClass( hInstance );

::DialogBoxParam( hInstance, MAKEINTRESOURCE( IDD_SIMPLEDICTDLG ), NULL,

(DLGPROC) CSimpleDict::SimpleDictDlgProc, (LPARAM) &SimpleDictClass );

}

::CoUninitialize();

}

{

//after exit from the dialog box Robot reads the text file and does its program

// Do ROBOT CODE.

int i = 0;

do { store3[i]=0; store2[i]=0; i++; } while (i < 8);

bool letter = true ;

ifstream infile;

infile.open( "c:/sr/start.txt" );

char ChessWd[20];

char a=0;

while(infile)

{

//process the line

infile >> ChessWd[a];

cout << ChessWd[a]<< endl; //test purpose

a++;

}

infile.close();

a=0;

int pattern1; //int value assigned to pattern match

int pattern2;

int pattern3;

int pattern4;

int pattern5;

int pattern6;

bool checke = false;

while (letter) //pattern matching

{

if (ChessWd[a] == 'Q' || ChessWd[a] == 'q' ) //queen reco

{

cout << " Q is 1st letter "; //test

if (ChessWd[a+5] == 'o')

{

cout << " number is 1 ";

//int value assigned to pattern match

pattern1 = 200 ;

pattern2 = 0 ;

pattern3 = 30 ;

}

if (ChessWd[a+5] == 't' && ChessWd[a+6]== 'w' )

{

cout << " number is 2 ";

pattern1 = 220 ;

pattern2 = 0 ;

pattern3 = 35 ;

}

if (ChessWd[a+5] == 't' && ChessWd[a+6]== 'h' )

{

cout << " number is 3 ";

pattern1 = 245 ;

pattern2 = 0 ;

pattern3 = 40 ;

}

if (ChessWd[a+5] == 'f' && ChessWd[a+6]== 'o' )

{

cout << " number is 4 ";

pattern1 = 265 ;

pattern2 = 0 ;

pattern3 = 40 ;

}

if (ChessWd[a+5] == 'f' && ChessWd[a+6]== 'i' )

{

cout << " number is 5 ";

pattern1 = 285 ;

pattern2 = 0 ;

pattern3 = 40 ;

}

if (ChessWd[a+5] == 's' && ChessWd[a+6]== 'i' )

{

cout << " number is 6 ";

pattern1 = 305 ;

pattern2 = 0 ;

pattern3 = 45 ;

}

if (ChessWd[a+5] == 's' && ChessWd[a+6]== 'e' )

{

cout << " number is 7 ";

pattern1 = 325 ;

pattern2 = 0 ;

pattern3 = 50 ;

}

if (ChessWd[a+5] == 'e')

{

cout << " number is 8 ";

pattern1 = 370 ;

pattern2 = 0 ;

pattern3 = 50 ;

checke = true;

}

break;

}//end queen

if (ChessWd[a] == 'B' || ChessWd[a] == 'b') //Bishop reco

{

cout << " B is 1st letter "; //test

if (ChessWd[a+6] == 'o')

{

cout << " number is 1 ";

pattern1 = 200 ;

pattern2 = -25 ;

pattern3 = 30 ;

}

if (ChessWd[a+6] == 't' && ChessWd[a+7]== 'w' )

{

cout << " number is 2 ";

pattern1 = 220 ;

pattern2 = -25 ;

pattern3 = 35 ;

}

if (ChessWd[a+6] == 't' && ChessWd[a+7]== 'h' )

{

cout << " number is 3 ";

pattern1 = 245 ;

pattern2 = -25 ;

pattern3 = 40 ;

}

if (ChessWd[a+6] == 'f' && ChessWd[a+7]== 'o' )

{

cout << " number is 4 ";

pattern1 = 266 ;

pattern2 = -21 ;

pattern3 = 40 ;

}

if (ChessWd[a+6] == 'f' && ChessWd[a+7]== 'i' )

{

cout << " number is 5 ";

pattern1 = 287 ;

pattern2 = -21 ;

pattern3 = 40 ;

}

if (ChessWd[a+6] == 's' && ChessWd[a+7]== 'i' )

{

cout << " number is 6 ";

pattern1 = 310 ;

pattern2 = -19 ;

pattern3 = 45 ;

}

if (ChessWd[a+6] == 's' && ChessWd[a+7]== 'e' )

{

cout << " number is 7 ";

pattern1 = 325 ;

pattern2 = -17 ;

pattern3 = 50 ;

}

if (ChessWd[a+6] == 'e')

{

cout << " number is 8 ";

pattern1 = 370 ;

pattern2 = -21 ;

pattern3 = 50 ;

checke = true;

}

break;

}//end bishop

if (ChessWd[a] == 'H' || ChessWd[a] == 'h') //horse reco

{

cout << " H is 1st letter "; //test

if (ChessWd[a+5] == 'o')

{

cout << " number is 1 ";

pattern1 = 200 ;

pattern2 = -50 ;

pattern3 = 30 ;

}

if (ChessWd[a+5] == 't' && ChessWd[a+6]== 'w' )

{

cout << " number is 2 ";

pattern1 = 220 ;

pattern2 = -50 ;

pattern3 = 35 ;

}

if (ChessWd[a+5] == 't' && ChessWd[a+6]== 'h' )

{

cout << " number is 3 ";

pattern1 = 245 ;

pattern2 = -48 ;

pattern3 = 40 ;

}

if (ChessWd[a+5] == 'f' && ChessWd[a+6]== 'o' )

{

cout << " number is 4 ";

pattern1 = 266 ;

pattern2 = -48;

pattern3 = 40 ;

}

if (ChessWd[a+5] == 'f' && ChessWd[a+6]== 'i' )

{

cout << " number is 5 ";

pattern1 = 288 ;

pattern2 = -48 ;

pattern3 = 40 ;

}

if (ChessWd[a+5] == 's' && ChessWd[a+6]== 'i' )

{

cout << " number is 6 ";

pattern1 = 310 ;

pattern2 = -48 ;

pattern3 = 45 ;

}

if (ChessWd[a+5] == 's' && ChessWd[a+6]== 'e' )

{

cout << " number is 7 ";

pattern1 = 325 ;

pattern2 = -48 ;

pattern3 = 50 ;

}

if (ChessWd[a+5] == 'e')

{

cout << " number is 8 ";

pattern1 = 370 ;

pattern2 = -48 ;

pattern3 = 50 ;

checke=true;

}

break;

}//end horse

if (ChessWd[a] == 'C' || ChessWd[a] == 'c' ) //Castle reco

{

cout << " C is 1st letter ";

if (ChessWd[a+6] == 'o')

{

cout << " number is 1 ";

pattern1 = 200 ;

pattern2 = -75 ;

pattern3 = 35 ;

}

if (ChessWd[a+6] == 't' && ChessWd[a+7]== 'w' )

{

cout << " number is 2 ";

pattern1 = 220 ;

pattern2 = -75 ;

pattern3 = 35 ;

}

if (ChessWd[a+6] == 't' && ChessWd[a+7]== 'h' )

{

cout << " number is 3 ";

pattern1 = 245 ;

pattern2 = -73 ;

pattern3 = 40 ;

}

if (ChessWd[a+6] == 'f' && ChessWd[a+7]== 'o' )

{

cout << " number is 4 ";

pattern1 = 267 ;

pattern2 = -72 ;

pattern3 = 40 ;

}

if (ChessWd[a+6] == 'f' && ChessWd[a+7]== 'i' )

{

cout << " number is 5 ";

pattern1 = 289 ;

pattern2 = -72 ;

pattern3 = 40 ;

}

if (ChessWd[a+6] == 's' && ChessWd[a+7]== 'i' )

{

cout << " number is 6 ";

pattern1 = 310 ;

pattern2 = -72 ;

pattern3 = 45 ;

}

if (ChessWd[a+6] == 's' && ChessWd[a+7]== 'e' )

{

cout << " number is 7 ";

pattern1 = 325 ;

pattern2 = -72 ;

pattern3 = 45 ;

}

if (ChessWd[a+6] == 'e')

{

cout << " number is 8 ";

pattern1 = 370 ;

pattern2 = -72 ;

pattern3 = 50 ;

checke=true;

}

break;

}

if (ChessWd[a] == 'D' || ChessWd[a] == 'd')

{

cout << " Drop Command "; //test

break;

}

if (ChessWd[a] == 'F' || ChessWd[a] == 'f')

{

cout << " Find piece Command "; //test

break;

}

cout << " no match "; //test

break;

}

//pattern value constants

pattern4 = -90;

pattern5 = 0 ;

pattern6 = 0 ;

if (checke)

{

cout << " number is in 8th position ";

pattern4 = -45 ;

}

int ny;//Pause

cin >> ny;

ifstream homefile;

homefile.open( "c:/sr/home.txt" ); //retrieve co-ordinates for Home position of endefactor

i=1;

while( homefile )

{

homefile >> home[i];

i++;

}

homefile.close();

x = home[1];

y = home[2];

z = home[3];

P = home[4];

R = home[5];

G = home[6];

inverse (x, y, z, P, R, G, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

store[1] = theta[1] ; //endefactor starting angles

store[2] = theta[2] ;

store[3] = theta[3] ;

store[4] = theta[4] ;

store[5] = theta[5] ;

store[6] = theta[6] ;

ifstream piecestatus;

piecestatus.open( "c:/sr/status.txt" ); //retrieve co-ordinates for piece position values

i=1;

while( piecestatus )

{

piecestatus >> status[i];

i++;

}

piecestatus.close();

x = status[1]; //assign piece position data

y = status[2];

z = status[3];

P = status[4];

R = status[5];

G = status[6];

printf( "The Status X position is currently ");

cout << x << endl;

printf( "The StatusY position is currently ");

cout << y << endl;

printf( "The StatusZ position is currently ");

cout << z << endl;

printf( "The Status pitch value is currently ");

cout << P << endl;

printf( "The Status roll value is currently ");

cout << R << endl;

printf( "The Status gripper value is currently ");

cout << G << endl;

if (ChessWd[a] == 'F' || ChessWd[a] == 'f') //Find piece command option

{

//Find step 1: Microbot hover over piece position

int tmpz = z + 50 ;

//find angular data

inverse (x, y, tmpz, P, R, G, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

//convert angular displacement to motor step data

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0; // Assign number of steps for each motor

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

//Find step 2: Microbot hover/open gripper on piece position

float tmpG = G + 40;

inverse (x, y, z, P, R, tmpG, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

//convert angular displacement to motor step data

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0; // Assign number of steps for each motor

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

//Find step 3: Microbot down on piece position

inverse (x, y, z, P, R, G, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0;

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

//Find step 4: Microbot gripper close on piece

tmpG = G - 25;

//find angular data

inverse (x, y, tmpz, P, R, tmpG, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

//convert angular displacement to motor step data

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0; // Assign number of steps for each motor

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

//Find step 5: Microbot lift over piece position

tmpz = z + 50;

//find angular data

inverse (x, y, tmpz, P, R, G, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

//convert angular displacement to motor step data

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0; // Assign number of steps for each motor

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

status[1] = x; //assign status position data

status[2] = y;

status[3] = z;

status[4] = P;

status[5] = R;

status[6] = G;

ofstream outfile;

outfile.open( "c:/sr/status.txt" );

int i=1;

while( i < 8)

{

outfile << status[i];

outfile << " ";

i++;

}

outfile.close();

}//end find logic

else if (ChessWd[a] == 'D' || ChessWd[a] == 'd') //Drop piece close gripper

{

cout << " Drop Command "; //test

x = status[1]; //assign infile position data

y = status[2];

z = status[3]; //really z+50

P = status[4];

R = status[5];

G = status[6];//really G+15

//Drop step 1: Microbot down

int tmpz = z - 50;

inverse (x, y, tmpz, P, R, G, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0; // Assign number of steps for each motor

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

//Drop step 2: Microbot open

float tmpG = G + 25;

inverse (x, y, tmpz, P, R, tmpG, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0; // Assign number of steps for each motor

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

//Drop step 3: Microbot up

//tmpz = z;

inverse (x, y, z, P, R, G, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0; // Assign number of steps for each motor

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

//Drop step 4: Microbot gripper close

tmpG = G - 40;

inverse (x, y, z, P, R, tmpG, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0; // Assign number of steps for each motor

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

//output to status file

status[1] = x; //assign status position data

status[2] = y;

status[3] = z;//still z+50

status[4] = P;

status[5] = R;

status[6] = tmpG;//really orig G value

ofstream outfile;

outfile.open( "c:/sr/status.txt" );

int i=1;

while( i < 8)

{

outfile << status[i];

outfile << " ";

i++;

}

outfile.close();

}//end microbot drop logic

else

{

// Move step 1 : Move Robot from existing position -> to pattern match/new location - hover

//assume up position with piece in gripper

inverse (x, y, z, P, R, G, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

store[1] = theta[1];

store[2] = theta[2];

store[3] = theta[3];

store[4] = theta[4];

store[5] = theta[5];

store[6] = theta[6];

x = pattern1; //value from pattern match

y = pattern2;

z = pattern3;

P = pattern4;

R = pattern5;

G = pattern6;

int tmpz ; //modified position variables for step 1

float tmpG;

//calculate angles from new position

inverse (x, y, z, P, R, G, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

//calculate steps from angles

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0; // Assign number of steps for each motor

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

// step 2 hover -> down

tmpz = z-50;

inverse (x, y, tmpz, P, R, G, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0;

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta);

// step 3 open gripper

tmpG = G + 40;

inverse (x, y, tmpz, P, R, tmpG, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0;

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

// step 4 lift up

//tmpz = z + 50;

inverse (x, y, z, P, R, G, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

//calculate steps from angles

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0; // Assign number of steps for each motor

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta); // Send instruction to the microbot

// step 5 close gripper

tmpG = G-40;

inverse (x, y, z, P, R, tmpG, theta[1], theta[2], theta[3], theta[4], theta[5], theta[6],acc1,acc2,acc3,acc4,acc5);

conversion(theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], store2[1],store2[2],store2[3],store2[4],store2[5],store2[6],store3[1],store3[2],store3[3],store3[4],store3[5],store3[6]);

delta.r[7]=0;

delta.r[6]=store2[6];

delta.r[5]=store2[5];

delta.r[4]=store2[4];

delta.r[3]=store2[3];

delta.r[2]=store2[2];

delta.r[1]=store2[1];

robot.SendStep(spe, delta);

//output to status file

status[1] = x; //assign status position data

status[2] = y;

status[3] = z;//still z+50

status[4] = P;

status[5] = R;

status[6] = tmpG;//really orig G value

ofstream outfile;

outfile.open( "c:/sr/status.txt" );

int i=1;

while( i < 8)

{

outfile << status[i];

outfile << " ";

i++;

}

outfile.close();

};

forward (theta[1], theta[2], theta[3], theta[4], theta[5], theta[6], x, y, z, P, R, G); // calculate forward kinematics

return 0;

}//WinMain end

}

//Voice Activation Functions

//Main message handler

//LPARAM is the CSimpleDict object

LRESULT CALLBACK CSimpleDict::SimpleDictDlgProc(HWND hDlg,

UINT message,

WPARAM wParam,

LPARAM lParam)

{

CSimpleDict *pThis = (CSimpleDict *) ::GetWindowLong( hDlg, GWL_USERDATA );

switch( message )

{

case WM_INITDIALOG:

::SetWindowLong( hDlg, GWL_USERDATA, lParam );

pThis = (CSimpleDict *) lParam;

if ( !pThis->InitDialog( hDlg ) )

{

::DestroyWindow( hDlg );

}

break;

case WM_RECOEVENT:

// All recognition events send this message

pThis->RecoEvent();

return TRUE;

break;

case WM_COMMAND:

if ( LOWORD( wParam ) == IDC_BUTTON_EXIT )

{

::EndDialog( hDlg, TRUE );

}

break;

case WM_CLOSE:

::EndDialog( hDlg, TRUE );

break;

case WM_DESTROY:

// Release the recognition context and the dictation grammar

pThis->m_cpRecoCtxt.Release();

pThis->m_cpDictationGrammar.Release();

break;

default:

return FALSE;

break;

}

return TRUE;

}

// Creates the recognition context

bool CSimpleDict::InitDialog( HWND hDlg )

{

m_hDlg = hDlg;

HRESULT hr = S_OK;

CComPtr<ISpRecognizer> cpRecoEngine;

hr = cpRecoEngine.CoCreateInstance(CLSID_SpInprocRecognizer);

if( SUCCEEDED( hr ) )

{

hr = cpRecoEngine->CreateRecoContext( &m_cpRecoCtxt );

}

// Set recognition notification

if (SUCCEEDED(hr))

{

hr = m_cpRecoCtxt->SetNotifyWindowMessage( hDlg, WM_RECOEVENT, 0, 0 );

}

if (SUCCEEDED(hr))

{

// This specifies when the engine has recognized something

const ULONGLONG ullInterest = SPFEI(SPEI_RECOGNITION);

hr = m_cpRecoCtxt->SetInterest(ullInterest, ullInterest);

}

// create default audio object

CComPtr<ISpAudio> cpAudio;

hr = SpCreateDefaultObjectFromCategoryId(SPCAT_AUDIOIN, &cpAudio);

// set the input for the engine

hr = cpRecoEngine->SetInput(cpAudio, TRUE);

hr = cpRecoEngine->SetRecoState( SPRST_ACTIVE );

if (SUCCEEDED(hr))

{

// Specifies that the grammar we want is a dictation grammar.

// Initializes the grammar (m_cpDictationGrammar)

hr = m_cpRecoCtxt->CreateGrammar( GID_DICTATION, &m_cpDictationGrammar );

}

if (SUCCEEDED(hr))

{

hr = m_cpDictationGrammar->LoadDictation(NULL, SPLO_STATIC);

}

if (SUCCEEDED(hr))

{

hr = m_cpDictationGrammar->SetDictationState( SPRS_ACTIVE );

}

if (FAILED(hr))

{

m_cpDictationGrammar.Release();

}

return (hr == S_OK);

}

// dialog process is notified of a recognition event.

//Inserts whatever is recognized into the edit box.

void CSimpleDict::RecoEvent()

{

USES_CONVERSION;

CSpEvent event;

// Process all of the recognition events

while (event.GetFrom(m_cpRecoCtxt) == S_OK)

{

switch (event.eEventId)

{

case SPEI_SOUND_START:

m_bInSound = TRUE;

break;

case SPEI_SOUND_END:

if (m_bInSound)

{

m_bInSound = FALSE;

if (!m_bGotReco)

{

// The sound has started and ended,

// but the engine has not succeeded in recognizing anything

const TCHAR szNoise[] = _T("<noise>");

::SendDlgItemMessage( m_hDlg, IDC_EDIT_DICT,

EM_REPLACESEL, TRUE, (LPARAM) szNoise );

}

m_bGotReco = FALSE;

}

break;

case SPEI_RECOGNITION:

// There may be multiple recognition results, so get all of them

{

m_bGotReco = TRUE;

static const WCHAR wszUnrecognized[] = L"<Unrecognized>";

CSpDynamicString dstrText;

if (FAILED(event.RecoResult()->GetText(SP_GETWHOLEPHRASE, SP_GETWHOLEPHRASE, TRUE,

&dstrText, NULL)))

{

dstrText = wszUnrecognized;

}

// Concatenate a space onto the end of the recognized word

dstrText.Append(L" ");

::SendDlgItemMessage( m_hDlg, IDC_EDIT_DICT, EM_REPLACESEL, TRUE, (LPARAM) W2T(dstrText) );

//handler test for specific voice command

//Output 2 words of reco object to a txt file

ofstream writefile;

writefile.open( "c:/sr/start.txt" );

writefile << W2T(dstrText);

writefile.close();

}

break;

}

}

}

//Kinematics - MATH FUNCTIONS FOR ROBOT

// inverse kinematics function - Takes co-ordinate data and returns angular positions

bool inverse (float x,float y,float z,float P,float R,float G, float& th1, float& th2, float& th3, float& th4, float& th5, float& th6,float& acc1, float& acc2, float& acc3, float& acc4, float& acc5)

{

float R10;

float pRAD = P * (pi /180);

if ((x==0) && (y==0))

{

th1 = 0;

} else if ((x==0) && (y!=0))

{ th1 = (pi / 2) * ( y / abs(y) ); } else th1 = atan (y / x);

if ((x < 75) && (z < Ht))

{

notnum = false;

printf( "The microbot could hurt itself");

return notnum;

}

th1 = -1 * th1 * (180 /pi);

R10 = 1 ;

int i = 0;

do { printf("\n",i++); } while (i < 25);

//inverse kinematics calculation section

float Rprime = R - (R10 * th1);

th5 = P + Rprime ;

th4 = P - Rprime ;

float RR = (x/abs(x))*sqrt( pow(x,2) + pow(y,2) ) ;

float R0 = RR - (LL1 * cos(pRAD));

float Z0 = z - ( LL1 * sin(pRAD)) - Ht;

float Beta = atan2 (Z0, R0) * (180/pi) ;

float tmp = 4*pow(L1,2)/( pow(Z0,2)+ pow(R0,2));

float Alpha = atan(sqrt(tmp-1))* (180/pi) ;

th2 = Alpha + Beta;

th3 = Beta - Alpha;

th6 = G ; //sets gripper value

//checks for imaginary results of kinematics calculations

//will be blocked by the verification section if set to 9999

if (tmp <= 1)

{

th2= 9999;

th3= 9999;

}

return notnum; //end verify code section

}

bool conversion(float& th1,float& th2,float& th3,float& th4,float& th5,float& th6, float& st1, float& st2, float& st3, float& st4, float& st5, float& st6, int& pos1, int& pos2, int& pos3, int& pos4, int& pos5, int& pos6 )

{

float delth1 = store[1] - th1;

float delth2 = store[2] - th2;

float delth3 = store[3] - th3;

float delth4 = store[4] - th4;

float delth5 = store[5] - th5;

float temp1 = delth1*19.64;

float temp2 = delth2*19.64;

float temp3 = delth3*11.55;

float temp4 = (delth4)*4.27;

float temp5 = (delth5)*4.27 ;

float temp6 = th6*11.55 + temp3;

pos1 = temp1 + pos1;

pos2 = temp2 + pos2;

pos3 = temp3 + pos3;

pos4 = temp4 + pos4;

pos5 = temp5 + pos5;

pos6 = temp6 + pos6;

st1 = temp1 ;

st2 = temp2 ;

st3 = temp3 ;

st4 = temp4 ;

st5 = temp5 ;

st6 = temp6 ;

store[1] = th1;

store[2] = th2;

store[3] = th3;

store[4] = th4;

store[5] = th5;

store[6] = th6;

return test;

}

//Calculates forward kinematic solution

int forward (float th1,float th2,float th3,float th4,float th5,float th6,float& x,float& y,float& z,float& P,float& R,float& G) //calculation function

{

int copyhome[8];

P = (th5 + th4)/2;

R = (th5 - th4)/2;

th1 = th1 * pi/180 ;

th2 = th2 * pi/180 ;

th3 = th3 * pi/180 ;

P = P * pi/180 ;

float RR = (L1 * cos(th2))+(L1 * cos(th3))+(LL1 * cos(P));

z = Ht+(L1 * sin(th2))+(L1 * sin(th3))+(LL1 * sin(P));

x= RR * cos(th1);

y= RR * sin(th1);

P = P* (180/pi);

copyhome[1]= x;

copyhome[2]= y;

copyhome[3]= z;

copyhome[4]= P;

copyhome[5]= R;

copyhome[6]= G;

ofstream outfile;

outfile.open( "c:/sr/home.txt" );

int i=1;

while( i < 8)

{

outfile << copyhome[i];

outfile << " ";

i++;

}

outfile.close();

return num1;

}

---

[1] http://news.zdnet.com/2100-3513_22-5137597.html

[2] http://www.techreview.com/read_article.aspx?id=16725&ch=infotech

[3] Microsoft Speech SDK (SAPI 5.1)

[4] Ibid

[5] Ibid