Voice to Chess Example
In this example we create an algo that processes user commands in the form of voice and then controls the computer to make chess moves. This algo did require some setup work, not show in the video. The setup consisted of a separate algo using the mouse input data source. We then clicked each position on the chess board and saved those to a csv file. This example does assume some basic familiarlity with algo, it is recommneded to start with easier examples if you're not comfortable yet.
The basic algo structure for voice to chess is as such:
Data Source - User Input (Voice Command) -> Events 1 and 2 (Get Positions) -> Trigger (Move Mouse)
Data Source - User Input (Voice Command)
This data source is a lot simpler than a funtion data source. All we do is select user input and then speech to text as the type. The output is automatically filled in and no parameters are required. Note: This does require settings an open ai api key on the key window for the speech to text transcription.
Events 1 and 2 (Get Positions)
We have the text of our voice commands from the data source, now we have to parse that and find the chess positions. The user could say A2 to A4 or just A2, A4 and we need to be able to handle multiple cases.
Settings
We have to create the calculation function for finding the chess position. For getting the first position, we ask the AI to find the chess position matching the patter of a letter A-H followed by a number 1-8, like on a chess board. For the second event, we can take the function we already created and modify it, using the modify button, and asking for the second position.
For both events, we'll use the data set tables to select the data source and rename it to the suggestion. There are no parameters required for these events and that page can be skipped for each.
Trigger (Move Mouse)
We've parsed the command into the two positions we need. Now we need to figure out where the program needs to click for us based on those commands.
Settings
The trigger calculation function can be a generic function that returns always True. We want the output function to always run IF there is new data for it. Actions only update if there is a reason to update, like new data. In this case if there is new data, new chess positions from voice command, we will always be updating.
The output function needs to take those two chess positions and using a mapping of chess positions to screen position as a parameter, click on the screen on the first position, followed by a click on the screen on the second position. We still have to make that mapping that we're passing in as a parameter but that's on the next page.
For the data set, we need to add BOTH the events and name them to the suggestions. Make sure to match up event position 1 and 2 to their correct name for the trigger.
Parameters
On the parameter page we have to create a setup function. A setup function is a function that is run once on startup of the program and creates the parameter value we need. In this case it is creating the mapping of the chess position to screen position mapping that the trigger function uses. To do this, we're using the csv file of screen positions that we mentioned all the way at the start. We're going to read that in based on how we clicked on the screen and map each of those positions to their corresponding chess position and return that mapping.
This is just one way of doing this, we could have made the setup data include the chess positions or we could have made setting the chess positions as part of the algo. Do whatever makes the most sense for you.
Wrapping Up
We've finished the whole algo and we are done with the creator now. Going back to the control page we can hit start on this algo. This will enable the input button. Hitting input will open up another window with controls for inputting our voice. Remember this requires setting the open ai api key! Use your voice command and the algo will click for you to play chess.