Voice To Text and Text To Voice

Voice To Text and Text To Voice

Youtube Demo (No audio, screen capture only)

• https://youtu.be/OuZHPqaFwY0

The sample app listens to user’s voice to identify one of the following words

• fish
• cat
• dog

the app shows up the corresponding image that is identified with voice2text. you can trigger the animation or the action too. It works offline without Internet connection.

• Voice To Text $9.99 is a plugin that allows you to convert speech to text using built-in iOS and Android APIs. Please see the documentation for more details. You need to purchase the plugin to test your app on a real device.

• Text To Speech $19.99/Year is a corona plugin that enables your applications on Android and iOS to speak various text in various languages. You can even control pitch, rate, and volume.. You need to purchase the plugin to test your app on a real device.

If you run it on Corona simulator, the sample project will work in debug mode and it returns a dummy word.

• v2t_lib.lua is the external library for voice2text and text2speech. It controls the two plugins.
• v2t_init.lua is the external code to setup voice2text and text2speech. The callback function is implemented to receive a result from voice2text
• v2t_popup.lua is the external code for a button to start voice2text

Sample project is here

• https://kwiksher.com/tutorials/Kwik4/VoicePopUp.zip

Please open the page 1.

Art Layers

• fish shape

• cat shape

• dog shape

  each layer’s name should be a word to be recognized with voice2text.

• v2tBtn is a button to turn on voice2text. It also randomly shows one of the words for user to speak.

• msgTxt is a text layer to show a word for user to speak. Also it shows a result of voice2text.

Kwik Components

Strictly Necessary to control the v2t Code process

• Add External Library for v2t_lib.lua

• but_v2t invokes voice2text

• code_init for v2t_init.lua

• msgTxt is Text replacement

• v2t_act is called when voice2text is processed and you may add actions like playAudio etc. The text from voice2text is returned in the message parameter

      print("voce2text", params.event.message )
  

• v2t_grp contains the layers. If voice2text returns one of the layer names in the group, the animation of the layer will be triggered.

• v2t_store is a variable of a table. The table store the layer that has been returned from voice2text

      UI.v2t_store['dog'] = layer['dog']
  

Option

• anim_{layer name} is one of linear, path anim, pulse, blink,bounce animation that will play when one of layer’s name is retrieved with voice2text. For instance, anim_fish linear animation will be played if ‘fish’ is spoken. You have to set the name as “anim_” + the name of the layer

Add External Library

• Name: v2tLib
• Path: v2t_lib.lua in the project folder

v2t_grp

• Create Groups

  

• dog, cat and fish are the member of v2t_grp. The Name of the group must be v2t_grp because it is referenced from the external code

  

v2t_store

• Create Variable

  

• The name of the variable must be v2t_store

  • Content: Formula/Boolean
  • Type: Local
  • This is a Table: checked
  • Before: selected
  • Keep track : checked
  

but_v2t

• Add button

  

• Add External code - v2t_popup.lua

  

msgTxt

• Text Replacement

  

• Text Replacement

  

The external code accesses layer.msgTxt of displayText object

anim_fish

• Linear Animation

  

• Name: anim_fish

• Starts: wait request

  

Build for device

Publish the project and you need to add the following settings for build4/build.settings file.

• build.settings

  the plugins needs to be declared in build.settings. Also add the android’s permissions.

  settings = {
      plugins  =  {
          ["plugin.voiceToText"] ={
              publisherId = "tech.scotth",
              supportedPlatforms = { iphone=true, android=true }
          },
          ["plugin.texttospeech"] = {
              publisherId = "com.spiralcodestudio"
          },
      },
      android  = {
          usesPermissions  ={
              "android.permission.WRITE_EXTERNAL_STORAGE",
              "android.permission.RECORD_AUDIO"
          },
      },
  
  

Now you can build the build4 folder with Corona Simulator for android or iOS device.

You may set the flag on for Publish setting to prevent build.settings from overwritten when you publish the project for preview again.

• Publish only pages_.lua(do not export main.lua, etc)

that’s all. Enjoy!

Notes

v2t_lib.lua, the sample project is configured voice2text for Japanese. You may change the language code for your preference.

local _M = {}
_M.lang = 'ja-JP'
--_M.lang = 'en-US'
--
...
...

v2t_init.lua is configured voice2text for Japanese to translate ‘cat’, ‘dog’ and ‘fish’, and ‘speak’ in Japanese characters. You may set them in your language. The words in Conv table is used to find the text string out of a result of voice2text. event.message contains a result from voice2text. The sample is configured with Japanese, so voice2text returns text in Japanese characters not a-z, A-Z Ascii. Thus Conv table is used there.

local v2tLib         = require("v2t_lib")
local v2t_STORE_LOAD = false
local isReady        = false
--
v2tLib.Conv = {cat="キャット", dog="ドッグ", fish= "フィッシュ", speak="えいごをよんでください"}
--
...
...
local function callback(event)
    if not event.isError then
		...
		...
                local _target = event.message
                for i = 1, layer.v2t_grp.numChildren do
                    local child = layer.v2t_grp[i]
                    -- Simple string.sub() won't work for UTF-8 strings, use UTF-8 plugin
                    if string.find(v2tLib.Conv[child.name],  _target) then
                        print(child.name)
                        v2tLib.speak(v2tLib.Conv[child.name])
                        UI.v2t_store[child.name] = child
                        child.alpha = 1