Interface: ISpeechPhraseElement
ISpeechPhraseElement Code Example
The following Visual Basic form code displays the properties of the ISpeechPhraseElement object. The ISpeechPhraseElement object is contained in the ISpeechRecoResult object returned by a RecoContext's Recognition event. This code shows two ways to create a recognition result object:
- Perform recognition directly
- Emulate recognition using the EmulateRecognition method
The example also demonstrates the use of the SpFileStream object in conjunction with the AudioOutputStream of the voice and the AudioInputStream of the recognizer.
To run this code, create a form with the following controls:
- Two command buttons called Command1 and Command2
- A list box called List1
- A text box called Text1
Paste this code into the Declarations section of the form.
The code will set the command button captions as shown in the illustration.
The Form_Load procedure creates a recognizer, a recognition context, and a grammar object. It loads the grammar object with sol.xml, the Solitaire grammar from the SAPI sample code. It then activates the command and control (C and C) and dictation components of the grammar, and places a Solitaire command in the text box. Users can enter whatever text they like in the text box, but best recognition results will be obtained from phrases matching the rules in the C and C grammar; for example, sentences such as "Move the black ten to the jack of diamonds," or "Play the red queen."
The command button captioned Recognition speaks text from the text box into an audio file, and then performs speech recognition of that file. The command button captioned EmulateRecognition simply calls the EmulateRecognition method.
When the speech recognition (SR) engine has completed recognition, it generates a Recognition Event that returns an ISpeechRecoResult object. The Recognition procedure instantiates each ISpeechPhraseElement in the result object's Elements property (a collection of ISpeechPhraseElement objects), and displays selected phrase element properties in columns in the list box.
AudioStreamOffset and AudioSizeBytes
The first two columns show the AudioStreamOffset and AudioSizeBytes properties. These two properties indicate the boundaries of an element in the input audio stream. AudioStreamOffset points to the beginning of the element and AudioSizeBytes is the element's length. The sum of an element's AudioStreamOffset and AudioSizeBytes is the same as the AudioStreamOffset of the next element.
In an ISpeechPhraseElement object created by the EmulateRecognition method, the AudioStreamOffset and AudioSizeBytes properties are zero.
AudioTimeOffset and AudioSizeTime
The next two columns are the AudioTimeOffset and AudioSizeTime properties, which delimit the phrase elements in 100-nanosecond units of time. AudioTimeOffset indicates the beginning time of the element, and AudioSizeTime is its time length. The sum of an element's AudioTimeOffset and AudioSizeTime is approximately the same as the AudioTimeOffset of the next element.
In an ISpeechPhraseElement object created by the EmulateRecognition method, the AudioTimeOffset and AudioSizeTime properties are zero.
DisplayAttributes
The next column is the DisplayAttributes property, which defines how the text of the element is displayed relative to text from other phrase elements. The SpeechDisplayAttributes enumeration lists the possible values of this property. All elements in the example above have a DisplayAttributes property of two, which is the value of the SDA_One_Trailing_Space constant, indicating that the element should be displayed with a trailing space.
EngineConfidence, ActualConfidence, and RequiredConfidence
The three numbers in parentheses are the three property values involving confidence in the recognition of the phrase element. The first of these is the EngineConfidence property, which represents the SR engine's level of confidence in the recognition. The ActualConfidence property reduces the EngineConfidence to one of three confidence levels: low, normal or high. The RequiredConfidence property specifies the confidence level that the ActualConfidence property must equal or surpass.
In an ISpeechPhraseElement object created by the EmulateRecognition method, the EngineConfidence and RequiredConfidence properties are zero. If the emulated phrase matches C and C rules, the ActualConfidence property is one; otherwise it is zero.
DisplayText and LexicalForm
The DisplayText property is the next column in the example. It consists of the recognized text of the phrase element, with normalization of numbers, ordinals, and currency values. The LexicalForm property, not shown in this example, returns the same text, but without normalization.
Pronunciation
To the right of the DisplayText is data from the Pronunciation property. Each number represents a phoneme, and the phonemes represent the pronunciation of the phrase element.
In an ISpeechPhraseElement object created by the EmulateRecognition method, the Pronunciation property is Empty.
RetainedStreamOffset and RetainedSizeBytes
The RetainedStreamOffset and RetainedSizeBytes properties are not shown in this example. If the current recognition context is retaining audio data, then RetainedStreamOffset and RetainedSizeBytes are the same as AudioStreamOffset and AudioSizeBytes, respectively; otherwise, both properties are zero.
In an ISpeechPhraseElement object created by the EmulateRecognition method, the RetainedStreamOffset and RetainedSizeBytes properties are zero.
Option Explicit
Const WAVEFILENAME = "C:\ISpeechPhraseElement.wav"
Dim MyRecognizer As SpeechLib.SpInprocRecognizer
Dim MyGrammar As SpeechLib.ISpeechRecoGrammar
Dim MyFileStream As SpeechLib.SpFileStream
Dim PhraseElem As SpeechLib.ISpeechPhraseElement
Dim MyVoice As SpeechLib.SpVoice
Dim WithEvents MyRecoContext As SpeechLib.SpInProcRecoContext
Private Sub Command1_Click()
On Error GoTo EH
List1.Clear
Set MyFileStream = MakeWAVFileFromText(Text1.Text, WAVEFILENAME)
MyFileStream.Open WAVEFILENAME
Set MyRecognizer.AudioInputStream = MyFileStream
EH:
If Err.Number Then ShowErrMsg
End Sub
Private Sub Command2_Click()
On Error GoTo EH
List1.Clear
MyRecoContext.Recognizer.EmulateRecognition Text1.Text
EH:
If Err.Number Then ShowErrMsg
End Sub
Private Sub Form_Load()
On Error GoTo EH
' Create Recognizer, RecoContext, Grammar, and Voice
Set MyRecognizer = New SpInprocRecognizer
Set MyRecoContext = MyRecognizer.CreateRecoContext
Set MyGrammar = MyRecoContext.CreateGrammar(16)
Set MyVoice = New SpVoice
Set MyVoice.Voice = MyVoice.GetVoices("gender=male").Item(0)
' Load Grammar with solitaire XML, set active
MyGrammar.CmdLoadFromFile "C:\sol.xml", SLOStatic
MyGrammar.CmdSetRuleIdState 0, SGDSActive 'Set MyRecoContext & MyRecoContext active
MyGrammar.DictationSetState SGDSActive 'Set Dictation active
Text1.Text = "play the eight of clubs"
Command1.Caption = "&Recognition;"
Command2.Caption = "&EmulateRecognition;"
EH:
If Err.Number Then ShowErrMsg
End Sub
Private Function PhonesToString(ByVal arrV As Variant) As String
Dim ii As Integer, S As String
On Error GoTo EH
If IsEmpty(arrV) Then
PhonesToString = ""
Else
For ii = 0 To UBound(arrV)
If Len(S) Then
S = S & "," & arrV(ii)
Else
S = arrV(ii)
End If
Next ii
PhonesToString = S
End If
EH:
If Err.Number Then ShowErrMsg
End Function
Private Sub MyRecoContext_Recognition _
(ByVal StreamNumber As Long, _
ByVal StreamPosition As Variant, _
ByVal RecognitionType As SpeechLib.SpeechRecognitionType, _
ByVal Result As SpeechLib.ISpeechRecoResult)
Dim X As String
Dim T As String
Dim A1 As Long, A2 As Long
Dim T1 As Long, T2 As Long
Dim C1 As Single, C2 As Integer, C3 As Integer
On Error GoTo EH
For Each PhraseElem In Result.PhraseInfo.Elements
'Audio data
A1 = PhraseElem.AudioStreamOffset
A2 = PhraseElem.AudioSizeBytes
X = Format(A1, "000000") & " " & Format(A2, "000000") & " "
'Time data
T1 = PhraseElem.AudioTimeOffset
T2 = PhraseElem.AudioSizeTime
X = X & Format(T1, "000000000") & " " & Format(T2, "000000000") & " "
'Display attributes
X = X & Format(PhraseElem.DisplayAttributes) & " "
'Confidences
C1 = PhraseElem.EngineConfidence
C2 = PhraseElem.ActualConfidence
C3 = PhraseElem.RequiredConfidence
T = "(" & Format(C1) & " " & Format(C2) & " " & Format(C3) & ")"
X = X & Left(T & " ", 14)
'Text and pronunciation
X = X & Left(PhraseElem.DisplayText & " ", 14)
X = X & PhonesToString(PhraseElem.Pronunciation)
List1.AddItem X
Next
EH:
If Err.Number Then ShowErrMsg
End Sub
Private Sub MyRecoContext_EndStream _
(ByVal StreamNumber As Long, _
ByVal StreamPosition As Variant, _
ByVal StreamReleased As Boolean)
On Error GoTo EH
'Recognition uses the Filestream, EmulateReco does not
If ActiveControl.Caption = "&Recognition;" Then MyFileStream.Close
List1.AddItem ""
EH:
If Err.Number Then ShowErrMsg
End Sub
Private Function MakeWAVFileFromText _
(ByVal strText As String, _
ByVal strFName As String) _
As SpFileStream
On Error GoTo EH
' Declare identifiers:
Dim FileStream As SpFileStream
Dim Voice As SpVoice
' Instantiate Voice and FileStream objects:
Set Voice = New SpVoice
Set FileStream = New SpFileStream
' Open specified .wav file, set voice output
' to file, and speak synchronously:
FileStream.Open strFName, SSFMCreateForWrite, True
Set Voice.AudioOutputStream = FileStream
Voice.Speak strText, SVSFIsXML
' Close file and return reference to FileStream object:
FileStream.Close
Set MakeWAVFileFromText = FileStream
EH:
If Err.Number Then ShowErrMsg
End Function
Private Sub ShowErrMsg()
' Declare identifiers:
Const NL = vbNewLine
Dim T As String
T = "Desc: " & Err.Description & NL
T = T & "Err #: " & Err.Number
MsgBox T, vbExclamation, "Run-Time Error"
End
End Sub