Cognitive Speech not working in West Europe #2802

wehnicde · 2020-01-07T09:49:18Z

Screenshots

Version

I use 4.7.1, but it doesn't work on 4.6.0 too.

"<meta name="botframework-directlinespeech:version" content="4.7.1">
<meta name="botframework-webchat:bundle:variant" content="full-es5">
<meta name="botframework-webchat:bundle:version" content="4.7.1">
<meta name="botframework-webchat:core:version" content="4.7.1">
<meta name="botframework-webchat:ui:version" content="4.7.1">"

Describe the bug

When I click on the microphone this error occurs.

On 19.12.2019 everything worked fine and when I tried on 20.12.2019 WITHOUT changing anything at the code it wouldn't work.

This is how I implement the Cognitive Speech Service:

<script>
webSpeechPonyfillFactory = window.WebChat.createCognitiveServicesSpeechServicesPonyfillFactory({
			//	credentials: {
						region: 'westeurope',
						subscriptionKey: 'XXXXX'
			//		}
			})
			window.WebChat.renderWebChat({
				directLine: window.WebChat.createDirectLine({
				//	token: token
					secret: 'XXXXXXXX',
					
				}),
				webSpeechPonyfillFactory: webSpeechPonyfillFactory,
				// disable timestamp
				groupTimestamp: false,
					
				//set language
				locale: 'en-US',
					
			}, 
				document.getElementById('webchat'));
			
         
      </script>

Expected behavior

It should recognize my voice and translate it to text.

Additional context

My bot runs on C#, but I tried on a JS-Bot too, same Error. WebSockets are enabled in the BotConfig.

[Bug]

The text was updated successfully, but these errors were encountered:

stevkan · 2020-01-07T23:51:48Z

Hi @wehnicde, I have been unable to repro your issue. I have a couple clarifying questions for you:

Which browser are you running Web Chat in? It looks like you are using Chrome but you are utilizing the webchat-es5.js package (vs. the regular webchat.js package). Can you try the regular package?
Is this the React version of Web Chat? Your code suggests it isn't, but the rendered web chat component in your image looks a lot like the "minimizable" version which is built in React.
There were some issues with the 4.7.1 build that have come to light recently and will be corrected in 4.7.2. It has been suggested you try build 4.5.3. Can you give this a go and see if it helps?
In createDirectLine(), can you try passing webSocket: true, as well?

wehnicde · 2020-01-08T07:04:23Z

Which browser are you running Web Chat in? It looks like you are using Chrome but you are utilizing the webchat-es5.js package (vs. the regular webchat.js package). Can you try the regular package?

I am running in Chrome but I need IE support too, thats why I use es5. (Does it have any disadvantages?)
I tried the normal package too, but it doesn't work either.

Is this the React version of Web Chat? Your code suggests it isn't, but the rendered web chat component in your image looks a lot like the "minimizable" version which is built in React.

It isn't. The "taskbar" on top is simple html by me (using style.display = none), it is the standart webchat.

There were some issues with the 4.7.1 build that have come to light recently and will be corrected in 4.7.2. It has been suggested you try build 4.5.3. Can you give this a go and see if it helps?

Didn't help :( When I change to 4.5.3 the webchat won't render with the error:

TypeError: e is not a function
    at t.<anonymous> (webchat.js:1)
    at t.createWebSpeechPonyfill (webchat.js:1)
    at t.value (webchat.js:1)
    at Io (webchat.js:1)
    at To (webchat.js:1)
    at Mo (webchat.js:1)
    at Ga (webchat.js:1)
    at Ka (webchat.js:1)
    at Is (webchat.js:1)
    at As (webchat.js:1)

If I take the speech stuff out, it works. With 4.6 it is rendering, did 4.6 change anything about the implementation of webSpeech?

This is my complete HTML-Code (clean version).

<!DOCTYPE html>
<html>
  <head>
    <title>ChatBot CleanHTML</title>
  </head>
  <body>			
    <div id="webchat" role="main"></div>
    <script 
      crossorigin="anonymous"
      src="https://cdn.botframework.com/botframework-webchat/4.5.3/webchat.js"
    ></script>

    <script>			
      window.WebChat.renderWebChat({
        directLine: window.WebChat.createDirectLine({
          webSocket: true,
          secret: 'XXXXXXXX',
        }),
        webSpeechPonyfillFactory: window.WebChat.createCognitiveServicesSpeechServicesPonyfillFactory({
          region: 'westeurope',
          subscriptionKey: 'XXXXXXXXX'
        }),
        locale: 'en-US',
      }, 
    document.getElementById('webchat'));
    </script>
  </body>
</html>

In createDirectLine(), can you try passing webSocket: true, as well?

Didn't change anything.

I have been unable to repro your issue.

I will try to generate a fresh bot and try to repro it by myself again. My problem is, that I already have 2 seperate bots (C# and JS) and it isn't working on both, so I really don't know what the problem could be.

Edit: Just generated a clean bot and put the secret in my html. Didn't work. So the bot-code shouldn't be the problem.

stevkan · 2020-01-11T00:03:51Z

@wehnicde, I'm a little confused on whether you are trying to get this to work in Chrome, IE, or both. If you are having issues with Chrome, I'm certain we can get it working. Your implementation is slightly different but really should work. And, if not, we can play with the code.

As for IE 11, as stated here, Internet Explorer does not meet the minimum requirements for supporting speech recognition and speech synthesis, at least as far as Cognitive Services Speech is concerned.

wehnicde · 2020-01-13T05:48:24Z

Sorry for the confusing. I want to have the webchat working on all browsers, but I know that speech isn't possible with IE11. Writing with the bot should work on every browser, speech only with Chrome/Firefox. Thats why I need to implement the es5 file (Am I right about that?)

What do you mean by "Your implementation is slightly different but really should work"? How is my implementation different? And how is the "normal" way? Maybe it will work with that.

markbschramm · 2020-01-15T05:08:35Z

I am getting the same problem in IE, Chrome, Edge Chrome.
webchat.js:2 WebSocket is already in CLOSING or CLOSED state.
I can provide a URL to a test page that shows the issue if you want. Any chance to DM that?
I'm still using the token generation to make it work. It did work but no longer does so some change was pushed on your end that broke it (we pushed no changes)

I should note that the bot still works in Chrome for typing, when you press the microphone for speech it does ask to allow, then the errors start pouring in and no actual listening seems to happen.

rvallireply · 2020-01-15T11:15:50Z

Adding my voice: getting same error in Chrome, a couple days ago Speech was working, now without changing anything in the code I'm getting webchat.js:2 WebSocket is already in CLOSING or CLOSED state.
Using westeurope region too.

I noticed this issue was just closed (10 hours ago): anything to do with it maybe? (regression?)
#2459

stevkan · 2020-01-15T16:58:46Z

@wehnicde, yes, the es5 version will work in all web browsers the same. Outside of the file package size if that is a concern, you should be just fine.

Never mind my comment about implementation. You're is just fine and acceptable. I thought I saw you doing something slightly different, but I was mistaken.

I did go ahead and create a new CS Speech service a second time (and a new bot for a clean slate tho I also confirmed with an established bot) and was able to repro. I will have an update when we know more.

corinagum · 2020-01-15T17:04:10Z

Thanks @stevkan for the repro. Adding @compulim to the thread.

CoHealer · 2020-01-15T20:28:52Z

Reassigning @stevkan for azure reporting only

compulim · 2020-01-15T23:27:01Z

This bug is very similar to Azure-Samples/cognitive-services-speech-sdk#127.

West Europe services is returning "Specified grammar type is not supported!" when we pass LUIS reference grammar ID.

When we take away LUIS reference grammar ID (in hacked code), it works with West Europe.

So I am assuming Cognitive Services in West Europe is having issue on handling LUIS reference grammar ID.

@stevkan could you file a ticket to them, mentioning LUIS reference grammar ID (a.k.a. IntentRecognizer) is not working West Europe? I have pinged @mahilleb-msft offline, but he is in Munich time zone, no response yet.

wehnicde · 2020-01-16T06:13:11Z

@compulim I am not using LUIS for my bot. Or does Cognitve Speech use LUIS by itself?

compulim · 2020-01-16T11:04:15Z

Although you may not using LUIS in your bot, Direct Line channel do provides a LUIS ID and requires Web Chat to pass it to Cognitive Services. The ID is called "LUIS reference grammar ID".

Even in your case, you are not using LUIS, Direct Line channel still send us the LUIS ID, so we still pass it to Cognitive Services. And I believe Cognitive Services has some issues handling LUIS ID in West Europe region.

We have contacted Cognitive Services team and they are investigating the issue.

wehnicde · 2020-01-16T12:03:36Z

Oh okay. Thanks for the clarification. And thanks for the investigation.

stevkan · 2020-01-16T23:07:25Z

@compulim, created a ticket earlier today in Cognitive Services Speech Uservoice account, here.

markbschramm · 2020-01-17T04:05:45Z

@stevkan The error seems different today. Not sure if you changed anything, I did not: (It's similar)
3SCRIPT12002: SCRIPT12002: WebSocket Error: Network Error 12002, The operation timed out

compulim · 2020-01-17T04:39:52Z

Web Chat only own the client-side. I believe the service team is working on a fix.

phenaxkian · 2020-01-17T10:10:30Z

I'm receiving a similar issue but my Speech is set to the UK South region. Would that also be affected by this?

alokraj68 · 2020-01-17T13:36:00Z

Hi, I am getting this same WebSocket issue in locale:en-US and speech region: eastus

Is there any fix for this? We are not able to use speech service because of this.

rvallireply · 2020-01-20T17:52:11Z

Any news about the "WebSocket is already in CLOSING or CLOSED state." issue? I tried with northeurope and it seems to fail too.
By the way, I am now confused about the right javascript code to use when connecting to cognitive speech. I saw a warning "We are deprecating authorizationToken, region, and subscriptionKey. Please use credentials instead." Could someone provide an up-to-date sample about how to use webchat with cognitive speech service today, please?

wehnicde · 2020-01-22T06:37:59Z

Closing this issue as it is working now. Thanks for the investigation.

compulim · 2020-01-22T17:22:15Z

Thanks @wehnicde for reporting the problem.

wehnicde added Bot Services Required for internal Azure reporting. Do not delete. Do not change color. bug Indicates an unexpected problem or an unintended behavior. Pending customer-reported Required for internal Azure reporting. Do not delete. labels Jan 7, 2020

stevkan self-assigned this Jan 7, 2020

srinaath added customer-replied-to Required for internal reporting. Do not delete. support Internal request for assistance from the Support Team labels Jan 7, 2020

rvallireply mentioned this issue Jan 15, 2020

Update speech options to use credentials #2759

Merged

1 task

corinagum removed the Pending label Jan 15, 2020

corinagum assigned compulim and unassigned stevkan Jan 15, 2020

corinagum added Triage-E and removed support Internal request for assistance from the Support Team labels Jan 15, 2020

CoHealer assigned stevkan Jan 15, 2020

corinagum added the external-cognitive-services label Jan 16, 2020

compulim changed the title ~~Cognitive Speech not working~~ Cognitive Speech not working in West Europe Jan 17, 2020

wehnicde closed this as completed Jan 22, 2020

compulim mentioned this issue Feb 4, 2020

Support Speech Recognition intent endpoint for Bot/LUIS priming #2880

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cognitive Speech not working in West Europe #2802

Cognitive Speech not working in West Europe #2802

wehnicde commented Jan 7, 2020

stevkan commented Jan 7, 2020 •

edited

Loading

wehnicde commented Jan 8, 2020 •

edited by stevkan

Loading

stevkan commented Jan 11, 2020

wehnicde commented Jan 13, 2020

markbschramm commented Jan 15, 2020 •

edited

Loading

rvallireply commented Jan 15, 2020

stevkan commented Jan 15, 2020

corinagum commented Jan 15, 2020

CoHealer commented Jan 15, 2020

compulim commented Jan 15, 2020 •

edited

Loading

wehnicde commented Jan 16, 2020

compulim commented Jan 16, 2020

wehnicde commented Jan 16, 2020

stevkan commented Jan 16, 2020

markbschramm commented Jan 17, 2020

compulim commented Jan 17, 2020

phenaxkian commented Jan 17, 2020

alokraj68 commented Jan 17, 2020

rvallireply commented Jan 20, 2020 •

edited

Loading

wehnicde commented Jan 22, 2020

compulim commented Jan 22, 2020

Cognitive Speech not working in West Europe #2802

Cognitive Speech not working in West Europe #2802

Comments

wehnicde commented Jan 7, 2020

Screenshots

Version

Describe the bug

Expected behavior

Additional context

stevkan commented Jan 7, 2020 • edited Loading

wehnicde commented Jan 8, 2020 • edited by stevkan Loading

stevkan commented Jan 11, 2020

wehnicde commented Jan 13, 2020

markbschramm commented Jan 15, 2020 • edited Loading

rvallireply commented Jan 15, 2020

stevkan commented Jan 15, 2020

corinagum commented Jan 15, 2020

CoHealer commented Jan 15, 2020

compulim commented Jan 15, 2020 • edited Loading

wehnicde commented Jan 16, 2020

compulim commented Jan 16, 2020

wehnicde commented Jan 16, 2020

stevkan commented Jan 16, 2020

markbschramm commented Jan 17, 2020

compulim commented Jan 17, 2020

phenaxkian commented Jan 17, 2020

alokraj68 commented Jan 17, 2020

rvallireply commented Jan 20, 2020 • edited Loading

wehnicde commented Jan 22, 2020

compulim commented Jan 22, 2020

stevkan commented Jan 7, 2020 •

edited

Loading

wehnicde commented Jan 8, 2020 •

edited by stevkan

Loading

markbschramm commented Jan 15, 2020 •

edited

Loading

compulim commented Jan 15, 2020 •

edited

Loading

rvallireply commented Jan 20, 2020 •

edited

Loading