Audio samples from "Emotionally Situated Text-to-Speech Synthesis in User-Agent Conversation"

In order to show the effect of emotion analysis in EmoSit-TTS, we provide the emotion predictions for the agent in the conversation for Sample 1-4.

Sample 1

User: Thanks for inviting me to work out with you, Joan.

Agent: Don't mention it, let's go in. (happy)

User: Yeah, this place looks great. Wow, look at her, she can certainly get down, can't she?

Agent: She sure can. Are you jealous, Leslie? (neutral)

User: A little, I wish I could do that.

Agent: You can! With a little practice. (happy)

FCTalkerDialogueGCN-basedEmoSit-TTS w/o emotion modeling(Proposed)EmoSit-TTS

Sample 2

User: Good morning, Mister Jones!

Agent: Good morning, Miss Bell! (neutral)

User: Nice to see you again.

Agent: Nice to see you too. It's been a long time. (happy)

User: How is everything?

Agent: Just fine, and you? (happy)

FCTalkerDialogueGCN-basedEmoSit-TTS w/o emotion modeling(Proposed)EmoSit-TTS

Sample 3

User: It does. I've learned some chinese folk songs this way. They are clearly presented and easy to follow.

Agent: Can I join you, katherine? I do want to have a try. (happy)

User: Why not? I downloaded a beautiful song the other day. We can learn it together now.

Agent: Great. Do you think we need a chinese dictionary? (happy)

FCTalkerDialogueGCN-basedEmoSit-TTS w/o emotion modeling(Proposed)EmoSit-TTS

Sample 4

User: This is tough to say, John, but I think we should break up.

Agent: Umm Are you serious? (surprise)

User: Yes, I mean it.

Agent: But why? Did I do anything wrong? (sad)

FCTalkerDialogueGCN-basedEmoSit-TTS w/o emotion modeling(Proposed)EmoSit-TTS

Samples for EmoSit-TTS trained on pseudo labels

User: Thanks for inviting me to work out with you, Joan.

Agent: Don't mention it, let's go in.

User: Yeah, this place looks great. Wow, look at her, she can certainly get down, can't she?

Agent: She sure can. Are you jealous, Leslie?

User: A little, I wish I could do that.

Agent: You can! With a little practice.

EmoSit-TTSEmoSit-TTS trained on pseudo labels

User: Waiter, can you come here for a moment?

Agent: Yes, is there something wrong?

User: I'm afraid you've made a mistake. This isn't what I ordered.

Agent: So sorry, I must have confused the orders. I'll check it.

User: And also this soup is cold.

Agent: I'm really sorry, I'll bring you a hot one right away.

EmoSit-TTSEmoSit-TTS trained on pseudo labels