Created a web application for transcribing speech real-time using Gemini Canvas

STEPS:

1. In Gemini, go to 'Create canvas app'. 

2. Input the prompt:  

Create a canvas app that can listen to English and then output the transcript in real time. 

===============

OUTPUT: 

https://share.gemini.google/Dx35HwxNjDMl

OBSERVATIONS

1. There are different accents to choose for the voice input. (See top right.)

2. The transcription is quite accurate. 

3. The transcript will automatically put down a fullstop after you pause at the end of a sentence. 

++++++++++++++++++++++++

See Explainer video by Mr Ha: https://www.youtube.com/watch?v=1Bo7zh22Ums&t=49s

Creating a script and an interactive worksheet for a Listening lesson

 

• In Gemini, I asked for a scaffolded monologue script on Endangered Animals for upper primary ESL students. The script should be about 1.5 minutes of Speaking Time, excluding the pauses.
• Gemini provided a table which included the scaffolded script, and 'pedagogical notes' for teachers. The notes contain learning points for each pause, and the students' task ('student action') at each pause.
• I then asked for a 'worksheet' that is derived from the suggested 'student actions'.
• It provided an immediately usable worksheet.
• I then downloaded the worksheet, went to Gemini Canvas, and asked it to convert the worksheet to an interactive version. It worked.

Creating an audio track using Elevenlabs: About the pauses in the prompt

 If the script contains directions about pauses, e.g., [7-second pause], the pauses will not be inserted in the output audio file. 

Poe suggests using Audacity to insert the pauses afterwards. 

Tried using Gemini to create an audio voice recording on endangered animals: OK but ...

 Just tried Gemini chat to generate an audio track from my prompt, 

Please create a 45-second monologue audio track which is an organgutan appealing to human beings not to do things to put them in danger. The audio track is for upper primary ESL students' listening practice. 

BUT IT COULD ONLY GENERATE A 30-SECOND SONG. 

Then I went to Gemini's 'Canvas' function, and used the same prompt. It worked, but: 

Even though I asked for a audio track, it only gave me the script. After 3 to 4 iterations, I finally got the audio (of 45 seconds).

But after I clicked 'play', I had to wait for almost a minute for the audio to load.

I asked for a Download function. It gave it to me on the next iteration. I could download the audio as a wav file. 

I got the Share Link, and tried it in a new tab. (I had to sign in to Poe.) Again, after clicking 'play', it took a minute for the audio to load. 

The Share Link: https://gemini.google.com/share/584aa17992db

I tried the Share link in an incognito browser, it didn't work. 

Created an audio 'How I became a language teacher by Dornyei' from a text using Elevenlabs in Poe: Very convenient

 

Dörnyei, Z., & Kubanyiova, M. (2014). Motivating learners, motivating teachers : building vision in the language classroom. Cambridge University Press., p 132

I then uploaded the audio file downloaded from Poe's Elevenlabs, and uploaded it to Substack to use in class: 

https://open.substack.com/pub/paulsze/p/how-i-became-a-teacher-of-english?r=5cf74s&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true