Supreet Singh (00:00) In general, what I see, is your AI companions or AI chatbots, AI conversational AIs, the whole setup. This will become a part of your regular life. I think this should become very common in general. I don't think it will replace teachers or any such thing. That is a very different aspect of education that one should have. But as we already discussed, it will really help with the continuity. It will really help you, you know, helping with your homework, live tutoring. Rishi Ahluwalia (00:33) Hey everyone, welcome to the Convo AI World Podcast. In this podcast, we interview builders, developers and teams that are building solutions for the conversational and voice AI space. And joining us today is Supreet Singh. He's the Senior Director of Engineering at one of the biggest EdTech giants in India called Physics Wallah. So it's lovely to have you here, Supreet, and looking forward to the podcast. Supreet Singh (00:58) Thanks Rishi, thanks for the warm welcome. Looking forward to the conversation. Rishi Ahluwalia (01:02) Absolutely. All right, so Supreet before we dive deep into the podcast and the conversation, right? So I think our audience will probably want to understand your journey in shaping how India learns today. So ⁓ what first drew you to EdTech? And how does that connect with your current interest in the space of conversational AI? Supreet Singh (01:27) I think I have a rather interesting story and background. Probably, a lot of people from smaller towns will be able to relate to that. I'm from a small city called Patiala. It's a very famous city, but not really famous or in general popular about its education infrastructure. Most of the students, they used to go to quota for preparing for IIT-JEE exams (competitive exam for Engineering). And I had no idea about like there's an exam called IIT-JEE. My sister prepared for it so I had an idea and there was this company which came to our city. They opened up a coaching center and we got this awareness, there's something called IIT and it's supposed to be a very good college. So that was how we got to know about, you know, IIT-JEE and all these competitive exams. It was an eye opener, I would say. And that led me to like think, there's so many opportunities in this world, so many things going on and just because of lack of awareness, lack of education, limitations of geography, limitations of affordability, people are not able to prepare and have an equal advantage in terms of such exams. I could see a lot of coaching centers back in the day had very high fees. Even I was like, I think I topped the exam so I got some discount. But in general, if that was not the case, it would have been very, very expensive even for me to afford that. The geographical and affordability factors are something which EdTech really covers. We are able to offer such low-priced courses to all the like pincodes across the country, right? That is how my fantasy, in general, started. I joined an EdTech a couple of years back, let's say three to four years into my career. And since then I've been in EdTech. It's been, I think eight, nine years now. And I've been really, the focus has really shifted to not just affordable, how to really make it personalized. From you know one-to-few model to one-to-many model to now super hyper-personalized one-to-one model is what we are looking at, in general, is what EdTech is trying to achieve and I think that is the next natural step in the EdTech story, that is what i feel. Rishi Ahluwalia (03:35) No, wonderful. I think that's probably the dream of anybody who is coming from a tier-2, tier-3 city to be part of such prestigious organizations powering the next brain of the country and the nervous system of the country. So great insights there. All right. So coming to the next question, and I think this is more related to where you are working right now. So, ⁓ Supreet, from your perspective at Physics Wallah, how do you see conversational AI moving beyond bots with predefined answers to becoming a true partner in teaching and learning? Supreet Singh (04:11) Right. I think before the advent of LLMs, it was all very fantasy zone, like somebody answering you, like the precise answer that you're looking for. That has changed a lot recently. The further shift will be more about context and adaptability. So we do have, let's say, predefined bots which are operating on certain scripts. They're relatively static, I would say. Slowly and gradually, in general, the teaching aspect demands dynamic answers, what the student has a doubt? What do they want to understand? You are stuck on a certain problem due to a certain issue like back in the day, my teacher used to, they used to know like where am I stuck because of what Conversational AI you know powered by LLMs with more contextual learning, with more context of what you are doing, the potential to listen, interpret, and respond in ways wherein it feels like an actual teacher is responding with the empathetic approach, the adaptability they have. Even at Physics Wallah, I see AI becoming not just a help desk or a chat buddy anymore. I think, eventually, the direction that we see in the future might be more towards a core teacher or a personalized-AI teacher sort of a thing. That is a dream. Everybody in the EdTech sector is working on that. Rishi Ahluwalia (05:27) No, absolutely. I think adaptability and adoption is going to play a very, very big role here. And hold on to that thought because I think we can probably deep dive into more of such questions in the next section. Now, let's focus on building conversational AI. So I think since you have seen the transition from offline to online and obviously, to blended learning in the past decade that you have been operating in the EdTech sector. So ⁓ I think this question is more towards engineers now. So how open are engineers towards, you know, embracing new technologies like conversational AI as the next leap in EdTech? Supreet Singh (06:10) I think that's an interesting one. As an engineer, whenever something new comes, it's a mix of emotions, I think. It's a mix of what new we'll get to see? You know, like what are the possibilities? What all can be done? There’s obviously some aspect of skepticism as well, right? What is actually possible? Where will we get stuck or what can we really build? People are also excited, I would say. At the same time, they're also aware of the challenges, you know, specifically with respect to LLM, it's about the accuracy, the cost, the user trust, because in a normal probably ChatGPT conversation, you get a wrong answer, you get a hallucination. Probably you can tackle with it, deal with it. But if you get a wrong answer, that is a problem that leads to mistrust that, in general, has like bigger repercussions. So the openness is there, especially because engineers in EdTech do have this luxury of seeing real world impact happening right in front of their eyes. You release something, students use it, you get that feedback that loop is very fast, very interesting, right? Seeing the impact of your work changing lives in real time is something really great and that is what keeps the enthusiasm up, I think, from an engineering standpoint. Rishi Ahluwalia (07:29) Yeah, I think I'm glad you mentioned enthusiasm there because I think that's that's just part of being in engineering and embracing the next big thing in technology. That's what differentiates engineers working in the field. Great. So I think the other question that I have in mind over here and I think same question might come in the mind of the viewers as well, Supreet, is India as a nation, right? So how do you approach India as a nation when it comes to multilingual and accent diversity, especially in ⁓ education, since Physics Wallah, for example, offers very affordable courses, primarily focusing on students in tier-2 and tier-3 cities. So how can conversational AI ⁓ make a difference there? Supreet Singh (08:15) I think this is a problem which is very core to India. I don't think any other country faces this. Every few kilometers, your language or dialect, something or the other thing changes. It's more of a local-first problem, I would say. Models need to be trained not just on Hindi or English. It can be Hinglish, it can be some mix of some language. It can be Punjabi, Tamil, Telugu mix of this and that. Not just that even in Punjab, I speak Punjabi, right? So there are 10 different ways of speaking Punjabi. So accent, speech, diversity, clarity, everything. There's a lot of factors, right? It's a fairly complicated problem. So we focus on building systems that don't penalize students on how they speak, right? Everybody can have a different accent. It's our responsibility to understand, adapt and you know, combining maybe hybrid models like, you know, ASR, context clues, fallback strategies. That's how we are at least trying to tackle problems. It's currently not a solved problem. I would say, I think this will mature over time and, ⁓ it will need a lot of effort, to really make it work. But yeah, that is something unique to India. And it's a good opportunity that we have here. Rishi Ahluwalia (09:25) Absolutely, I think all of us who belong to India can relate to that. Not even the accent and the language, right? Even the food and the habits change every 20 kilometers. So that's why India is such a diverse nation. So I think the other question I had for you, Supreet, and this is in line with what you answered before. So with such language diversity and such accents, so there is also a balance that you need to identify for real-time performance versus the accuracy in answers that LLM would deliver and how really a text-to-speech engine will voice that out. And all of that is streamed on an audio channel such as like, Agora has their own, ⁓ we have our own conversational AI engine that powers the entire line. So ⁓ again, how do you balance real-time performance versus accuracy when it comes to understanding if the AI is answering correctly or limiting itself to information that has been fed or is it still relying on open-source data mode? Supreet Singh (10:34) Alright, alright, alright. That's an interesting one. This is a problem not limited to EdTech probably. It's a like overall bigger problem to solve. Specifically, in live learning when we talk about it, it's a real-time thing. You cannot have latency. As soon as you have latency, it will definitely lead to mistrust. It will lead to a disengagement in general. You'll have to be responsive obviously. Of course, you'll have to make sure the answer is correct to a certain extent. Maybe a certain draft stage which can be defined in the background. So techniques like streaming responses, caching, frequently respond queries like usually, on a certain topic students have a certain set of doubts, right? It's not like somebody will come and ask a very different doubt which has never been asked in the past. That is something that there's different kind of techniques which can be used to really, you know, improve the speed while maintaining the accuracy. Then we do have our slides. We do have all the context, you know, you either ask it out on a video or a slide on a certain question. So that also again helps a lot with accuracy. The key is transparency, though. Even if you don't know the answer, you cannot be like confident and give the wrong answer. You have to be very, very clear. You have to be very, very transparent and accept if you are not aware of the answer and give that like clearly as a disclaimer here. I am not aware of this answer, right? You cannot be wrong. Essentially that just kills trust in EdTech. In EdTech, you cannot afford to do that. Rishi Ahluwalia (12:03) Yeah, I think that that's another interesting aspect, right? Latency is obviously it powers the overall experience with respect to how quickly the answers are generated and how accurate they are. Plus, the LLM also needs to understand the context and those can actually lead to building learning maps or competitiveness that the student has engaged or developed over a period of time. So yeah, that's a great perspective again, Supreet. Thanks for that. The other thing ⁓ I wanted to bring up is that since we are talking about building conversational AI, students can be very mischievous, right? So how do you expect AI to handle vague or incomplete student questions without frustrating them. And again, this goes both ways, right? So if students give incomplete questions, then the LLM can hallucinate, like you mentioned before, or it can give incomplete or inaccurate answers. I think prompting again plays a very big role here. But in terms of LLM, how do you imagine to handle vague or incomplete student questions without frustrating the experience? Supreet Singh (13:10) Well, yeah, this is a pretty common issue, I would say. In general, students can be mischievous. They can ask whatever they want to ask. Specifically, when there's no accountability, there's no physical teacher in front of you. Normally, what human teachers would do is they would clarify. They would clearly say, what do you want to know? They ask follow up questions. I think there's something conversational AI can do as well. Rather than giving vague answers or unnecessarily trying to be precise when you're not, can ask guiding questions. Is this related to this specific chapter? Or is this related to this previous chapter? Is this the topic that you're referring to? And then students can articulate better. LLMs are very well suited for this because they can generate clarifying dialogue rather than, you know, just giving a flat answer like, 'I don't know'. Right? It's about turning this ambiguity into a teachable moment. In general, I think this really teaches a if you have doubt, you have to be clear, if you're not clear you get a follow-up question, you clarify the situation better and then you get your desired answer. I think this is something that works and that is the correct direction for such a situation. Rishi Ahluwalia (14:24) I think that's a great point again, because I think it all depends on AI impersonating an actual teacher as well, right? Because that happens in a normal environment as well, where the student might be hesitant to ask a particular question, or they might not be able to frame a question in a certain manner. So I think you bring up a great point where the LLM can act as a guide along with being the teacher and the tutor so that the maximum output can be extracted out of the student. And this also helps in confidence building rather than just focusing on learning. So yeah, that's another great aspect. All right, so one more question for the building conversational AI section. How do you see conversational AI adapting to both fast and slow learners in live classrooms? And I think this question is connected to the last question we discussed, Supreet. The last question was more focused on students who might have low self-esteem or low confidence, but there are also students who have ample confidence, right? So they might just end up interrupting the conversational AI more than once. And that can also lead to preserved questions or preserved information. So how do you anticipate to solve this Supreet Singh (15:41) So I think Rishi, that's a good question. This is a real world problem, I think. Even in classrooms, it totally depends on the skill set of the teacher. How do they really balance out? ⁓ Who's a slow learner relatively? Who's a fast learner? How do you really personalize your learning? I have been fortunate enough during my coaching days that we had really amazing teachers like they really helped students out if you need extra time, whatever is necessary right? I think this is where your LLMs and your conversational AI bots can really shine. Personalization is like literally the holy grail which we have been trying to chase in so many years and I think the current setup brings us as close to that as possible. The AI can have that capability to really tweak their pace, really tweak their responses on the basis of how a student is asking questions. Even if we take the example of let's say a test if a certain student lags on certain topics. If you are giving a practice test, the AI engine can clearly understand, you know, the student is weak on this topic and maybe they can switch to easier questions and explain the topic better. Even in, let's say in a group live setting also. This means the teacher gets insights on which student needs more attention. While AI can quietly support those who are struggling without slowing down the entire class. So maybe with some extra questions, maybe with some extra nudges, maybe with some additional practice papers or practice tests. These are the directions which can be taken. And this is something where I think AI can really shine, specifically conversational AI. Rishi Ahluwalia (17:17) And I think it goes back to when you talked about AI adapting and having context. And it does not really matter if these are slow learners or fast learners. So that's where I think AI can really come into the picture. So Supreet, thank you for those responses so far. Now let's move to the next section. And let's deep dive into the live classes environment which you are currently handling at Physics Wallah as well. And let's see how conversational AI can play a significant role and create the impact that it can. So in terms of conversational AI, right? So where do you think it can create ⁓ the biggest or the most significant impact in terms of live classes, especially in the tier-2, tier-3, and even tier-4 cities where Physics Wallah is operating. Because you'll always have those challenges with network. And primarily, the audience and the students use their phones, which might be of low configuration as well. So how do you anticipate conversational AI coming into the picture and where do you see the biggest impact being created? Supreet Singh (18:32) Yeah, so I think even with the, you know, ascent of all the Jio-frenzy 4G coming in, we still have limitations like, 1.5 GB per day. So there are like so many solutions at PW also that we have implemented and just to make sure the experience is great despite low network bandwidth. Primarily, in terms of learning, there are three areas which really need attention. And we focus on as well. The first one is around doubt resolution. Whenever you are understanding a topic, when you are trying to understand something, if you don't understand a certain aspect in between, you lose track. You suddenly get into that vicious circle of, if I don't understand this, how will I understand the next part? Another part which we already talked about is personalization as per my learning needs. So if I need some more time to understand a topic, I'll probably spend want to spend more time on it. And of course, moderation. Since these are virtual classes, students can be naughty. They can create some kind of disturbances in classrooms as well. This is some something where again AI helps us. Doubt resolution is giving instant context-aware answers depending on where you are in the lecture. Personalization is as we already discussed, it's where the real magic lies. How do you really tailor that experience for students depending on their needs? And in terms of moderation, definitely that is something really underrated. It can really help manage the distractions in class, filter out maybe chat messages which are not really relevant, could be inappropriate as well. And even nurture the students to be back on track during a session. We spent enough time, we have gone a bit astray. You guys really need to come back and get back on the pace. So yeah, there's a lot of exciting things in live classes which can be really helped with the help of these technologies. Because you really don't have time to respond, right? It's a real-time live class. So yeah, these things really help. Rishi Ahluwalia (20:27) Yeah, I think just one more thing I would like to add from my standpoint as well that many of these competitive exams that these students are giving, right, they also have a point where they probably need to do interviews with the panel as well. So maybe AI can help groom them in a manner as well and probably act as a practice test-bed where they can practice their communication skills as well, along with the technical skills, the complexity of those skills. So that's another area that might come in handy as well. Great, great. And the next question I have, Supreet, is more around the role of AI in live classes, right? So we usually understand that a co-tutor or let's say a teaching assistant is where the conversational AI can obviously create the biggest impact or can be the biggest impact in terms of use-case adoption. But with all the opportunity comes a set of limitations as well. So what would be ⁓ your response when it comes to the limitations versus the opportunities when AI acts as a co-tutor or a teaching assistant in live classes? Supreet Singh (21:39) I think it's all about focusing on the right set of experiences. AI can handle your repetitive tasks like you know on this certain topic this question will come rather than teacher waiting for the whole class. Your AI can answer these questions, summarize certain key points and that really frees up the teacher's time to focus on higher order interactions. To focus on areas which like matter more, wherein a teacher can absolutely add more value than, let's say a prepared AI bot sort of a thing. I think, but the limit obviously comes with empathy and authority. Students will listen to teachers, but if you are, if there's some AI bot or a teacher, you know, talking to you, there can be a lack of empathy and authority is something which we have to explore yet. I think that's something we learn with time how exactly do students respond to such authority figures. So a student might accept an AI hint, but they'll still look up to the teacher for like a reassurance, like motivation probably. In general, apart from studies, regular mentorship, the right push that you really need when you're studying. I see AI as an enhancer. not a replacement. I think probably this is valid for a lot of other industries as well. In our jobs, something similar is happening. So AI really is an enhancer, I would say, not really a replacement. But yeah, we'll have to see what the future holds. Exciting times ahead, I think. Rishi Ahluwalia (23:06) No, absolutely. Definitely exciting times OK, Supreet, so moving on, I have another interesting question for you. Now, I think we have focused a lot on the during live class experience. But I think there is also a significant part where conversational AI will come into play in terms of pre-class or post-class experience. So how do you envision AI companions continuing the learning support beyond the classroom? Supreet Singh (23:36) I think this is the more interesting aspect of conversational AI is where I personally believe we might see a lot of, you know, new interesting things coming up. During a classroom, you still have your teacher, you still have your peers and everything, right? After the class is where you really need that support, where you, maybe you need help with your homework, right? You need help, you know, practice questions with the AI companions, probably. Your AI companion can probably guide you, ⁓ this is where you are making a mistake, right? This is all just where I think things might go eventually. It can really help students who are struggling with, let's say, finishing their homework or completing their assignment or, you know, getting stuck on a certain topic. What do I revise? What do I focus more on? Maybe, you know, a good thorough analysis of your tests and you get to know ⁓ what topics you should really focus on and spend time on. So this continuity bridges the gap between your real classroom setup and your self-study setup. I think this will make your learning experience a lot more holistic and reduce that barrier between that live-class setup and a self-learning module setup. Rishi Ahluwalia (24:47) Absolutely great. think in the context of competitive exams, right? So these bright minds when they are at work and trying to maybe think of new ideas or new explanations or new theories. So AI can probably ⁓ become that post-class companion that can ⁓ maybe guide them or maybe just give them responses and also act as a companion. Or to build or bridge the gaps that you clearly mentioned, Supreet. So yeah, wonderful. All right. So I think the other aspect of the podcast that we wanted to deep dive into is the future outlook. Right now, obviously, since you have been in the EdTech sector for over a decade, you've already seen the transition happening. And very quickly, the transition is happening, and especially now with so much AI infra in place. And things moving at the pace of not. So what's your prediction for conversational AI in the next five years with respect to education sector? Supreet Singh (25:49) I think it's a rather like a question which sounds easy actually is very difficult. I have seen as you correctly pointed out more like almost a decade worth of EdTech probably even more than that. I never really thought we would be at this stage so soon. It's really difficult. It's really fun to see how fast we are advancing. ⁓ In general, what I see, is your AI companions or AI chatbots, AI conversational AIs, the whole setup. This will become a part of your regular life. I think this should become very common in general. I don't think it will replace teachers or any such thing. That is a very different aspect of education that one should have. But as we already discussed, it will really help with the continuity. It will really help you, you know, helping with your homework, live tutoring. maybe personalized assessments, right? Your AI tutors become a part of your life. You are a part of your study routine, right? Especially in a country like India, wherein the teacher is to student ratio is a bit off, right? This is where in we can bridge the gap and really help students out with their learning journeys, in general. Rishi Ahluwalia (26:55) And absolutely, I think, and with the population that we have in India and especially the young population, right? So we still have 60% population, which is under the age of 30, if I remember that stat correctly. But still, we are a young nation and education is always at the core. So yeah, I think it's hard to predict the next five years, but thanks for giving us that insight. On that thought, let's do a quick rapid fire here, Supreet. So I'm not going to put you in a spot, but let's do a quick rapid fire. So a conversational AI product you admire. Supreet Singh (27:34) So I think we have a lot of AI tools in place. ChatGPT is doing a great job. There's a lot of other tools as well. So I think it is going to be about specific workloads that specific AI bots can do. I think with the context of conversational AI. Agora's AI conversational part is doing a great job, is what I personally felt. I tried out the demo I played around, the accuracy, the speed, the latency numbers are pretty low. It really feels like a real conversation. The accuracy to understand accents, different languages. That whole thing is binded really well and I think the experience has been pretty spot on for me. Rishi Ahluwalia (28:15) Great, that's like music to my ears, Supreet. Thank you so much for that. All right, so next rapid fire question. So one feature you would love to see in the next generation of AI Engine. Supreet Singh (28:27) I think personally, I feel ⁓ keeping the context tied for a longer duration is something we all really want in general. Nobody wants to prompt again and again, give the context again, remind the I was talking about this, right? Like true contextual memory, right? You need to have that ability to stay with the student, you know, from the start of the learning journey till the end, because that whole combination of journey is what really matters, right? Right from the first class to the day of exam. Maybe it's too much to ask for, probably. I'm not sure. But if we can really achieve that, that'll be super duper helpful for students in their learning journeys. Rishi Ahluwalia (29:06) And I think it would be beneficial for both students and their families, right? Because that will probably end up in analytics for learning improvement plans and possibly improvement heat maps as well. So yeah, that's another great response there, Supreet. All right, so the last question for the rapid fire is, what do you expect to be the most surprising student behavior in live classes? Supreet Singh (29:31) I think not just live class, it's my personal opinion as well. Students can be shy. A lot of times I observe people, know, shy away from questions that they really have. People rather not have their doubts clear than, you know, ask for the answers to their doubts. I think probably AI is something which can help. On the side, ask your doubts and you do not really have to be in the center of focus of the classroom, right? I think students are just afraid of focus in a classroom. So yeah, that is something pretty surprising which still exists to this day. Rishi Ahluwalia (30:05) Yeah, I think to that point, I'd like to add one thing. AI does not judge. So I think those students might find it easier to ask those questions, even if those questions do not make logic or, you know, are not the right kind of questions as well. But that's what maybe AI can augment further. All right. So. I think ⁓ the other thing our viewers will be very interested here, Supreet, is considering your background and the diversity that you have seen in the sector. So do you have any advice for maybe innovators or startups that might be looking to experiment with conversational AI and eventually bind conversational AI into their product portfolio. Supreet Singh (30:52) That gave me wise men vibes. I'm not that old. But I think this is something everybody should really like. Nothing has changed probably with the advent of technology. Technology has always been there. Technology will keep on evolving. ⁓ I think everybody needs to focus on the problem. If you have a very well defined problem, you can very well find the technology to solve that problem. You need to be aware of the technologies available, but your focus has to be on the problem statement. Education has a lot of high cycles, right? But building those meaningful products can be really helpful and easy when you start from the problem statement itself. So a lot of products that are built in they start their journey from student feedback, right? So we float out a lot of forms. We do ask students a lot of feedback. Teachers ask students a lot of feedback. Like what do you really think is bothering you? What is really not helping in your journey? Right? So starting from that, using the right set of tools, tools are just tools, right? What are you really doing with those tools or what are you using these tools for is what matters. I firmly believe that. Rishi Ahluwalia (32:02) Yeah, I think it's more about keeping it simple and focusing on the problem at hand rather than looking to create new problems might be the best advice out there. Thank you. Thank you, Supri. That makes so much sense. All right, so Supreet, I'll probably not bother you with more technical questions there, but I think the viewers on this podcast who have tuned in would love to know ⁓ more about you beyond your professional life. So outside of work, what inspires you the most? Supreet Singh (32:35) Like genuinely from the bottom of my heart, I'm a person who travels a lot. Like even two days back, I was not in the city. Probably the weekend before that, probably the weekend before that. The change in environment, meeting new people, looking at new situations, looking in different, different environments is something that really motivates me. Really gives me that energy, gives me that refreshment to think, right? Apart from that, of course, you might find me if not outside the city, I'll be in my balcony sipping a cup of coffee with some book. I'm reading Elon Musk's biography right now. Like I really think what people have done in their lives. Great people like Elon Musk, Dell, Steve Jobs, right? They've done so much. These books really give you that insight, right? What can be done, what should be done, how like as a motivation. Of course, everybody has their own journey and way of doing things, but this really gives you a kick. ⁓ You did this by the age of this, using these many tools. But that gives me like a really good kick, pushes me and prepares me for the, upcoming Monday for sure. Rishi Ahluwalia (33:42) Yeah, absolutely. That's great. Travel is really a medicine which is talked about very, very less. And especially with the stressful environments that all of us are working in, travel can definitely be the change we all need. So thank you so much, Supreet, ⁓ for your time today. Again, it was a pleasure to have you. And thank you so much again for your time. Supreet Singh (34:05) Thank you so much Rishi for having me. It's always great to share whatever knowledge I have attained with the audiences. And I really hope it brings and adds some value to somebody's life. Rishi Ahluwalia (34:19) No, indeed, absolutely. Thank you so much again, Supreet. Supreet Singh (34:22) Thank you so much for your time, Rishi.