6 months ago
2
OpenAI announced the motorboat of a caller benchmark, IndQA, for evaluating AI systems that absorption connected Indian languages, connection patterns successful the country, and fashionable disciplines there.
The ChatGPT-maker noted that it worked with implicit 250 Indian experts crossed languages and domains successful bid to make its caller IndQA benchmark, spanning 2,278 questions crossed 12 languages and 10 taste domains.
“While our purpose is to make akin benchmarks for different languages and regions, India is an evident starting point. India has astir a cardinal radical who don’t usage English arsenic their superior language, 22 authoritative languages (including astatine slightest 7 with implicit 50 cardinal speakers), and is ChatGPT’s 2nd largest market,” said OpenAI successful a blog post.
The topics that IndQA focuses connected are Architecture & Design, Arts & Culture, Everyday Life, Food & Cuisine, History, Law & Ethics, Literature & Linguistics, Media & Entertainment, Religion & Spirituality, and Sports & Recreation.
The languages covered see Bengali, English, Hindi, Kannada, Marathi, Odia, Telugu, Gujarati, Malayalam, Punjabi, Tamil, and adjacent Hinglish.
“We usage IndQA to measure however caller frontier models execute and illustration advancement implicit the past mates years. With IndQA we tin spot that OpenAI’s models person improved importantly implicit clip connected Indian languages (with caveats), but inactive person important country for improvement. We look guardant to improving show and sharing results for aboriginal models,” noted OpenAI successful its blog post.
The institution is seeking to grow its beingness successful India, arsenic it sets up its archetypal India bureau successful Delhi and looks to prosecute endowment successful the country, arsenic good arsenic advertises its products to Indian customers.
Last month, OpenAI announced that users successful India could get its low-cost offering, ChatGPT Go, disposable for escaped for a year.
