IndicNLP

It is not just a medium for communication. It is the medium of the mind. It shapes how we think. It is what helped us outsmart other animals, by carrying the knowledge of medicine, astronomy and etc over to us from the past. It is also the realm, where poetry lives. It is a time capsule of our history. It is where our ancestors are still alive. It is what comes close to what we might call heaven.

There have been thousands of languages in history, languages with written form and grammar has survived more compared to languages without a written form. Currently, Technology is playing the role of Written form and Grammar.

Language is Beautiful

Language is not a bunch of words strung together that we exchange in order to get our needs met.

What is IndicNLP?

IndicNLP is an open source initiative for NLP on Indigenous languages.

Indian languages are complex and bring about unique challenges for NLP practitioners. In IndicNLP, we try to build language models, tools and techniques for NLP on Indian languages and use them to solve NLP tasks. We also try to consolidate and curate all open source work and data in NLP on Indian Languages.There are just so many things to do. And we need your help!

You can’t code for every permutation combination of rules that is possible with a Language. Artificial Intelligence is a way to learn the rules from the data and solve the problem.

Why do we need AI for this?

When something is beautiful, it is very hard to understand. ...You can think of AI as Software 2.0.

Data

We aim to collect datasets that focus on regional indian languages and keep them in the public domain for everyone forever. There are hundreds of regional language and numerous dialects in India. Availability of datasets in the open domain encourages and helps build AI applications.

Check out the data that we have open sourced so far.

Challenges

Training Data Pre-processing Data Curation Data Collection

Why Open Source?

We all love Linux. Current cloud computing or AI wouldn’t be possible with Linux being open sourced. Most of the regional languages in India doesn’t have the market size to attract investments from big corporations to build the software in a profitable way. Open source initiative is it Kick start to build the tools or software and get the wheels rolling.