Home
Bridging India's language barrier by AI-driven multilingual transcreation of videos
Chitralekha is an open-source video transcreation platform for video subtitling and voice-over generation across various Indic languages, using ML model support (ASR for Transcription, NMT for Translation and TTS for voice-over)
Chitralekha offers support for editing YouTube videos and mp3 audio files, transcription generation process (Ex: Models, YouTube Original Source captions, Custom subtitle files and manually entered subtitles), translation generation process (ex: model, manually entered subtitles) and voice-over generation process (manually recorded and TTS model)
In the current world, there are numerous informative videos available online. Mostly they are associated with very few languages. The usefulness of the content can be increased by creating sub-titles and voice-overs for these across various Indic languages. With millions of hours of video contents, it becomes harder to manually create the multi-lingual sub-titles and voice-over. This is where Chitraleka comes to the rescue.
The existing state-of-the-art ASR, Translation and TTS ML models can power the Chitralekha tool, and provide the platform for Transcriptionists/Translators/voice-over recording persons to create multi-lingual sub-titles and voice-over at scale with high accuracy.
- Support all possible video sources and languages
- Build a reliable & scalable platform beneath Chitralekha
- Keep the UI simple and intuitive
Chitralekha supports importing videos and optional subtitles from YouTube. It also enables export of the subtitles in standard formats which can be used to update videos on YouTube.
Chitralekha supports translating the transcription into English and 12 Indian languages supported by IndicTrans model. Eventually it would be a Plug & Play feature.
Chitralekha supports transcribing the input video with IndicASR for English and 9 Indian languages. This automatically creates timestamped transcription cards which can be edited. Eventually it would be a Plug & Play feature.
The transcription output can optionally further be paraphrased to be grammatically correct sentences if proper documentation of the content is required. The paraphrased transcript can then be translated.
Chitralekha supports editing the transcriptions both in the source and target languages in Roman characters with IndicXlit support.
Chitralekha supports editing of voice-over both in translation language.
In case of an API fail for a Machine Generated source type Transcription or Translation task, when status shows as ‘processing failed’, the Org Owner or Project Manager can regenerate the response by clicking on the unlock icon on the right of the task row.
Org Owners can clone multiple projects from one project template and thereby create up to a maximum of 20 projects at once. Go to Projects under Organisation – Click on Create Bulk Projects from Templates – enter the number of projects to be created – select an existing project from the drop-down menu as template for the project settings to be cloned – enter the titles of the projects to be created – click on create project.
When there are multiple speakers in a video, Chitralekha offers a speaker tagging feature. When adding a new video under a project, select ‘Multiple Speakers’ and enter the speakers’ details in the given section before ‘creating’ the video. When transcription is then auto generated, the various speakers will be tagged as per given details.
Various reports are available for viewing depending on the role of the user – Task Report, Project Report, User Report and Language Report. (Refer to 'Reports and Dashboard' for more details). It is also possible for users depending on their role to enable the option for various E-mail reports to be sent via E-mail (Please refer to the section, ‘Email Reports’ for further information).
Apart from uploading videos individually, the project managers have the option to also upload videos in bulk and there create multiple tasks at once. When a project is selected, under the ‘Videos’ section, next to the ‘Create a New Video/Audio’ tab, there is a second tab, ‘Bulk Video Upload’. Clicking on this will allow the user to upload a csv file with details of the videos and tasks to be created. Click on ‘info’ next to ‘Bulk Video Upload’ to view a sample of a csv file. Use this sample as a template to create the csv to be uploaded. (For more details on bulk video uploads, please refer to the section, ‘Bulk Video Upload’).
A glossary feature is available in Chitralekha which maybe useful during time of translation. While doing translation tasks, in cases where the user would like a certain word to be translated/transliterated in a specific way: for example, the English word 'Python' to remain as 'Python' and not be translated to the local language across the video the user may add this correction in the glossary. This correction could be applied across the video. It will also be possible for users to view the glossary.
The newly added glossary will be used to automatically replace words in newly created Translation Edit tasks.
It is possible for the Organisation Owners to upload a list of glossary in bulk to be applied across tasks.
If the user wishes to download the final Voice Over output as an integrated (audio and video) file, the option to ‘Enable Video Integration’ should be checked in Project Settings.
To check what tasks are currently being processed, the task queue status icon (sand timer) on the top right of the page, will show the list of tasks that are in queue for processing.
Once a translation task is completed, if the need arises to reopen the task to make further edits, the ‘Reopen’ icon on the right side of the task row maybe clicked. Please note that when a translation task is reopened, the corresponding voice over task will be deleted.
The merge icon visible on top of each card in transcription and translation tasks can be used to merge the current card with the next card to form a single card. The time duration will also be merged.
The split icon visible on top of the transcription task screen next to the info icon, will allow users to split the contents of a card into two cards. Place the cursor next to the word after which the card is to be split, and click on the icon. The time duration will also be reset to match this splitting.
Checking the transliteration box under settings in the translation tasks page will allow the users to type a word in English script in the translation card and choose a corresponding word in the target language of the task from a drop down menu.
Enabling the Superscript/Subscript option under settings in the transcription/translation tasks page will permit the users to convert numbers to superscript or subscript in equations where necessary.
The find and replace icon on top of the transcription and translation task page will allow users to find a word from the task by entering the word in the find text box and replacing it with any new word that can be entered in the replace text box.
git clone --recurse-submodules https://github.com/AI4Bharat/Chitralekha
Any information/help/discussion required, can be taken up using the following link :
https://github.com/AI4Bharat/Chitralekha-App/discussions