Update model

README.md

@@ -4,10 +4,19 @@ license: cc-by-4.0
 # Whisper-Tiny-hindi
 
 This is a fine-tuned version of [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny), fine-tuned on the following datasets:
-- [Shrutilipi](https://ai4bharat.iitm.ac.in/datasets/shrutilipi) (AI4Bharat): Shrutilipi is a labelled ASR corpus obtained by mining parallel audio and text pairs at the document scale from All India Radio news bulletins for 12 Indian languages - Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Sanskrit, Tamil, Telugu, Urdu. The corpus has over 6400 hours of data across all languages. Out of which hindi is ~ 1600 hours
-- [IITM Madras SpringLab](https://asr.iitm.ac.in/dataset) (CC BY 4.0 License): This data was collected on payment basis using the following vendors -- Mediscribe India, Desicrew, and Crescendo. Preliminary checking of quality of transcriptions was done by our partners at KL University as well as by SPRING Lab members. The data consists mostly of mock conversations as well as monolgues on different topics.
-
-The model is trained on around 2500 hours of hindi speech & optimized for ASR tasks in hindi, with a particular focus on high-accuracy transcription.
+| Dataset                                | Hours (Hi) | License                           | Source                                                                 |
+|----------------------------------------|------------|-----------------------------------|------------------------------------------------------------------------|
+| **Shrutilipi**                         | ~1,558 h   | CC BY 4.0                         | [ai4bharat/shrutilipi](https://huggingface.co/datasets/ai4bharat/Shrutilipi)                                       |
+| **IITM Madras SpringLab**              | ~900 h     | CC BY 4.0                         | [SpringLab](https://asr.iitm.ac.in/dataset)                            |
+| **Common Voice 11.0 (Mozilla)**        | ~20 h      | CC 0 1.0 (public domain)          | [mozilla/commonvoice](https://huggingface.co/datasets/mozilla-foundation/common_voice_11_0)          |
+| **IndicSUPERB**                        | 150 h      | Apache License 2.0                | [ai4bharat/indic-superb](https://github.com/AI4Bharat/IndicSUPERB)                           |
+| **snow-mountain**                      | 67.6 h     | CC BY-SA 4.0                      | [bridgeconn/snow-mountain](https://huggingface.co/datasets/bridgeconn/snow-mountain/)        |
+| **yodas**                              | ~200 h     | CC BY 3.0                         | [espnet/yodas](https://huggingface.co/datasets/espnet/yodas)                      |
+| **IndicVoices-R_Hindi**                | 75 h       | CC BY 4.0                         | [SPRINGLab/IndicVoices-R_Hindi](https://huggingface.co/datasets/SPRINGLab/IndicVoices-R_Hindi)        |
+| **Lahaja**                             | 12.5 h     | CC BY 4.0                         | [ai4bharat/lahaja](https://ai4bharat.iitm.ac.in/datasets/lahaja)    |
+| **fleurs**                             | 30.0 h     | CC BY 4.0                         | [google/fleurs](https://huggingface.co/datasets/google/fleurs)      |
+
+The model is trained on around 3000 hours of hindi speech & optimized for ASR tasks in hindi, with a particular focus on high-accuracy transcription.
 
 ## How to use
 The Whisper model is intrinsically designed to work on audio samples of up to 30s in duration. However, by using a chunking algorithm, it can be used to transcribe audio samples of up to arbitrary length. This is possible through Transformers pipeline method. Chunking is enabled by setting chunk_length_s=30 when instantiating the pipeline. With chunking enabled, the pipeline can be run with batched inference. It can also be extended to predict sequence level timestamps by passing return_timestamps=True:
@@ -28,8 +37,8 @@ The Whisper model is intrinsically designed to work on audio samples of up to 30
 
 >>> ds = load_dataset("mozilla-foundation/common_voice_11_0", "hi", split="validation")
 >>> sample = ds[0]["audio"]
->>> prediction = asr_pipe(sample.copy(), batch_size=8, return_timestamps=True)["text"]
-हमने उस उम्मीदवार को चुना।
+>>> prediction = asr_pipe(sample.copy(), return_timestamps=True)
+{'text': ' हमने उस उम्मीदवार को चुना', 'chunks': [{'timestamp': (0.0, 4.42), 'text': ' हमने उस उम्मीदवार को चुना'}]}
 ```
 
 ## Intended Use
@@ -43,29 +52,29 @@ The Whisper model is intrinsically designed to work on audio samples of up to 30
 ### Model Performance
 Whisper Normalization is counter-productive for hindi since it takes the meaning out of a sentence for e.g. consider the hindi phrase:
 ```
-हमने उस उम्मीदवार को चुना।
+'क्षेत्रफल बढ़ने से उत्पादन बढ़ा।'
 ```
 
 After whisper normalization:
 ```
-हमन उस उमम दव र क चन
+'कषतरफल बढन स उतप दन बढ'
 ```
 
 So, we use [indic-normalization](https://github.com/anoopkunchukuttan/indic_nlp_library/blob/4cead0ae6c78fe9a19a51ef679f586206df9c476/indicnlp/normalize/indic_normalize.py#L325) for evaluation. Indic-norm produces the below output:
 ```
-हमने उस उम्मीदवार को चुना।
+'क्षेत्रफल बढ़ने से उत्पादन बढ़ा।'
 ```
 
 `openai-whisper/tiny` baseline results on `google/fleurs -- hindi`:
 ```
-Word Error Rate (WER) with whisper norm: 172.60 % 
+Word Error Rate (WER) with whisper norm: 172.60  % 
 Word Error Rate (WER) with indic norm: 196.57 % 
 ```
 
 The model achieves the following benchmarks on the held out test set `google/fleurs -- hindi`:
 ```
-Word Error Rate (WER) with whisper norm: 14.21 % 
-Word Error Rate (WER) with indic norm: 22.15 % 
+Word Error Rate (WER) with whisper norm: 10.10 % 
+Word Error Rate (WER) with indic norm: 18.94 % 
 ```
 
 Indic normalization retains diacritics and complex characters in Hindi text, which can increase the Word Error Rate (WER) when compared to Whisper's default normalization but produces more semantically accurate transcriptions.

generation_config.json

@@ -134,7 +134,7 @@
     "<|yo|>": 50325,
     "<|zh|>": 50260
   },
-  "language": "hindi",
+  "language": "hi",
   "max_initial_timestamp_index": 50,
   "max_length": 448,
   "no_timestamps_token_id": 50363,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cc0fed0f51094d792cc0ec1b7bceeb0338ba8f67be6d930031de2e97fe01fe07
+oid sha256:69a5b462d4d16d023a96334bdc27f4dfcadaa6371f1fd99807f185c7e1ed6291
 size 151061672

tiny.pt → pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0997e4eaa33849bd7bd360e149adf025d2f1a805d9276ef7d398df82597b4d9f
-size 151083171
+oid sha256:af01ddad36a7559387ea79f40a3657669a17b5663bf968d0b7b95e975345a621
+size 151099494