Caption Quality Task Force
The Global Alliance of Speech-to-Text Captioning has launched the Caption Quality Task Force, and we need your help to build our database of captioning samples. This is a multi-year data-gathering and caption-monitoring project. The Task Force will evaluate broadcast captioning quality using the NER (number, edition error and recognition error) model. Data will be gathered across the U.S. with goals of extending that reach globally.
It will take consumers and captioners working together to protect the quality of captioning services available and to also protect the profession that provides these services. There is not one without the other. Left unattended, everyone loses.
Education and awareness, benchmark standards, and service-provider certifications are needed to achieve the goal of providing and protecting equal and effective access.
As this data is collected, it will be evaluated against the WER (Word Error Rate) and NER for accuracy. The results will be shared with the person who submits the data, and that specific information can be used for reporting complaints to the station and when filing FCC complaints. The results will be organized, stored, and available to various entities for quality orders, legislation, and public education and awareness with regard to equal and effective access accommodations.Goal:
Gather two samples of live programming from various time slots on a weekly basis to show consistency in caption quality (e.g, was quality bad due to system failure on one day, or is quality bad daily due to poor-quality service?)
Additionally, we want samples from all markets. Our findings will be posted periodically throughout the study, and reports will be made at each board meeting.
Join us and get involved
Volunteers should email the Task Force at email@example.com to participate in ongoing region and station monitoring assignments.
Click this link to upload captioning samples (Google account required for upload).
Potential data collection with the help of 25 people:
1 file month 1 file a week
25 files a month 100 files a month
300 files a year 1,200 a year
Potential data collection with the help of 75 people:1 file a month 1 file a week 2 files a week