Caption Quality Task Force
The Global Alliance of Speech-to-Text Captioning has launched the Caption Quality Task Force, and we need your help to build our database of captioning samples. This is a multi-year data-gathering and caption-monitoring project. The Task Force will evaluate broadcast captioning quality using the NER (number, edition error and recognition error) model. Data will initially be gathered via local sources across the U.S. with goals of extending that reach globally.
It will take consumers and captioners working together to protect the quality of captioning services available and to also protect the profession that provides these services. There is not one without the other. Left unattended, everyone loses.
Education and awareness, benchmark standards, and service-provider certifications are needed to achieve the goal of providing and protecting equal and effective access.
As this data is collected, it will be evaluated against the WER and NER for accuracy. The results will be shared with the person who submits the data, and that specific information can be used for reporting complaints to the station and when filing FCC complaints. The results will be organized, stored, and available to various entities for quality orders, legislation, and public education and awareness with regard to equal and effective access accommodations.
The goal is to gather two files a week from each station in the top 25 markets, with samples taken at different times. This gives us the ability to show the consistency in caption quality (e.g, was quality bad due to system failure on one day, or is quality bad daily due to poor-quality service?) Additionally, we want samples from all markets. Our findings will be posted periodically throughout the study, and reports will be made at each board meeting.
Join us and get involved
Volunteers should email the Task Force at firstname.lastname@example.org to participate in ongoing region and station monitoring assignments.
Click this link to upload captioning samples (Google account required for upload).
U.S. Based 25 Top Markets, one network in each city (25-member task force)
1 file month 1 file a week
25 files a month 100 files a month
300 files a year 1,200 a year
25 Top Markets, three main networks in each city (75-member task force)
1 file a month 1 file a week 2 files a week
75 files a month 300 files a month 600 files a month
900 a year 3.600 a year 7,200 files a year