Log in

Caption Quality Task Force 

The Global Alliance of Speech-to-Text Captioning has launched the Caption Quality Task Force, and we need your help to build our database of captioning samples. This is a multi-year data-gathering and caption-monitoring project. The Task Force will evaluate broadcast captioning quality using the NER (number, edition error and recognition error) model. Data will initially be gathered via local sources across the U.S. with goals of extending that reach globally.

It will take consumers and captioners working together to protect the quality of captioning services available and to also protect the profession that provides these services. There is not one without the other. Left unattended, everyone loses.

Education and awareness, benchmark standards, and service-provider certifications are needed to achieve the goal of providing and protecting equal and effective access.

As this data is collected, it will be evaluated against the WER and NER for accuracy. The results will be shared with the person who submits the data, and that specific information can be used for reporting complaints to the station and when filing FCC complaints. The results will be organized, stored, and available to various entities for quality orders, legislation, and public education and awareness with regard to equal and effective access accommodations.

The goal is to gather two files a week from each station in the top 25 markets, with samples taken at different times. This gives us the ability to show the consistency in caption quality (e.g, was quality bad due to system failure on one day, or is quality bad daily due to poor-quality service?) Additionally, we want samples from all markets. Our findings will be posted periodically throughout the study, and reports will be made at each board meeting.

Join us and get involved 

Volunteers should email the Task Force at taskforce@speechtotextcaptioning.org to participate in ongoing region and station monitoring assignments.

Click this link to upload captioning samples (Google account required for upload).

  1. When recording a sample, please make note of the following:
    • Record 2-5 minutes of live captioned video with corresponding audio
    • *Note: If captions are unusable, please submit with or without sound*
    •  In video, include station identification, time, and any logos
 2. Form will require the following information:
    • Date/Time of video/photo 
    • Network (ABC, CBS, FOX, NBC, etc)
    • Station City
    • Source of sample (Dish, Facebook, Hulu, network website, etc)

Data-gathering goals:

U.S. Based 25 Top Markets, one network in each city (25-member task force)

1 file month 1 file a week
25 files a month 100 files a month
300 files a year 1,200 a year

25 Top Markets, three main networks in each city (75-member task force)

1 file a month 1 file a week 2 files a week
75 files a month 300 files a month 600 files a month
900 a year 3.600 a year 7,200 files a year

Join Our Email List

Contact Us

455 Massachusetts Ave. NW
Suite 123
Washington, DC, USA 20001
(604) 639-3924

Follow Us

      Global Alliance of Speech-to-Text Captioning is a 501( c )(6) tax exempt non-profit organization registered in the District of Columbia. | © 2019-2023 Global Alliance . Al rights reserved.

Powered by Wild Apricot Membership Software