Menu
Log in
Log in

Caption Quality Task Force 

The Global Alliance of Speech-to-Text Captioning has launched the Caption Quality Task Force, and we need your help to build our database of captioning samples. This is a multi-year data-gathering and caption-monitoring project. The Task Force will evaluate broadcast captioning quality using the NER (number, edition error and recognition error) model. Data will be gathered across the U.S. with goals of extending that reach globally.

It will take consumers and captioners working together to protect the quality of captioning services available and to also protect the profession that provides these services. There is not one without the other. Left unattended, everyone loses.

Education and awareness, benchmark standards, and service-provider certifications are needed to achieve the goal of providing and protecting equal and effective access.

As this data is collected, it will be evaluated against the WER (Word Error Rate) and NER for accuracy. The results will be shared with the person who submits the data, and that specific information can be used for reporting complaints to the station and when filing FCC complaints. The results will be organized, stored, and available to various entities for quality orders, legislation, and public education and awareness with regard to equal and effective access accommodations.

Goal: 

Gather two samples of live programming from various time slots on a weekly basis to show consistency in caption quality (e.g, was quality bad due to system failure on one day, or is quality bad daily due to poor-quality service?)

Additionally, we want samples from all markets. Our findings will be posted periodically throughout the study, and reports will be made at each board meeting.


Join us and get involved 

Volunteers should email the Task Force at taskforce@speechtotextcaptioning.org to participate in ongoing region and station monitoring assignments.

Click this link to upload captioning samples (Google account required for upload).

  1. When recording a sample, please make note of the following:
    • Record 2-5 minutes of live captioned video with corresponding audio
    • *Note: If captions are unusable, please submit with or without sound*
    •  Include entire video frame (depicting any logos, time, station ID, etc)
 2. Form will require the following information:
    • Date/Time of video/photo 
    • Network (ABC, CBS, FOX, NBC, MyNet, etc)
    • Station City
    • Source of sample (Dish, Facebook, Hulu, network website, etc)

Data-gathering goals:

Potential data collection with the help of 25 people:

1 file month 1 file a week
25 files a month 100 files a month
300 files a year 1,200 a year

Potential data collection with the help of 75 people:

1 file a month 1 file a week 2 files a week
75 files a month 300 files a month 600 files a month
900 a year 3.600 a year 7,200 files a year

Join Our Email List


Contact Us

455 Massachusetts Ave. NW
Suite 123
Washington, DC, USA 20001
(604) 639-3924
info@speechtotextcaptioning.org

Follow Us


      Global Alliance of Speech-to-Text Captioning is a 501( c )(6) tax exempt not-for-profit organization registered in the District of Columbia. | © 2019-2023 Global Alliance . Al rights reserved.

Powered by Wild Apricot Membership Software