WHITE PAPER: Multimedia in Public Safety



Contents

1. Background
2. Technology implementation
3. Automated language translation
4. Device location
5. Multimedia
6. Use cases
7. Future text messaging
8. About AGENT511 TEXTBLUE

1. Background

At the time the “voluntary initiative” to deliver text-to-911 was agreed to by the four major wireless carriers and NENA, and APCO in 2012¹, it was imagined that SMS text messaging was only an interim solution.  Native Real-time text (RTT) was thought to be around the corner and would readily replace text.  After all, RTT is true Internet based messaging that is the foundation of the IMS 3GPP specifications and Next-Generation 9-1-1 i3 standards^2,3.  That is say that SMS text not only still exists, but continues to evolve and expand.

So to do the various enhancements to SMS text that include text back (from 9-1-1), automated language translation, MMS pictures and recorded video, and new services such as mobile browser device location and streaming video.  In addition, service providers continue to add new carriers and third-parties such as IoT sensor and alarm networks such as ADT.

Text messaging is hardly dead as other transactional and messaging chat services are under exploration such as social media and internationally popular apps such as WhatsApp, WeChat, and more.  And lastly, there is an entirely new group of cloud data providers such as RapidSOS that integrate text and multimedia directly into their portal solutions, thereby complementing, and sometimes conflicting with the role of call handling (CPE) and dispatch software (CAD).

This has created an innovative and competitive environment in public safety for text, multimedia, location, and additional data services that helps to expedite accurate dispatch and improve situational awareness for first responders in order to alleviate the impact of tragedies.  This white paper explores the various technologies, use cases, and evaluates their potential.

2. Technology implementation

2.1 Text call routing

Upon receipt of the first SMS text message within a text-to-911 call, the text control center (TCC) specified in the ATIS standard J110⁴ requests the routing location from the wireless carrier.  The location and its accuracy are provided to the call taker.  The initial accuracy ranges from a few to several thousand meters and may be rebid for a more accurate dispatchable location.  The location coordinates are submitted to another FE, the Location to Service Translation server (LoST) which returns a PSAP identifier.  The LoST server is able to return instructions in order to re-route text calls to another agency in the event a PSAP unavailable.

Once the text call is routed, the PSAP is notified of the incoming text call as shown in Figure 1.  Depending on the solution, the text call is distributed via CPE equipment which will alert the next available call taker. 

Figure 1: Text call routing

2.2 Integration & technology options

2.2.1 Browser

The leading TCC vendors offer a web browser solution for receiving text messages that combines audio and visual notifications and, in some cases, browser notifications.  These platforms offer, at minimum, two-way text, location rebid, and internal and external transfers.   Platforms such as Intrado Web Text include premium features such as text back and automatic language translation as well as optional multimedia capabilities.  Browsers are secured with username and password and IP whitelisting.  Interfaces to recording and analytics service providers are also available.  Browser solutions may be deployed within an ESInet using a web proxy and/or firewall rules that permit external public IP addresses.

2.2.2 Browser and OTT 

AGENT511 offers a unique, patented solution that combines browser notifications with a parallel pseudo voice call to the dispatch center that notifies the call taker of the incoming text session and puts them in a “Not Available” status for the duration of the session.  The pseudo call includes ANI, and if it leverages next-generation IP voice, the location PIDF-LO (descriptive element).  This solution has been deployed in leading PSAP ACD environments and allows the call taker to change status to “not busy” for the duration of the call.

2.2.3 Portal providers

There are several new portal providers who have included additional call data and facilitate an ecosystem of cloud data applications.  AGENT511 text-to-911, along with optional features, are integrated into RapidSOS Portal to create a seamless workflow with single sign-on and integrated location and mapping.  Now a text call is accompanied not only by its carrier supplied location but Google Android or iPhone iOS device location and additional data sources such as Ride Share Apps like Uber.  Each location is plotted along with the accuracy in the form of a radius.  Carriers and device providers are beginning to introduce Z-Axis in the form of height and/or altitude in feet or meters.

Figure 2:  text-to-911 embedded within RapidSOS portal

2.2.4 CPE

Text may be integrated with call handling, computer aided dispatch (CAD), and recording systems.  One of the key considerations for any solution is the distribution and notification for new incoming text calls.   In order to archive text calls, PSAP’s leverage text call recording via SIPREC or web services APIs⁵.

2.3 Provisioning process

Text-to-911 services are provisioned as follows:

1. Select your integration model and kick-off the process with your browser vendor or CPE provider.
2. Submit your request for service (RFS) to the relevant wireless carriers, including, at minimum the major providers such as Verizon Wireless, AT&T, and T-Mobile.  
3. Your text provider will request provisioning via the TCC.  If you are migrating service from one TCC the other, the new TCC will coordinate the migration and testing with the current TCC.  If you are migrating service within the same TCC, the process is very seamless and a migration date is selected.  
4. Your provider will provision accounts on their platform.  If a Portal provider such as RapidSOS Portal is selected, the provider will exchange information with the Portal provider.  Once provisioned, you should be able to test the services.  IP whitelisting on the PSAP firewall is likely required.
5. Once all parties have agreed to the live cutover, the service goes live and may be verified by the PSAP and carriers.

The process likely requires 4-6 weeks; however, the wireless carriers are permitted up to 180 days to provision the PSAP.

3. Automated language translation

3.1 Text-to-911

Foreign language character support across handsets, carriers, and TCC’s may differ.  Unicode character support which covers languages such as Simplified Chinese, Korean, and Arabic, non-exclusively, is typically available.  The challenge is that most PSAP’s do not have multilingual call takers nor policies for multilingual call handling so text translation is a challenge.

There have been several industry efforts to provide live translation⁶, however, due to the volume of text calls, current business models do not sustain the service, especially across popular languages.  Text uniquely offers the ability to parse, automatically detect the language, and translate messages using automated machine translation services.  As such, some vendors are incorporating side-by-side best effort automated translation as shown in Figure 2.  

3.2 PSAP initiated use case

There is an additional use case for language translation; namely, the ability to initiate abound text messages to the wireless user in a foreign language after the initial text or voice call for service.  The initial call may have used a paid translation service such as Language Line or an on-premise language specialist.  In the event the dispatcher wishes to convey instructions to the wireless user such as “the uniformed officer is at the door,” they may do so by selecting the foreign language on the text platform and initiating two-way text conversation with side by side native and translated messages.  This enhances dispatch accuracy and reduces the time required to adequately deliver instructions or acquire information after the initial request for dispatch services.

4. Device location

4.1 Wireless carrier

Text-to-911 calls offer both carrier ROUTE and DIPSATCH location based upon a combination of tower centroid, triangulation, trilateration, and ELS.  Centroid typically represents the tower location and in the absence of a more accurate location (that may be quickly acquired) is presented to the LoST (or ECRF) server for ROUTE location.  Today, many text calls use more accurate locations that are acquired via a combination of centroid, signal strength, and GPS device location.  Accuracies may be in the several meter range and location is described a shape (such as circle, point, ellipse, sphere, or ellipsoid), coordinates, accuracy, and confidence.  Shapes are industry standard and defined by the PIDF-LO⁷.  Carrier location may be acquired only as part of the 9-1-1 call to the PSAP in accordance with the Wireless Communication Act of 1999.

4.2 Additional data portals

Portal providers such as RapidSOS have closely worked with mobile phone operating system vendors such as Google and Apple to provide more accurate device location that is provided during a 9-1-1 voice or text call.  In addition, as an example RapidSOS will provide additional information via app providers such as Uber such as the vehicle, license plate number, and driver in the event a passenger texts 9-1-1 using the app while using the service.   There are several options to display the information – within the portal and integrated with CPE – however, they provide an important service by augmenting carrier location, and in the case of RapidSOS, placing the additional data on a map.

4.3 Wireless browser device location

Another way to acquire device locatin similar to the portal vendor is via authorized sharing of browser location⁸.  This is being done by initiating a text message to the wireless user and having the user click the one-time encrypted secure link.  The user is asked to authorize sharing their location information with the PSAP, for which upon successfully authorized, will provide the call taker access to the device location.  This location will include coordinates, accuracy, and any other details such as Z-axis as supported by the mobile phone and provider.  Device location works on devices that support HTML5 features such as location sharing. 

This capability is especially valuable for text backs where 9-1-1 location is not available due to privacy and for which the user authorizing sharing.  This opt-in avoids the process of requesting a subpoena to acquire location. 

4.4. Z-Axis location

Z-Axis location is now becoming available for text as a result of FCC actions and those of the wireless carriers to deliver Z in the PIDF-LO.  Wireless carriers, starting with AT&T⁹, are beginning to deliver new shapes such as the Ellipsoid that includes a Z-axis in meters.  Calculations may be required by the browser or CPE to provide a useful altitude that can be correlated to a building height or structure for the dispatched location.

5. Multimedia

5.1 Multimedia messaging service (MMS)

The voluntary text-to-9-1-1 deployment did not require the TCC’s to provide MMS (Multimedia Messaging Service) nor SMS text back.  There has been limited carrier support for 9-1-1 pictures, audio, and video. In order to overcome this gap, an MMS message from the number 911911 may be sent to the wireless user requesting pictures and video.  Once solicited, MMS pictures and videos returned via 911911 for a configured duration and from the respective wireless user, are delivered to the PSAP.  Thereafter, multimedia may be deemed expired and blocked.  PSAP’s may determine whether media responses should be provided to the call taker, dispatcher, or an administrative media group. 

Pictures and videos are forwarded to the 9-1-1 center as a link to the multimedia file which may then be displayed in the browser or CPE.  Pictures are received as .JPEG and video typically, 3GP or mp4, depending on the handset and carrier.  Videos are typically restricted to 1 MB and less than 60 seconds.   Because the files are encoded by the carrier for delivery, the messaging content is secure.

Figure 3:  solicited MMS picture & video responses

5.2 Streaming video

Like MMS, browser-based streaming video is initiated by the PSAP via SMS text message with a secure, one-time encrypted web link.  The sending SMS number is 9-1-1 if part of an active 9-1-1 text session or 911911 if the video request is a result of an active 9-1-1 voice call or previous text session.  If there is an active 9-1-1 voice call, the voice call will take precedent over streaming video audio content.   The link will be sent to the wireless user and they will be asked for permission to share their camera and microphone.  If they accept, video and audio content will be shared with the PSAP via streaming multimedia player such as embedded browser in the PSAP.  The dispatched can share the video with supervisors and first responders by simply sharing the web link via mobile data or SMS.  The wireless user will not have access to the PSAP’s camera or audio via streaming video.  Once the PSAP has determined the video session is over, the link will become inactive and the conversation is archived and an audiovisual file is available to PSAP administrators.

Figure 4:  process to initiate video session with user

There are a number of objections to providing streaming video to already overburdened call takers, however, there are several relevant, important use cases which are accessible without disturbing call takers and intelligently creating workflows filter and route traffic to trained personnel.

6. Use cases

6.1. Fire / Rescue Incident

A wireless user sees fire or incident requiring rescue and calls or texts 9-1-1.  The call taker acquires the available details and location and begins dispatching fire and rescue.  In order to acquire information about the magnitude of the response, the dispatcher asks the user to share video and sends the link for which the user clicks and shares their camera.  Streaming video shows the dispatcher that the fire is quickly growing and additional alarms are required.  A link to the live video is shared via mobile data terminal (MDT) with first responders.   Therein, a Fire Captain on route, plans the response 5 minutes prior to arriving on-scene.

Video may be especially useful for accidents where police have not arrived on scene and the magnitude of the event may be useful for dispatch.

6.2 Medical assistance

A wireless user calls 9-1-1 about a hiker who has fallen with a major leg break and bleeding.  The call taker dispatches rescue and emergency medical service.  They send a link to the wireless user who clicks the link and shares device location and low-quality video.  The location information shrinks the dispatch area and the video is focused exclusively on the hiker’s leg.  An MMS picture is also provided.  A trained EMS dispatcher is provided the video link reviews the video and picture and provides some pre-arrival instructions to the wireless user.  The link is shared with paramedics on route to better prepare their response as assist the dispatcher with any additional instructions.

6.3 Landmark

An injured skier dials 9-1-1 from the trees.  They are stuck and need assistance but cannot provide accurate location.  A video link is sent to the wireless user by the dispatcher, the user clicks the link, and shares their location and camera.  The dispatcher conferences a supervisor familiar with the slopes and identifies the run and a nearby lift and dispatches rescue services and notifies the ski patrol.

6.4 Active shooter

A student observes an active shooter in process and is able to text 9-1-1.  The dispatcher responds and sends a link via text to the user’s phone.  The wireless user clicks to share their camera and the dispatcher provides the link to a command vehicle.  An officer uses the video to ascertain additional details about the location of shooter and students.    The information, along with CATV footage, and other calls provides advanced situational awareness to lead officers on the scene.

6.5 Flash Mob

A bystander is a witness to a flash mob robbery at a luxury store in the mall.  The wireless user hides, calls 9-1-1, and receives a link to share their precise location and shares video of the robbery in progress.  The dispatcher is able to see store names and provide landmark information to first responders along with descriptions of the robbers and weapons.  

6.6 Mental health

A student is experiencing depression and contacts a school mental health center.  The counselor creates an instant video call on demand by initiating a text to the student with a link to the video session.   Both parties are able to access two-way audio and video to create a connectiveness and the counselor is able to assess the student’s condition and either contacts public safety to dispatch emergency services or sets an in-person appointment for later in the day.

7. Future text messaging

Naturally, the conversation of text messaging leads to what’s next?  SMS text messaging is in itself a specific technology and especially outside North America and in international airports and tourist destinations, social and Internet communications apps such as WhatsApp, WeChat, and Messenger are gaining popularity.  These apps provide interfaces, sometimes via third-parties which provide access to messaging, multimedia, and location.  In additional to universal foreign character handling, some apps offer additional “presence” capabilities, including real-time bubbles when the other party is speaking.  This capability delivers a text experience that is less transactional and more synonymous with the experience of a voice call.

This leads us to Real-Time Text (RTT) which is viewed as the future of messaging for Next-Generation 9-1-1.  RTT calls are initiated on a carrier (native) mobile client to the number 9-1-1 and may be accompanied by one or two-way audio as shown in Figure 4.  RTT could also be invoked as a result of a voice call (from an RTT client).  There are numerous use cases that are encapsulated by RTT that reflect the needs of the hearing and speech impaired communities along with wireless users without hearing and speech impairments.

Figure 5: (a) RTT Verizon dialing app; (b) RTT keyword and screen

RTT is encapsulated in industry standards such as 3GPP IMS and NENA i3.  RTT calls include device location and are routed via Next-Generation infrastructure over the ESInet.   Early standards called for RTT to be terminated by the PSAP using TTY, however, practically, most PSAP’s are likely to opt for Internet-based solutions like their Next-Generation CPE¹⁰.  Some have expressed the desire to use their SMS text browser solutions with the TCC, however, the complementary 9-1-1 voice call creates a complication that requires further investigation.

8. About AGENT511 TEXTBLUE

AGENT511 TEXTBLUE platform delivers seamless text and multimedia workflows for the largest (and smallest) public safety agencies.  It is integrated with the leading text control centers (TCC) as well as commercial and private carrier SMS/MMS multimedia gateways for US and International markets.  TEXTBLUE incorporates Next-Generation 9-1-1 and modern web interfaces to deliver communications to call handling, CAD, and recorders.

In addition, the platform is coupled with a number of differentiating features such as patented¹¹ parallel voice dialing to ensure call takers never miss a call as well as text back, language translation, natural language triage, and streaming video.  The platform is embedded in RapidSOS Portal for seamless call handling and dispatch.

1 https://www.fcc.gov/document/fcc-adopts-text-911-rules

2 https://cdn.ymaws.com/www.nena.org/resource/resmgr/standards/nena-sta-010.3b-2021_i3_stan.pdf

3 https://datatracker.ietf.org/doc/html/rfc5194

4 https://www.techstreet.com/standards/atis-j-std-110-01-v002?product_id=1899053

5 https://tools.ietf.org/html/rfc7866

6 https://www.police1.com/police-products/police-technology/software/cad/articles/techfest-event-looks-to-the-future-of-translation-for-text-to-911-k3Td3fT7SCzjOLgR/

7 https://datatracker.ietf.org/doc/html/rfc5985

8 https://www.w3schools.com/html/html5_geolocation.asp

9 https://www.engadget.com/911-vertical-location-fcc-025655574.html

10 https://cdn.ymaws.com/www.nena.org/resource/resmgr/standards/nena-inf-042.1-2021_rtt_appv.pdf

11 US Patent 9386407B2

Copyright © 2022, Good Egg Media LLC DBA AGENT511 

Northbrook, Illinois, USA. All rights reserved. This document is the property of AGENT511. It contains material distributed under license agreement. Reproduction or use other than as provided for in the license agreement is prohibited. 

Information contained herein consists of AGENT511 copyrighted material and, as indicated, copyrighted material furnished by AGENT511’s third party suppliers, which is used herein by permission of said suppliers. 

Published by AGENT511, 425 Huehl Road, Suite 11B, Northbrook, Illinois 60062.