The Measuring Broadband America mobile measurement effort uses much of the existing fixed measurement infrastructure to test the performance of volunteers' Android and/or iPhone smartphone mobile broadband services. Most recent technical descriptions of the methodology and data processing approaches for fixed measurement efforts are available as part of the Measuring Broadband America Program.
In order to protect the privacy of volunteers participating in the mobile broadband performance measuring effort, the data collection approach, nature of data collected, and the data post-processing differ in significant ways from the fixed effort. Broadband performance data is collected via an Application running on a volunteer's Smartphone to measure the upload and download speed, latency of connections, packet loss and jitter, as well as, the wireless performance characteristics of the broadband connection and the kind of handsets and versions of operation systems tested. The application does not collect any unique or personal identifiers that could be directly linked to a particular handset. Collected data is uploaded to the cloud and then is available for post-processing, including a technical privacy analysis for any publicly released dataset.
Detailed technical descriptions of the kinds of data collected, sources of the data, and software data structures are available in the Mobile Data Dictionary.
Researchers and developers interested in the program are welcome at regular meetings and are encouraged to review the record of recent meetings by referencing the General Docket No. 12-264 using the FCC's Electronic Comment Filing System (ECFS http://apps.fcc.gov/ecfs), or past meetings beginning in 2010 by searching ECFS in related proceedings CG Docket No. 09-158, CC Docket No. 98-170, and WC Docket No. 04-36. For other inquires, contact Rajender Razdan.
Frequently Asked Questions
What is the Mobile Testing Architecture?
The smartphone measurement application is designed to be installed on a user's smartphone either directly or via an app store, such as Google Play or iTunes. The application is free to download, but carrier charges may apply. The application will run continuously in the background, periodically performing measurements. This automated functionality is not available on iOS
How do I remove the application?
End users may uninstall the application at any time via their smartphone's normal application uninstallation procedure.
How does the app determine the best measurement server?
When starting a measurement cycle, the application runs a brief latency test to measurement servers in the application's configuration. The nearest measurement server with the lowest round-trip latency is selected as the target for all subsequent measurements (throughput, latency, packet loss and jitter).
Will it disrupt my using my phone, e.g. generate cross-traffic?
Prior to each measurement, the Application checks the volunteer's mobile broadband use, (e.g. cross-traffic within the local smartphone). The app will measure cross-traffic which will then be included in environmental data reported in the test results. There is a "test when idle" setting which prevents scheduled tests running when the phone is actively in use.
How does the app capture location, carrier and network data?
The approximate location of the phone is determined by using the smartphone's own internal capabilities. These typically use a combination of nearby cell towers, nearby WiFi access points and GPS (if available).
The carrier and other network characteristics, such as bearer, signal strength data, etc. are recorded from APIs available from the Android Operation System and iOS (though less information is available on the iOS). These metrics are recorded before and after every active measurement.
How does the app communicate data?
All communications between the software application and the Data Collection Service on the backend hosted infrastructure are initiated by the application and securely encrypted over SSL. The software application communicates with the measurement servers over a variety of TCP and UDP ports. ICMP is used to determine the server with the lowest round-trip latency.
How does the app know to configure and update tests?
For Android users the app will frequently download, when opened, updates from a central server to its configuration to maintain the most current schedule of automated tests that it should execute. For iOS users the app will update as long as automatic updates are enabled.
How does the app deal with my privacy?
What tests does the app run?
Active Performance Test Metrics - All active tests designed and developed for the program by the FCC's contractor (www.samknows.com), in collaboration with industry, academia and other stakeholders, adhere to relevant RFCs and are publicly disclosed as part of the open methodology. All times are measured in microseconds and all speeds in bytes per second. The tests include:
UDP Latency, Packet Loss and Jitter
This test uses a fixed-rate stream of UDP traffic, running between client and a test node. A bi-directional 64 kbps stream is used with the same characteristics and properties (i.e., packet sizes, delays and bitrate) as the G.711 codec.
The client initiates the connection, thus overcoming NAT issues, and informs the server of the rate and characteristics that it would like to receive the return stream with.
The standard configuration uses 500 packets upstream and 500 packets downstream.
The client records the number of packets it sent and received (thus providing a packet loss rate) and the jitter observed for packets it received from the server. The server does teh same but with the reverse traffic flow, thus providing bi-directional packet loss and jitter.
Jittere is calculated using the PDV approach described in section 4.2 of RFC5481. The 99th percentile will be recorded and used in all calculations when deriving the PDV.
Measures the download and upload speed of the given connection in bits per second by performing multi-connection GET and POST HTTP requests to a target test node. Binary non-zero content, herein referred to as the payload, is hosted on a web server on the target test node. The test operates for either a fixed duration (in seconds) or a fixed volume (in MB). It can also report the recorded average throughput at multiple intervals during the test (e.g. once every 5 seconds). The client will attempt to download as much of the payload as possible for the duration of the test. The payload and all other testing parameters are configurable and may be subject to change in the future.
Four separate tests are supported:
- Single connection GET
- Multi connection GET
- Single connection POST
- Multi connection POST
How many threads do the tests execute?
Tests are executed using three concurrent connections. Each connection used in the test counts the numbers of bytes of the target payload, transferred between two points in time, and calculates the speed of each thread as Bytes transferred/Time (seconds).
Factors such as TCP slow start and congestion are taken into account by repeatedly downloading small chunks (default 256KB) of the target payload before the real testing begins. This "warm up" period will complete when three consecutive chunks are downloaded at the same speed (or within a small tolerance -- default is 10% -- of one another). In a multi connection test, three individual connections are established (each on its own thread) and are confirmed as all having completed the warm up period before timing begins.
Content downloaded is copied to /dev/null or equivalent (i.e. it is discarded). Payload content for upload tests is generated and streamed on the fly from /dev/urandom (or equivalent).
The following is an example of the calculation performed for a multi connection test using three concurrent connections.
- S = Speed (Bytes per second)
- B = Bytes (Bytes transferred)
- C = Time (Seconds) (between start time point and end time point)
- S1 = B1 / C1 (speed for Thread 1 calculation)
- S2 = B2 / C2 (speed for Thread 2 calculation)
- S3 = B3 / C3 (speed for Thread 3 calculation)
- Speed = S1 + S2 + S3
- Example values from a 3MB payload:
- B1 = 3077360 C1 = 15.583963
- B2 = 2426200 C2 = 15.535768
- B3 = 2502120 C3 = 15.536826
- S1 = B1/C1 = 197469.668017
- S2 = B2/C2 = 156168.655454
- S3 = B3/C3 = 161044.475879
- S1 + S2 + S3 = Total Throughput of the line = 197469.668017 + 156168.655454 + 161044.475879 = 514682 (Bps) * 0.000008 (Mbps/Bps) = 4.12 Mbps
What data is provided for the aspects of my broadband performance specific to the wireless radio connectivity provided by my carrier?
Availability of information about the wireless network will vary by operator, device support (smartphone/dongle) and territory. What is available under optimum conditions is detailed below:
- Location inferred from cell tower triangulation, WiFi triangulation, or GPS
- Is the device connected to a data network?
- Active cell tower ID and signal strength (RSSI)
- Visible neighboring cell towers, including cell IDs and RSSI
- Active network operator ID ("MNC") and name
- Active network country code ("MCC") and name
- SIM's network operator ID and name
- SIM's network country code and name
- Bearer (CDMA, GPRS, EDGE, all of the 3G variants, LTE, etc)
- Phone type (GSM, CDMA)
- Is the device in a roaming state?
How often do tests run? What's the app test schedule?
The Android application will periodically download a predefined test schedule randomized to execute tests. This functionality is not available on iOS devices. For Android users, upon opening the app, the app tests will be scheduled to occur up to three times in a 24 hour period lasting a maximum of 8 seconds per test. There is no fixed timings for these tests. Data usage for these scheduled tests will vary depending on the connection (3G/4G/5G) but will not exceed the limit set under the cellular data limit.
How much data will the program use by default?
By default the app will limit the total monthly data traffic used for execution of scheduled tests to a maximum of 600 MB. Volunteers can adjust the data cap to suit their preference. The higher the configured data cap the more frequently the app will schedule automated tests to execute. Volunteers will also be able to set the date of the month when data caps should reset to align app usage with their mobile broadband service billing.