The Measuring Broadband America mobile measurement effort uses much of the existing fixed measurement infrastructure to test the performance of volunteers' Android and/or iPhone smartphone mobile broadband services. Most recent technical descriptions of the methodology and data processing approaches for fixed measurement efforts are available as part of the February 2013 Report.
In order to protect the privacy of volunteers participating in the mobile broadband performance measuring effort, the data collection approach, nature of data collected, and the data post-processing differ in significant ways from the fixed effort. Broadband performance data is collected via an Application running on a volunteer's Smartphone to measure the upload and download speed, latency of connections and packet loss, as well as, the wireless performance characteristics of the broadband connection and the kind of handsets and versions of operation systems tested. The application does not collect any unique or personal identifiers that could be directly linked to a particular handset. Collected data is uploaded to the cloud and the available for post-processing, including a technical privacy analysis for any publicly released dataset.
Detailed technical descriptions of the kinds of data collected, sources of the data, and software data structures are available in the Mobile Data Dictionary.
The source code of the application is made available under the GPL2 Open Source license on the FCC GitHub repository
Researchers and developers interested in the program are welcome at regular meetings and are encouraged to review the record of recent meetings by referencing the General Docket No. 12-264 using the FCC's Electronic Comment Filing System (ECFS http://apps.fcc.gov/ecfs), or past meetings beginning in 2010 by searching ECFS in related proceedings CG Docket No. 09-158, CC Docket No. 98-170, and WC Docket No. 04-36. For other inquires, contact James Miller.
Frequently Asked Questions
What is the Mobile Testing Architecture?
The smartphone measurement application is designed to be installed on a user's smartphone either directly or via an app store, such as Google Play or iTunes. The application is free to download, but carrier charges may apply. The application will run continuously in the background, periodically performing measurements.
How do I remove the application?
End users may uninstall the application at any time via their smartphone's normal application uninstallation procedure.
How does the app determine the best measurement server?
When starting a measurement cycle, the application runs a brief latency test to measurement servers in the application's configuration. The nearest measurement server with the lowest round-trip latency is selected as the target for all subsequent measurements (throughput, latency and packet loss).
Will it disrupt my using my phone, e.g. generate cross-traffic?
Prior to each measurement, the Application checks the volunteer's mobile broadband use, (e.g. cross-traffic within the local smartphone) .No tests will be executed if the device is transferring more than 64kbit/s at the time a test is scheduled to execute. Tests that are skipped are rescheduled to execute at a later time.
How does the app capture location, carrier and network data?
The approximate location of the phone is determined by using the smartphone's own internal capabilities. These typically use a combination of nearby cell towers, nearby WiFi access points and GPS (if available).
The carrier and other network characteristics, such as bearer, signal strength data, etc. are recorded from APIs available from the Android Operation System. These metrics are recorded alongside every active measurement or, in some cases, separately.
How does the app communicate data?
All communications between the software application and the Data Collection Service on the backend hosted infrastructure are initiated by the application and securely encrypted over SSL. The software application communicates with the measurement servers over a variety of TCP and UDP ports. ICMP is used to determine the server with the lowest round-trip latency.
How does the app know to configure and update tests?
The smartphone application software does not automatically upgrade itself. However, it will frequently download from a central server updates to its configuration to maintain the most current schedule of automated tests that it should execute..
How does the app deal with my privacy?
What tests does the app run?
Active Performance Test Metrics - All active tests designed and developed for the program by the FCC's contractor (www.samknows.com), in collaboration with industry, academia and other stakeholders, adhere to relevant RFCs and are publicly disclosed as part of the open methodology. All times are measured in microseconds and all speeds in bytes per second. The tests include:
UDP Latency and Packet Loss
Measures the round trip time of small UDP packets between the application and a target test node. Each packet contains consists of an 8-byte sequence number and an 8-byte timestamp. If a packet is not received back within three seconds of sending, it is treated as lost. The test records the number of packets sent each hour, the average round trip time of these and the total number of packets lost. The test will use the 99th percentile when calculating the summarized minimum, maximum and average results.
As with the availability test, the UDP latency and packet loss test operates continuously in the background. It is configured to randomly distribute the sending of the echo requests over a fixed interval, reporting the summarized results once the interval has elapsed.
Measures the download and upload speed of the given connection in bits per second by performing multi-connection GET and POST HTTP requests to a target test node. Binary non-zero content, herein referred to as the payload, is hosted on a web server on the target test node. The test operates for either a fixed duration (in seconds) or a fixed volume (in MB). It can also report the recorded average throughput at multiple intervals during the test (e.g. once every 5 seconds). The client will attempt to download as much of the payload as possible for the duration of the test. The payload and all other testing parameters are configurable and may be subject to change in the future.
Four separate tests are supported:
- Single connection GET
- Multiconnection GET
- Single connection POST
- Multiconnection POST
How many threads do the tests execute?
Tests are executed using three concurrent connections. Each connection used in the test counts the numbers of bytes of the target payload, transferred between two points in time, and calculates the speed of each thread as Bytes transferred/Time (seconds).
Factors such as TCP slow start and congestion are taken into account by repeatedly downloading small chunks (default 256KB) of the target payload before the real testing begins. This "warm up" period will complete when three consecutive chunks are downloaded at the same speed (or within a small tolerance -- default is 10% -- of one another). In a multiconnection test, three individual connections are established (each on its own thread) and are confirmed as all having completed the warm up period before timing begins.
Content downloaded is copied to /dev/null or equivalent (i.e. it is discarded). Payload content for upload tests is generated and streamed on the fly from /dev/urandom (or equivalent).
The following is an example of the calculation performed for a multiconnection test using three concurrent connections.
- S = Speed (Bytes per second)
- B = Bytes (Bytes transferred)
- C = Time (Seconds) (between start time point and end time point)
- S1 = B1 / C1 (speed for Thread 1 calculation)
- S2 = B2 / C2 (speed for Thread 2 calculation)
- S3 = B3 / C3 (speed for Thread 3 calculation)
- Speed = S1 + S2 + S3
- Example values from a 3MB payload:
- B1 = 3077360 C1 = 15.583963
- B2 = 2426200 C2 = 15.535768
- B3 = 2502120 C3 = 15.536826
- S1 = B1/C1 = 197469.668017
- S2 = B2/C2 = 156168.655454
- S3 = B3/C3 = 161044.475879
- S1 + S2 + S3 = Total Throughput of the line = 197469.668017 + 156168.655454 + 161044.475879 = 514682 (Bps) * 0.000008 = 4.12 Mbps
What data is provided for the aspects of my broadband performance specific to the wireless radio connectivity provided by my carrier?
Availability of information about the wireless network will vary by operator, device support (smartphone/dongle) and territory. What is available under optimum conditions is detailed below:
- Location inferred from cell tower triangulation, WiFi triangulation, or GPS
- Is the device connected to a data network?
- Active cell tower ID and signal strength (RSSI)
- Visible neighboring cell towers, including cell IDs and RSSI
- Active network operator ID ("MNC") and name
- Active network country code ("MCC") and name
- SIM's network operator ID and name
- SIM's network country code and name
- Bearer (CDMA, GPRS, EDGE, all of the 3G variants, LTE, etc)
- Phone type (GSM, CDMA)
- Is the device in a roaming state?
How often do tests run? What's the app test schedule?
The application will periodically download a predefined test schedule randomized to execute tests on the following criteria:
How much data will the program use by default?
By default the app will limit the total monthly data traffic used for execution of scheduled tests to a maximum of 100MB. Volunteers can adjust the data cap to suit their preference. The higher the configured data cap the more frequently the app will schedule automated tests to execute. Volunteers will also be able to set the date of the month when data caps should reset to align app usage with their mobile broadband service billing.