Sensor Recording

Discover your Android Smartphone!

Readout, display and record smartphone sensor data

Frequently Asked Questions

• Q0: Why is the data recording still running, even if I "kill" the app from the operating system?
• A: The recording is implemented as so-called service, to ensure that it is not stopped in standby mode. It can only be finished with the button "Stop" in the menu "Sensors". Thus, all the files will be finalised properly as well.
  Since version 1.33, the active status of the sensors is indicated by an icon in the notification bar.
• Q1: Why is there only one csv file per day, but several kml files?
• A: The csv files contain just one line per data set. When the data recording is stopped and started again, the file is just continued. Its filename contains only the current date. In contrast, the kml file contains a lead in command sequence at the begin and a lead out sequence at the end of file. This can't be simply continued. Therefore a new file is started with each recording. For distinction, each file has the date and time in the name.
  As of version 1.35, there is one kml file per day. Multiple records are split within this file in several "Placemark" sections.
• Q2: Why can't I see a track plot on the map when opening the related kml file?
• A: Probably, you didn't finish the recording properly with the "stop" button. This is required to write the closing command sequence into the kml file. In order to repair incomplete kml files, just start the app once again and wait for the related message. This feature is available as of version 1.9.
• Q3: How can I monitor my own sleeping behaviour?
• A: Before the data recording, make sure that the battery is fully charged. Start the app and set the relevant sensors (e.g. gyroscope and acceleration) to "view + record". All others should be switched "OFF" in order to save battery power. Then start recording and set the smartphone into standby mode, so that no unintentional operation will happen. Finally attach the device somehow to your body or pyjama. (You could just stick it into a sock.) When you awake the next morning, your nightly movements have been recorded in the csv file and can be evaluated with an appropriate tool, e.g. MS Excel.
• Q4: Why are some data not recorded in standby mode? (See also Q16)
• A: This is a very difficult question. The behaviour in standby mode seems to be different for each type of smartphone and android version (or hardware drivers, respectively). On my Huawei U8860 with android 4.0.3, it worked for all sensors until Dec 2013, but not any more since Jan 2014. I have absolutely no idea, what has changed in between. (Maybe the behaviour also depends upon the battery charge level. But this is only a vague suspicion.) This article in the internet (https://sites.google.com/a/bug-br.org.br/android/technical-documents/sensors-and-standby-mode) describes the following workaround: Implement a receiver, which is triggered by the action SCREEN_OFF, indicating the transition to standby. When this occurs, re-register the listeners for all sensors. On my Huawei, this "bug fix" re-activated at least two sensors: accelerometer and gyroscope. This is better than nothing, but still less than in 2013. Btw: The position data was always and is still recorded in standby mode.
  My Samsung Galaxy S3 Neo always records all types of sensors. However, in standby mode the sample period goes slightly up and is fluctuating unpredictable.
  As of app version 1.85, the increase of the sample period should be fixed. The appropriate countermeasure was usage of "WakeLock". (However, this could lead to a faster discharge of the battery.)
• Q5: I have problems to open the csv file in Excel. What could be wrong with this?
• A: csv stands for "comma separated values". This is simply an ASCII file which contains the data values: text and numbers. However, the separator used by the app is not the comma, but the semicolon instead. If in your Excel the comma is configured as separator, it will not handle the semicolons as expected. In this case, there are two options: Either find and replace all separator characters with any text editor before opening the File with Excel, or configure Excel to use the semicolon accordingly. This configuration is different for every version of Windows and Excel. It may even depend upon your country and language settings. Please refer to the related help function.
• Q6: What was the reason for introducing the new columns "excel time" and "abs value" in the csv file?
• A: The values in both columns could be derived with some simple calculations in Excel itself. But they are so useful for creating diagrams, so I decided to implement them as a built-in function. The "abs value" is just the 3-dimensional square root of the x,y,z values in front.
• Q7: What is the meaning of the strange numbers in the column "excel time"?
• A: The time format of Excel is a floating-point number, counting the days since 00.01.1900 (sic!). The position after decimal point corresponds to the time of day. For example 41649.5 stands for 10.Jan 2014 12:00:00 o'clock at noon. The figures can be changed to a readable text by selecting the desired date/time format for this column in Excel. This applies for the related diagram as well.
• Q8: Why does the data recording stop occasionally? Especially when I start other big apps, such as a web browser.
• A: Without possible countermeasures, services running in the background may be stopped by android, if the system is short of memory. As of app version 1.31, this issue should be fixed. (The appropriate countermeasure was usage of startForeground).
• Q9: What is the meaning of the message "Standstill detected: restart of sensors!"?
• A: Under some circumstances (e.g. after a restart of the smartphone), the displayed speed over ground is bigger than zero, even if the device is at a standstill. If this is detected, the app tries to restart the sensors.
  If this doesn't help, please stop the data recording, exit the app, remove it from the app list and start it again. Now the speed should be correct.
• Q10: What is the meaning of the various angle values (heading, course and bearing)?
• A: All the angles are related to a north oriented coordinated system, starting from 0° (North), with bigger values when the angle is turning clockwise: e.g. 90° (East), 180° (South), 270° (West), up to 359.9° (close to North).
  • Heading is the direction of the longitudinal axis of a vehicle (or, in this case, the axis of the smartphone). It is derived from the orientation sensor, measured by the magnetic compass.
  • Course (over Ground) is the direction of movement, measured by the GPS position sensor. In stationary case (i.e. speed = 0), the course is undefined.
  • Bearing is the direction to go between two points, e.g. from your current position to a target position, as it is used in the function "Search" (Find my car!).
  In the search function, all three angles are shown graphically in the compass rose. If you want to approach your target position, you have to move into the direction of the bearing. That means: your course vector (red) has to match the bearing vector (green) as close as possible.
• Q11: What do the following messages mean?
  1. x kml file(s) repaired!
  2. x kml file(s) cleaned!
• A: The kml files have a well-defined format. Example:

  <?xml version="1.0" encoding="UTF-8"?>
  <kml xmlns="http://www.opengis.net/kml/2.2">
    <Document>
   
      <Placemark>
        <Info>name;Sensor Recording;variant;Pro;version;1.54;code;28</Info>
        <Info>Start at 2015-02-13 06:59:16.422 +0100 (MEZ)</Info>
        <Style>
        <LineStyle>
          <color>0xff0000ff</color>
          <width>4</width>
        </LineStyle>
        </Style>
        <LineString>
          <extrude>0</extrude>
          <tessellate>0</tessellate>
          <altitudeMode>clampToGround</altitudeMode>
          <coordinates>
  
            8.7817395,53.0552003,0.0
            8.7817948,53.0552671,0.0
            8.7817973,53.0552485,0.0
  
          </coordinates>
        </LineString>
        <Info>Stop at 07:26:36.711</Info>
      </Placemark>
  
    </Document>
  </kml>
  

  1. If the recording is interrupted for whatever reason, the end sequence is missing. In this case, the file can't be displayed with Google Earth and has to be repaired.
  2. If the recording is started and directly stopped again without any coordinates, an empty <Placemark> sequence is generated, which is waste. This needless sequence may irritate Google Earth and should be cleaned.
• Q12: How can I process a csv file with MS Excel, e.g. in order to generate a graphical presentation like the following? (Click on the following diagram to open the related Excel file.)
  
• A: The following example is a step-by-step instruction for MS Excel 2010. Other versions may have different commands. Follow the instructions accordingly:
  • Open the csv file with Excel with a double click.
    Make sure that you have selected the dot "." as decimal separator.
    Ribbon "File": Select Option / Advanced / Editing Options / Decimal separator / "."
  • Save the file as type "Excel Workbook (*.xlsx)".
  • Make some optical adjustments as desired: Column size, Borders, Font type, Fill Color, etc.
  • Select Column C ("excel time").
    Ribbon "Home": Menu "Number", select "Time".
    Possibly, re-order columns as requested.
  • Select the line starting with "1-13+ (sensors)".
    Ribbon "Data", click on the icon "Filter".
    
  • Optionally: Select the line below.
    Ribbon "View": Select "Freeze Panes", to fix the lines above.
  • Select again the line above "1-13+ (sensors)".
    In the left-hand column A, select the desired data type, e.g. "1" for accelerometer.
  • Now copy the desired data area to a new sheet.
    Possibly, make additional calculations as requested.
  • Select the columns you want to display, always starting with "excel time".
    Ribbon "Insert": Select Charts / Scatter / Scatter Type (Markers, Straight Lines, ...).
  • Move the new appearing window with a right mouse click and "Move Chart" to a new sheet.
  • Adjust the diagram as requested.
    Ribbon "Layout": Chart Title, Axis Titles, Gridlines, etc.
• Q13: Is the data recording affected by the settings of the units for measured values?
• A: No! The data in the recorded files is always stored in the basic unit, e.g. m/s for speed.
• Q14: Why does my smartphone show neither temperature nor pressure?
• A: Not every smartphone has every type of sensor built-in. This can easily be seen in the start screen (Sensors). However, the battery temperature is measured, displayed and stored in the csv file under "action / BATTERY_CHANGED", together with voltage, charge level, etc. Tested on Samsung Galaxy S3 Neo, Huawei U8860 Honor and others. See following screenshot, at the bottom:
  
• Q15: Why is it not possible to record data with this app and to perform video recording with the camera app simultaneously?
• A: The reason is the microphone level. It cannot be shared between different apps. Therefore, switch "OFF" the microphone level in the "Config" page. Then you can record data (without microphone, of course) and do video recording in parallel. This feature is possible since version 2.03/2.04.
• Q16: Why does the data recording stop under Android 8 (Oreo) after some minutes, if the app is running in the background, e.g. when the screen is switched OFF? (See also Q4)
• A: Google's new policy is: Battery charge has a higher value (!) than app functionality. As a result, background apps are stopped without further ado, when they consume a lot of power. Possible workaround: Keep the screen ON permanently. That will even burn more power, but the app works at least. Adjust the screen to low brightness, in order to spare the battery as much as possible.
  
  All solutions from the Internet did not work out. Keywords for experts: startForegroundService (intent), in the service itself: startForeground (id, notification), wakeLock, whitelist, return START_STICKY.
  If you know a working solution, please let me know!
  
  Addendum: On my Huawei P10 Lite with Android 8.0 there is an option that works. But if that applies to other devices as well, I can't say. It's worth a try to look for it:
  • Settings
  • Battery
  • Launch
  • Select App
  • Switch from "Manage automatically" to "... manually"
  • In the pop-up window, set the switch "Run in background"
  Good luck!
• Q17: Since Version 3.15, the menu "Read back Data" allows the reading of kml files as well. Are these equivalent to csv files?
• A: No! The kml files do not contain sensor data at all, only positions. And here, we are recording just longitude, latitude and altitude. No accuracy, course and speed over ground is recorded. The course and speed is reconstructed provisional from position differences (of course only from GPS, not from Net). But this is just a makeshift - better than nothing. So the csv files have to be preferred, if they exist.
• Q18: As of May 30th 2020 the most weather maps are not shown any more - Exception: USA with Alaska and Hawaii. What is the reason?
• A: This only affects websites with https - unfortunately most of the maps. Furthermore, the effect occurs only on older Android versions (less than 5) auf. The reason is the so-called "cross-sign certificates". For further information, see stackoverflow or comodo.
• Q19: As of Android 10, there is a new permission ACCESS_BACKGROUND_LOCATION. What is it good for?
• A: This is an option to save battery charge by switching OFF the display - while the Device location continues to be recorded in the background. For further explanation, see the Background Location Demo.
• Q20: Until Android 10 ("Q"), the path for the app's home directory "SensorRecording" was, for example:
  /storage/emulated/0/SensorRecording
  As of Android 11 ("R") it is:
  /storage/emulated/0/Android/data/net.braun_home.sensorrecording.pro/files/SensorRecording
  Why is this?
• A: The reason is a new Google policy (called "scoped storage"), which shall avoid that apps can access any storage on the whole device (called "legacy storage").
  See also: Target API level 30 and new permission MANAGE_EXTERNAL_STORAGE. This permission must be requested from Google and is granted only for a few special apps (e.g. File Manager, Virus Scanner). For "Sensor Recording", my request was rejected, so I have to live with "Plan B":
  At the 1st start of the app, the directory is copied to the new place, where it is used for reading and writing of all data file from now on. Please note, that this directory is removed when the app is de-installed. If you want to keep the data, you have to save them before.
• Q21: Since the "SensorRecording" directory has been moved to ".../Android/data/...", I cannot longer access it from a PC. What's going wrong?
• A: Memory access is severely restricted under Android 11 - for security reasons against unauthorized access. There are only two directories (and their subdirectories) left for external access: "DCIM" and "Pictures". To copy files from other directories to a PC, they must first be copied there with a file manager. Attention! Not every file manager can access ".../Android/data/...", e.g. the "Total Commander" doesn't. If you have problems, try another one. To enable the external access to the device, the USB connection has to be set to "PTP" (Picture Transfer Protocol).
• Q22: The battery data set consists of 4 values: level, voltage, temperature and mixed. What is the meaning of "mixed"?
• A: It is a combination of 7 bits, stored in a float value. Thus, ignore the decimal places. The integer part has to be interpreted like this:
  

      Bits  6 5 4 3 2 1 0
            | `---´ `---´
            |   |     |
            |   |    charge status (1=unknown, 2=charging, 3=discharging, 4=not charging, 5=full)
            |  charge plug (0=no plug, 1=AC, 2=USB, 4=wireless)
           screen (0=OFF, 1=ON)
      

  The graphical presentation in the menu "Graphics" is:
  - "charge": charge status == 2 (charging)
  - "screen": screen == 1 (ON)
• Q23: The app initially works on my smartphone, but after a while there is a crash with "OutOfMemoryException". Why?
• A: The cause is probably a weak memory configuration of the device. Remedy: In the "Config" menu, reduce the update period and/or the number of selected sensors to what is absolutely necessary. Since Version 8.00 there is also the possibility to select a "memory limit" (15 or 30 minutes, 1, 2 of the 4 hours). Then the data in the internal memory are no longer summed up endlessly, instead, new data overwrite old ones in a so-called ring buffer. The data recording in the csv and kml files will, of course, continue seamlessly.