Home > Electronics > Timestamp support for OpenLog

Timestamp support for OpenLog

OpenLog board

OpenLog board (Image) © Sparkfun


I needed a tool that can play back recorded serial stream “in real time”, so that the playback perfectly reflects the stream that was originally sent. Rather than building my own, OpenLog project seemed to be a perfect starting point. The board itself is very small, about the size of a micro-sd socket that is on the bottom of the board. By default the device records serial data received to standard ftdi-pinout serial port. The device is Arduino UNO compatible and code can be compiled and uploaded with the Arduino IDE. The device is configured by editing CONFIG.TXT file on the sd-card. If the file is not present on the card, the device creates one with default settings at the startup.

Example of config file:

115200,26,3,0,1,1,0,1
baud,escape,esc#,mode,verb,echo,ignoreRX,timestamp

As you can see, I added the timestamp option at the end. If this option is enabled, each received byte in the log file is prefixed with 4-byte unsigned long timestamp. The timestamp is in little endian order and in milliseconds. Unsigned long type provides about 55 days before counter rolls over. The value can be decoded for example in python with struct.unpack function:

buf = f.read(4)
timestamp = struct.unpack("<L", buf)[0]
char = f.read(1)

The logged files will be four times larger than without timestamping, but with large sd-cards this does not really make a difference. I quickly tested the performance and the device still seemed to be able to log the data without data drops at 115200 bps speed.

The modified codes are in my git repository: https://github.com/JanneMantyharju/OpenLog
I also made a pull request to the official repository, but let’s see will they accept it.

The python script that plays back the stream in real time can be found from the examples folder.

Advertisements
Categories: Electronics Tags: , , ,
  1. dennis
    11/11/2016 at 13:13

    Dear antibore,

    With pleasure i was reading your blog (backreading to 2011 lol). Especially the thesis grenade gave me a good laugh.

    However, I do have a more serious question. I am using the openlog as (of course) a serial recorder, in command mode. The system that outputs data controls the filenames by sending escape and append commands.

    However, the system i have is by far not accurate in its own timing. Every second in the system is 1.01-1.06 seconds in real time dependent of temperature and voltage so it has quite a drift after some days.

    What i would like to do is have the openlog print the 4 byte timestamp only at the beginning of each newly made file. Is this something I could implement myself easily or would it require many many changes in your git .ino code?

    That said, even if i manage to do so; what is the accuracy of the openlog timer? I am not looking after a maximum drift of a second in the span of a week, but 30 minutes off after 7 days would already be a bit too bad.

    Could you please tell me how to approach this in the best way and answer on the accuracy?

    Best regards

    Dennis

    • 11/11/2016 at 13:21

      The change itself would be quite straightforward, but you would need to add some hardware to openlog. It does not have a realtime clock and the wall time needs to come from somewhere. My code will just timestamp every byte starting from the time 0, since I was interested on playing the recorded data back at the same rate it was recorded. If I remember correctly, the openlog has a chrystal for internal clock, so the drift is not that bad. If you would use some realtime clock, they usually have internal temperature sensor to compensate the drift.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: