Daniel Beer Atom | RSS | About

UART AFSK radio receiver

11 Jul 2016

Recently, while experimenting with an RTL-SDR device, I spotted a particularly strong intermittent signal on 456.89 MHz. The signal appears in bursts lasting a few seconds several times per minute.

After a bit of hunting on the radio spectrum management database, I deduced that this was coming from a transmitter on top of a nearby water tower. Following an OIA request to the Dunedin City Council, who owns the equipment, I was given some helpful information by Malcolm Wallace at Communication Specialists, and I decided to attempt to implement a software demodulator to read the transmitted data in realtime.

Looking at the waterfall diagram of a captured signal in baudline, we can deduce the modulation scheme. The symmetrical sidebands, of a lower amplitude than the main carrier, indicate an AM signal. Within the AM signal, there appears to be a FSK signal carrying UART data (see the abrupt sideband shifts towards the bottom of the diagram). There are additional spikes (“shadows”) in the spectrum due to tone-generator distortion.

The FSK signal in this particular capture uses mark and space frequencies of around 2 kHz, with about 200 Hz of separation. The baud rate is 300 bps, with 8 bits per word, 1 stop bit and even parity. This data comprises packets transmitted between devices on a local SCADA network.

Demodulation algorithm

The software is arranged into separable stages. Each stage takes arbitrary-sized chunks of input data, buffers and processes them and produces new chunks of data for the next stage.

The data at various stages includes complex (I/Q) samples, real samples and digital (binary state) samples. Some stages emit hybrid data consisting of a combination of a complex or real signal and a set of digital signals encoded in a bitmask.

The data processing chain consists of the following stages:

Sample parsing

The parser stage receives blocks of binary data (const uint8_t *) from stdin. This data is expected to be RTL-SDR format (8 bit unsigned I/Q samples). The output from this stage is complex I/Q samples.

Down-mixing and decimation

We’re only interested in activity within a relatively narrow band. This stage first mixes the data to center on the nominal carrier frequency. It then applies an FIR lowpass filter to cut out all signals outside of the band of interest, and then decimates the signal by a large ratio. Typically, we’re decimating from 2048 ksps to about 20 ksps.

This is the most computationally expensive step in the processing chain. Note that because of the large decimation ratio, a lot of processing time can be saved by only calculating the convolution at points centered around samples that are actually going to make it through the filter. This technique, applied generally, results in decimating filters whose runtime does not increase with decimation ratio – a higher ratio simply means a wider kernel applied at proportionally fewer points.

In order to apply an FIR filter, we need to keep some context either side of the buffer block we’re currently processing. We process the middle of the buffer, and then slide some of the data from the end back down to the start of the buffer to use as context and the beginning of the next block.

The output from this stage is complex I/Q samples, but at a vastly reduced rate.

Carrier detection and tracking

The RF oscillator in the transmitter might have an accuracy of 10 ppm. At 456.89 MHz, this translates to a possible error in the carrier frequency of up to 4 kHz – twice the tone frequency and many times the tone separation!

This stage collects blocks of data corresponding to roughly 1/10th of a second. Over each block, we estimate the average frequency by averaging the arctangent of ratios of successive samples. Provided that the carrier is significantly stronger than the background noise and not over-modulated, this gives us a fairly good estimate of the carrier frequency.

We use frequency stability and amplitude estimates to decide when a carrier is present. We also use the frequency estimates to mix the signal again so that the carrier appears at a fixed offset (about -2 kHz).

The output from this stage is complex I/Q samples, plus a binary carrier detect signal.

AM demodulation

A popular textbook method of AM demodulation is envelope detection. However, this performs very poorly in the presence of noise.

Fortunately, due to the behaviour of our previous stage, we now have the signal shifted to a fixed offset, effectively eliminating carrier frequency error. If we ensure that this offset results in the signal being centered with 0 Hz in the middle of one of the side-bands, then we can demodulate just by applying an FIR lowpass filter.

The cut-off for the filter should be chosen to be less than the distance from 0 Hz to the carrier. Then we cut out the carrier and lower side-band, leaving an AM-demodulated complex signal.

The output from this stage is complex I/Q samples, plus the binary carrier detect signal forwarded from the previous stage.

FM demodulation

FM demodulation is accomplished by a simple operation: we estimate the instantaneous frequency at each sample by dividing the sample following by the sample preceeding, and calculating the arctangent of this ratio.

The output from this stage is real samples, plus the binary carrier detect signal forwarded from the previous stage.

Noise reduction

The sample rate at this point is many times in excess of the rate at which we expect tone frequency changes. We can take advantage of this for noise reduction by applying a simple FIR lowpass filter on the real samples.

The output from this stage is real samples (see the scope trace below), plus the binary carrier detect signal forwarded from the previous stage.

Binarization

The purpose of this stage is to convert the real-valued output from the FM signal to a realtime binary-valued signal. We do this in a way that doesn’t require specification ahead of time of fixed tone frequencies (although we do require a minimum separation).

The incoming real-valued signal is buffered into slices of a fixed size – roughly 20 bit-times. We expect that we will see both the high and the low level over this time period. For each time period, we assemble a histogram of the values present in the period, its predecessor, and its successor. We then pick out the two largest peaks from the histogram and use them to set Schmitt-trigger levels for the period under examination.

The output from this stage is the binary carrier detect signal, plus a binary data signal. There is no further need for the analog samples.

UART decoding

This stage consists of a bunch of state machines. The data signal is fed through a debouncer for glitch removal and then into an ordinary UART sampling state machine. The carrier detect signal is fed through a transition-detecting state machine.

This stage does not emit data in blocks. Rather, it invokes a callback for three kinds of events: carrier detect, carrier loss, and UART word received.

The callback invoked by the last stage prints events on stdout. For example, feeding the captured trace shown in the waterfall above, with appropriate settings, results in the following output:

CONNECT
> ff -1111111111
> ff -1111111111
> ff -1111111111
> ff -1111111111
> 00 ---------11
> 0d -1-11-----1
> 00 ---------11
> 23 -11---1---1
> 43 -11----1--1
> 60 ------11-11
> 0b -11-1-----1
> 00 ---------11
> 00 ---------11
> 00 ---------11
> 02 --1-------1
> 00 ---------11
> 5d -1-111-1--1
> 48 ----1--1-11
> 00 ---------11
NO CARRIER

You can download the source code here:

• rtlafsk.cpp

Compile the program with a version of GCC that supports C++11 (the version bundled with any relatively recent Linux distribution should suffice):

g++ -O1 -Wall -std=c++11 -o rtlafsk rtlafsk.cpp

You can get help with:

./rtlafsk --help

A typical use is to pipe in data from the RTL-SDR program like so:

rtl_sdr -g 30 -f 456972300 | ./rtlafsk --carrier-frequency -82750

On a modestly powerful laptop, this should run fast enough to process 2048 ksps data in realtime with less than 15% CPU load.

You will likely need to adjust the default settings for your scenario. Do not use the RTL-SDR’s automatic gain control, as this will interfere with carrier detection. Instead, specify a fixed gain (e.g. 30 dB, as shown above).