Skip to content

rtsdiff(circular=True) can runaway on an aliased solution #240

Closed

Description

@florisvb
Image

Bug report: rtsdiff(circular=True) locks onto an aliased solution after a large one-step change in angle

Summary

With circular=True, pynumdiff.rtsdiff can lock onto a solution whose derivative is off by a multiple of
2pi/dt (order 1) or keeps growing by 2pi/dt per step (order 2). The trigger is a single large step in the angle,
smaller than pi (so wrapping is not involved), together with a high log_qr_ratio (a high filter gain). The result
is completely wrong: |dx/dt| of thousands of rad/s on a signal that is constant after one step. The same call with
circular=False on the same data (the data needs no unwrapping) gives the correct derivative.

Reproduction (pynumdiff 0.3, dt = 0.01)

x = np.r_[np.zeros(10), np.full(10, 2.0)]        # one step of 2 rad, well below pi
pynumdiff.rtsdiff(x, 0.01, order=2, log_qr_ratio=12, circular=True)    # dx/dt runs away to ~6000 rad/s
pynumdiff.rtsdiff(x, 0.01, order=2, log_qr_ratio=12, circular=False)   # max |dx/dt| ~114 rad/s, decays to 0

Expected: the two calls agree, because no wrap ever occurs.
Actual: after the step, the circular derivative grows by exactly 2pi/dt = 628.3 rad/s per step and never recovers.
With order 1 (log_qr_ratio 8, a 3 rad step) the derivative settles at exactly 2
pi/dt = 628.3 rad/s.
forwardbackward=True does not fix it: the forward and backward passes are blended, so the wrong derivative persists.
x_hat gives no sign of the problem: it is returned wrapped, and an aliased trajectory matches the data modulo 2*pi.

How large a step it takes (forwardbackward=False, synthetic step, printed by the script): order 1 aliases from a
2.5 rad step at log_qr_ratio >= 10 (2.7 rad at 8), order 2 from 1.55 rad at log_qr_ratio 12 and 1.3 rad at 14, and
order 3 from 2.2 rad at 14. Nothing below pi aliases at log_qr_ratio <= 6.

Real data: the course direction (direction of the ground velocity, rad) of a Drosophila flight trajectory sampled at
100 Hz. The fly slows to 1.5 cm/s, so its course swings by 2.1 rad in one frame. log_qr_ratio = 12 is what
pynumdiff.optimize chooses for these data (order 2, forwardbackward, bandlimit 10 Hz; median over 40 trajectories).
Across 3,321 trajectories, circular=True gives max |dx/dt| > 300 rad/s for 136 and circular=False for 4.

Probable cause

The wrapped measurement model, y = wrap(x), cannot tell x_k from x_k + 2pim_k for any integer sequence m_k. Some of
those sequences are themselves solutions of the constant-derivative model: x_k + 2pink (velocity offset 2pin/dt,
order >= 1) and x_k + pi
nk(k-1) (acceleration offset 2pin/dt^2, order >= 2). Once the filter is on such a
trajectory, every wrapped innovation is ~0 and nothing pulls it back. A high-gain filter can be pushed onto one in a
single step: after a large step it builds up a large velocity and acceleration, its next prediction overshoots the
measurement by more than pi, and the wrapped innovation then takes the branch 2*pi away. The step itself does not
have to be larger than pi.

Possible fixes

  • When circular=True, unwrap the measurements (np.unwrap along axis, over the non-NaN samples) and run the
    ordinary filter. Each measurement is then compared with the previous measurement, not with the prediction, so an
    overshooting prediction cannot pick the wrong branch. That is enough as long as consecutive samples differ by less
    than pi, which the wrapped innovation also needs.
  • Or keep the wrapped innovation, but choose the branch from the previous measurement (or the previous posterior),
    not the prediction: innovation = (y_unwrapped_from_last_measurement - C @ x_pred).
  • At least: detect it (a predicted change per step of more than pi, or a derivative near a multiple of 2*pi/dt) and
    warn.

Run: python rtsdiff_circular_aliasing.py (needs numpy, matplotlib, pynumdiff >= 0.3). Prints the numbers above,
the smallest step that aliases for several orders and log_qr_ratios, and saves rtsdiff_circular_aliasing.png beside
this file.
"""
import os
import warnings
from importlib.metadata import version

import matplotlib.pyplot as plt
import numpy as np

warnings.simplefilter('ignore', UserWarning) # pynumdiff warns that CVXPY is missing
import pynumdiff

DT = 0.01

1. Synthetic: a constant angle with one step of 2 rad

x_step = np.r_[np.zeros(10), np.full(10, 2.0)]

2. Real: course direction (rad) of Drosophila trajectory 20121002_184808_2605 (van Breugel et al. 2014), 100 Hz.

The step between samples 7 and 8 is 4.18 rad, i.e. -2.10 rad after wrapping: groundspeed falls to 1.5 cm/s there.

x_course = np.array([-2.265256, -2.269022, -2.284692, -2.315834, -2.362328, -2.422941, -2.503468, -2.691861,
1.486292, 0.793781, 0.62091, 0.455346, 0.225893, -0.023637, -0.296044, -0.746268, -1.377319,
-1.821492, -2.05377, -2.15313, -2.145599])
groundspeed = np.array([0.289, 0.287, 0.276, 0.249, 0.206, 0.154, 0.099, 0.043, 0.015, 0.051, 0.08, 0.097, 0.103,
0.101, 0.085, 0.064, 0.06, 0.073, 0.091, 0.105, 0.116]) # m/s, for reference

ORDER, LOG_QR = 2, 12

def both(x, order=ORDER, log_qr_ratio=LOG_QR, forwardbackward=False):
'''(x_hat, dxdt_hat) with circular=True on x, and with circular=False on np.unwrap(x)'''
circ = pynumdiff.rtsdiff(x, DT, order, log_qr_ratio, forwardbackward=forwardbackward, circular=True)
plain = pynumdiff.rtsdiff(np.unwrap(x), DT, order, log_qr_ratio, forwardbackward=forwardbackward)
return circ, plain

print(f'pynumdiff {version("pynumdiff")}, numpy {np.version}; dt = {DT}, 2*pi/dt = {2 * np.pi / DT:.1f}\n')
np.set_printoptions(precision=1, suppress=True, linewidth=140)
for name, x in [('synthetic 2 rad step', x_step), ('real course direction', x_course)]:
for fb in (False, True):
(, dc), (, dp) = both(x, forwardbackward=fb)
print(f'{name}, order {ORDER}, log_qr_ratio {LOG_QR}, forwardbackward={fb}')
print(' circular=True dx/dt:', dc)
print(' circular=False dx/dt:', dp)
print()

(, d1c), (, d1p) = both(np.r_[np.zeros(10), np.full(10, 3.0)], order=1, log_qr_ratio=8)
print('synthetic 3 rad step, order 1, log_qr_ratio 8: final dx/dt circular', round(d1c[-1], 1), ' plain',
round(d1p[-1], 1), '\n')

print('Smallest one-step change (rad) after which circular=True diverges from circular=False (by > 1 rad/s),')
print('forwardbackward=False; "-" = none below pi')
steps = np.arange(0.05, np.pi, 0.05)
print('order ' + ''.join(f'{f"lq={lq}":>8}' for lq in (2, 4, 6, 8, 10, 12, 14)))
for order in (1, 2, 3):
row = []
for lq in (2, 4, 6, 8, 10, 12, 14):
bad = [J for J in steps if np.abs(np.subtract(*(r[1] for r in both(np.r_[np.zeros(10), np.full(10, J)],
order, lq)))).max() > 1]
row.append(f'{bad[0]:8.2f}' if bad else f'{"-":>8}')
print(f'{order:5d} ' + ''.join(row))

Plot: angle (top) and derivative (bottom) for the two examples

fig, axes = plt.subplots(2, 2, figsize=(10, 6), sharex='col', constrained_layout=True)
for col, (name, x) in enumerate([('Synthetic: one 2 rad step', x_step),
('Real: Drosophila course direction', x_course)]):
t = np.arange(len(x)) * DT
(xc, dc), (xp, dp) = both(x)
ax = axes[0, col]
ax.plot(t, x, 'ko', ms=4, label='measured (wrapped)')
ax.plot(t, xc, '-', color='tab:red', label='x_hat, circular=True')
ax.plot(t, np.angle(np.exp(1j * xp)), '--', color='tab:blue', label='x_hat, circular=False on np.unwrap(x)')
ax.set_ylabel('angle (rad)')
ax.set_title(f'{name}\norder {ORDER}, log_qr_ratio {LOG_QR}, dt {DT}')
ax.legend(fontsize=7)
ax = axes[1, col]
ax.plot(t, dc, '-o', color='tab:red', ms=3, label='circular=True')
ax.plot(t, dp, '--o', color='tab:blue', ms=3, label='circular=False on np.unwrap(x)')
for n in (-10, -5, -1, 1, 5, 10):
ax.axhline(2 * np.pi / DT * n, color='0.8', lw=0.5, zorder=0)
ax.set_yscale('symlog', linthresh=100)
ax.set_ylabel('dx/dt (rad/s); grey: multiples of 2π/dt')
ax.set_xlabel('time (s)')
ax.legend(fontsize=7)
out = os.path.join(os.path.dirname(os.path.abspath(file)), 'rtsdiff_circular_aliasing.png')
fig.savefig(out, dpi=150)
print(f'\nSaved {out}')

Activity

  1. added a commit that references this issue on Oct 27, 2026
    f4184e3
  2. pavelkomarov commented on Nov 3, 2026

    @pavelkomarov
    Collaborator

    I suspect that what you wrote with Claude isn't minimal @florisvb. I'm auditing to try to simplify.

  3. added a commit that references this issue on Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions