Repository navigation
Description
Bug report: rtsdiff(circular=True) locks onto an aliased solution after a large one-step change in angle
Summary
With circular=True, pynumdiff.rtsdiff can lock onto a solution whose derivative is off by a multiple of
2pi/dt (order 1) or keeps growing by 2pi/dt per step (order 2). The trigger is a single large step in the angle,
smaller than pi (so wrapping is not involved), together with a high log_qr_ratio (a high filter gain). The result
is completely wrong: |dx/dt| of thousands of rad/s on a signal that is constant after one step. The same call with
circular=False on the same data (the data needs no unwrapping) gives the correct derivative.
Reproduction (pynumdiff 0.3, dt = 0.01)
x = np.r_[np.zeros(10), np.full(10, 2.0)] # one step of 2 rad, well below pi
pynumdiff.rtsdiff(x, 0.01, order=2, log_qr_ratio=12, circular=True) # dx/dt runs away to ~6000 rad/s
pynumdiff.rtsdiff(x, 0.01, order=2, log_qr_ratio=12, circular=False) # max |dx/dt| ~114 rad/s, decays to 0
Expected: the two calls agree, because no wrap ever occurs.
Actual: after the step, the circular derivative grows by exactly 2pi/dt = 628.3 rad/s per step and never recovers.
With order 1 (log_qr_ratio 8, a 3 rad step) the derivative settles at exactly 2pi/dt = 628.3 rad/s.
forwardbackward=True does not fix it: the forward and backward passes are blended, so the wrong derivative persists.
x_hat gives no sign of the problem: it is returned wrapped, and an aliased trajectory matches the data modulo 2*pi.
How large a step it takes (forwardbackward=False, synthetic step, printed by the script): order 1 aliases from a
2.5 rad step at log_qr_ratio >= 10 (2.7 rad at 8), order 2 from 1.55 rad at log_qr_ratio 12 and 1.3 rad at 14, and
order 3 from 2.2 rad at 14. Nothing below pi aliases at log_qr_ratio <= 6.
Real data: the course direction (direction of the ground velocity, rad) of a Drosophila flight trajectory sampled at
100 Hz. The fly slows to 1.5 cm/s, so its course swings by 2.1 rad in one frame. log_qr_ratio = 12 is what
pynumdiff.optimize chooses for these data (order 2, forwardbackward, bandlimit 10 Hz; median over 40 trajectories).
Across 3,321 trajectories, circular=True gives max |dx/dt| > 300 rad/s for 136 and circular=False for 4.
Probable cause
The wrapped measurement model, y = wrap(x), cannot tell x_k from x_k + 2pim_k for any integer sequence m_k. Some of
those sequences are themselves solutions of the constant-derivative model: x_k + 2pink (velocity offset 2pin/dt,
order >= 1) and x_k + pink(k-1) (acceleration offset 2pin/dt^2, order >= 2). Once the filter is on such a
trajectory, every wrapped innovation is ~0 and nothing pulls it back. A high-gain filter can be pushed onto one in a
single step: after a large step it builds up a large velocity and acceleration, its next prediction overshoots the
measurement by more than pi, and the wrapped innovation then takes the branch 2*pi away. The step itself does not
have to be larger than pi.
Possible fixes
- When circular=True, unwrap the measurements (np.unwrap along
axis, over the non-NaN samples) and run the
ordinary filter. Each measurement is then compared with the previous measurement, not with the prediction, so an
overshooting prediction cannot pick the wrong branch. That is enough as long as consecutive samples differ by less
than pi, which the wrapped innovation also needs. - Or keep the wrapped innovation, but choose the branch from the previous measurement (or the previous posterior),
not the prediction: innovation = (y_unwrapped_from_last_measurement - C @ x_pred). - At least: detect it (a predicted change per step of more than pi, or a derivative near a multiple of 2*pi/dt) and
warn.
Run: python rtsdiff_circular_aliasing.py (needs numpy, matplotlib, pynumdiff >= 0.3). Prints the numbers above,
the smallest step that aliases for several orders and log_qr_ratios, and saves rtsdiff_circular_aliasing.png beside
this file.
"""
import os
import warnings
from importlib.metadata import version
import matplotlib.pyplot as plt
import numpy as np
warnings.simplefilter('ignore', UserWarning) # pynumdiff warns that CVXPY is missing
import pynumdiff
DT = 0.01
1. Synthetic: a constant angle with one step of 2 rad
x_step = np.r_[np.zeros(10), np.full(10, 2.0)]
2. Real: course direction (rad) of Drosophila trajectory 20121002_184808_2605 (van Breugel et al. 2014), 100 Hz.
The step between samples 7 and 8 is 4.18 rad, i.e. -2.10 rad after wrapping: groundspeed falls to 1.5 cm/s there.
x_course = np.array([-2.265256, -2.269022, -2.284692, -2.315834, -2.362328, -2.422941, -2.503468, -2.691861,
1.486292, 0.793781, 0.62091, 0.455346, 0.225893, -0.023637, -0.296044, -0.746268, -1.377319,
-1.821492, -2.05377, -2.15313, -2.145599])
groundspeed = np.array([0.289, 0.287, 0.276, 0.249, 0.206, 0.154, 0.099, 0.043, 0.015, 0.051, 0.08, 0.097, 0.103,
0.101, 0.085, 0.064, 0.06, 0.073, 0.091, 0.105, 0.116]) # m/s, for reference
ORDER, LOG_QR = 2, 12
def both(x, order=ORDER, log_qr_ratio=LOG_QR, forwardbackward=False):
'''(x_hat, dxdt_hat) with circular=True on x, and with circular=False on np.unwrap(x)'''
circ = pynumdiff.rtsdiff(x, DT, order, log_qr_ratio, forwardbackward=forwardbackward, circular=True)
plain = pynumdiff.rtsdiff(np.unwrap(x), DT, order, log_qr_ratio, forwardbackward=forwardbackward)
return circ, plain
print(f'pynumdiff {version("pynumdiff")}, numpy {np.version}; dt = {DT}, 2*pi/dt = {2 * np.pi / DT:.1f}\n')
np.set_printoptions(precision=1, suppress=True, linewidth=140)
for name, x in [('synthetic 2 rad step', x_step), ('real course direction', x_course)]:
for fb in (False, True):
(, dc), (, dp) = both(x, forwardbackward=fb)
print(f'{name}, order {ORDER}, log_qr_ratio {LOG_QR}, forwardbackward={fb}')
print(' circular=True dx/dt:', dc)
print(' circular=False dx/dt:', dp)
print()
(, d1c), (, d1p) = both(np.r_[np.zeros(10), np.full(10, 3.0)], order=1, log_qr_ratio=8)
print('synthetic 3 rad step, order 1, log_qr_ratio 8: final dx/dt circular', round(d1c[-1], 1), ' plain',
round(d1p[-1], 1), '\n')
print('Smallest one-step change (rad) after which circular=True diverges from circular=False (by > 1 rad/s),')
print('forwardbackward=False; "-" = none below pi')
steps = np.arange(0.05, np.pi, 0.05)
print('order ' + ''.join(f'{f"lq={lq}":>8}' for lq in (2, 4, 6, 8, 10, 12, 14)))
for order in (1, 2, 3):
row = []
for lq in (2, 4, 6, 8, 10, 12, 14):
bad = [J for J in steps if np.abs(np.subtract(*(r[1] for r in both(np.r_[np.zeros(10), np.full(10, J)],
order, lq)))).max() > 1]
row.append(f'{bad[0]:8.2f}' if bad else f'{"-":>8}')
print(f'{order:5d} ' + ''.join(row))
Plot: angle (top) and derivative (bottom) for the two examples
fig, axes = plt.subplots(2, 2, figsize=(10, 6), sharex='col', constrained_layout=True)
for col, (name, x) in enumerate([('Synthetic: one 2 rad step', x_step),
('Real: Drosophila course direction', x_course)]):
t = np.arange(len(x)) * DT
(xc, dc), (xp, dp) = both(x)
ax = axes[0, col]
ax.plot(t, x, 'ko', ms=4, label='measured (wrapped)')
ax.plot(t, xc, '-', color='tab:red', label='x_hat, circular=True')
ax.plot(t, np.angle(np.exp(1j * xp)), '--', color='tab:blue', label='x_hat, circular=False on np.unwrap(x)')
ax.set_ylabel('angle (rad)')
ax.set_title(f'{name}\norder {ORDER}, log_qr_ratio {LOG_QR}, dt {DT}')
ax.legend(fontsize=7)
ax = axes[1, col]
ax.plot(t, dc, '-o', color='tab:red', ms=3, label='circular=True')
ax.plot(t, dp, '--o', color='tab:blue', ms=3, label='circular=False on np.unwrap(x)')
for n in (-10, -5, -1, 1, 5, 10):
ax.axhline(2 * np.pi / DT * n, color='0.8', lw=0.5, zorder=0)
ax.set_yscale('symlog', linthresh=100)
ax.set_ylabel('dx/dt (rad/s); grey: multiples of 2π/dt')
ax.set_xlabel('time (s)')
ax.legend(fontsize=7)
out = os.path.join(os.path.dirname(os.path.abspath(file)), 'rtsdiff_circular_aliasing.png')
fig.savefig(out, dpi=150)
print(f'\nSaved {out}')
Activity
Merge pull request #241 from florisvb/fix-240-circular-branch
pavelkomarov commentedon Nov 3, 2026
I suspect that what you wrote with Claude isn't minimal @florisvb. I'm auditing to try to simplify.
folded the #240 step regression check into test_circular_rtsdiff, so …