mckero/Look4Sat-mckero
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 20 Wiki Activity

fix(cw): stop silence and edge estimates from defeating the tone shift

Browse Source 
Two audit findings, both measured, both able to silently disable the feature.

A detection window landing in a keying gap used to collapse an established
shift to zero. CW is keyed, so gaps are normal: over 180 s of keyed audio at
1400 Hz, 11 of 90 detections saw no tone, and each one wiped the decode window
and left the next ~2 s buffered unshifted - outside the model's range and
therefore invisible to it. Absence of a tone is now absence of evidence and the
active shift is retained.

Hysteresis moved from shift space to tone space, anchored on the pitch that
produced the active shift. The old rule required a non-zero previous shift and
a needed shift, so it lapsed exactly where the jump is largest: at the 1200 Hz
edge one 12.5 Hz estimate hop flips between "inside" (shift 0) and "outside"
(a large shift). Measured 35 window drops in 60 detections for a 1205 Hz tone,
and 10 in 10 for a bare one-bin hop. A shift of zero is a real state, not the
absence of one. Slow drift still catches up, since the anchor bounds staleness
at the margin rather than letting it accumulate.

Detection prominence raised from 3.0 to 8.0. Pure noise peaks at 2.0-3.3 times
its own spectral mean, so 3.0 admitted roughly one noise window in five as a
"tone" - and a false tone is worse than none, since it moves a good signal out
of range. Keyed CW measures 47-51, so the gap is wide.

Shifted output is clamped to the +/-1.0 range the spectrogram assumes. The
Hilbert kernel's L1 gain is 2.51, so mixing overshoots: a full-scale square
wave measured 2.35 and even a plain sine 1.05.

The detection pool moved to core:domain as CwDetectionPool so its ring
behaviour can be tested directly - mutation testing showed the previous private
implementation was unreachable from any test. Its chronological-order contract
now has 11 tests driving the real class.

Removed the write-only detectedToneHz field.

74 CW tests pass, golden vectors included.
This commit is contained in:
mckero committed 2026-08-22 02:35:58 +00:00
1 parent f6db55b35c
commit fdb44af9ff
7 files changed
+603 -147
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

core/data/src/main/java/com/rtbishop/look4sat/core/data/cw/CwDeepDecoder.kt
+60 -71
View File
@@ -25,6 +25,7 @@ import android.util.Log
import com.rtbishop.look4sat.core.domain.cw.CwCtcDecoder
import com.rtbishop.look4sat.core.domain.cw.CwDeepBuffer
import com.rtbishop.look4sat.core.domain.cw.CwDeepSpectrogram
import com.rtbishop.look4sat.core.domain.cw.CwDetectionPool
import com.rtbishop.look4sat.core.domain.cw.CwToneShifter
import com.rtbishop.look4sat.core.domain.cw.ICwDecoder
import kotlinx.coroutines.CancellationException
@@ -79,7 +80,7 @@ class CwDeepDecoder(
* A capture chunk is ~100 ms, which is 4410 samples at the 44.1 kHz capture
* rate but only 320 after resampling to 3200 Hz. Gating on a single chunk
* reaching this size would therefore never fire, so chunks are accumulated in
* [detectBuffer] until enough audio is available.
* [detectionPool] until enough audio is available.
*/
const val DETECT_MIN_SAMPLES = 1280
@@ -132,20 +133,21 @@ class CwDeepDecoder(
/** Shift currently applied to incoming audio; 0 when the tone needs no move. */
private var activeShiftHz = 0f
/** Last detected tone, for logging and for the pitch readout while shifting. */
private var detectedToneHz: Float? = null
/**
* Tone that produced [activeShiftHz]. Hysteresis compares against this rather than
* against the previous shift, so the guard still holds where a shift flips between
* 0 and a large value - at the window edge, where the jump is largest.
*/
private var shiftAnchorToneHz: Float? = null
/** Wall clock of the last detection scan, throttling it to [DETECT_INTERVAL_MS]. */
private var lastDetectAtMs = 0L
/**
* Accumulates resampled chunks until [DETECT_MIN_SAMPLES] is reached. A single
* capture chunk is only 320 samples once resampled, so detection has to pool
* several of them.
* Pools resampled chunks until [DETECT_MIN_SAMPLES] is reached. A single capture
* chunk is only 320 samples once resampled, so detection has to pool several.
*/
private val detectBuffer = FloatArray(DETECT_MIN_SAMPLES)
private var detectFill = 0
private var detectWriteIndex = 0
private val detectionPool = CwDetectionPool(DETECT_MIN_SAMPLES)
/** Carries Hilbert filter history and mixer phase across capture chunks. */
private val streamingShifter = CwToneShifter.Streaming()
@@ -326,22 +328,21 @@ class CwDeepDecoder(
Log.i(TAG, "toneShift: setting changed to $enabled, dropping buffered audio")
dropBufferedAudio()
activeShiftHz = 0f
detectedToneHz = null
shiftAnchorToneHz = null
lastDetectAtMs = 0L
detectFill = 0
detectWriteIndex = 0
detectionPool.clear()
streamingShifter.reset()
}
if (!enabled) return resampled
accumulateForDetection(resampled)
detectionPool.add(resampled)
val now = System.currentTimeMillis()
val elapsed = now - lastDetectAtMs
if (detectFill >= DETECT_MIN_SAMPLES && elapsed >= DETECT_INTERVAL_MS) {
if (detectionPool.isReady && elapsed >= DETECT_INTERVAL_MS) {
lastDetectAtMs = now
runDetection(drainDetectionBuffer())
runDetection(detectionPool.drain())
}
// Streaming keeps the Hilbert filter history and mixer phase across chunks;
@@ -349,39 +350,6 @@ class CwDeepDecoder(
return streamingShifter.process(resampled, activeShiftHz, CwDeepSpectrogram.SAMPLE_RATE)
}
/**
* Collect resampled chunks until [DETECT_MIN_SAMPLES] is available.
*
* A ring buffer rather than a sliding array: detection is throttled to
* [DETECT_INTERVAL_MS] but the buffer fills in 400 ms, so for the remaining 1.6 s
* every chunk arrives at a full buffer. Shifting the array down one slot per sample
* cost 320 copies of 1280 floats per chunk - measured at 24320 whole-array moves per
* 10 s of audio, on the capture thread. Overwriting the oldest slot is O(1).
*/
private fun accumulateForDetection(chunk: FloatArray) {
if (chunk.isEmpty()) return
// A chunk longer than the buffer can only contribute its tail.
val start = maxOf(0, chunk.size - detectBuffer.size)
for (i in start until chunk.size) {
detectBuffer[detectWriteIndex] = chunk[i]
detectWriteIndex = (detectWriteIndex + 1) % detectBuffer.size
if (detectFill < detectBuffer.size) detectFill++
}
}
/** Copy the buffered audio out in chronological order, oldest sample first. */
private fun drainDetectionBuffer(): FloatArray {
val out = FloatArray(detectFill)
// When full, the oldest sample sits at the write cursor; otherwise at index 0.
val oldest = if (detectFill == detectBuffer.size) detectWriteIndex else 0
for (i in 0 until detectFill) {
out[i] = detectBuffer[(oldest + i) % detectBuffer.size]
}
detectFill = 0
detectWriteIndex = 0
return out
}
/**
* Discard buffered audio that was shifted by a now-stale amount.
*
@@ -399,30 +367,52 @@ class CwDeepDecoder(
val analysis = CwToneShifter.analyse(sample, CwDeepSpectrogram.SAMPLE_RATE)
val previousShift = activeShiftHz
// Ignore small changes. Dropping the window costs 20 s of context, so a tone
// wandering by a few Hz - or a detector estimate landing on an adjacent 12.5 Hz
// scan bin - must not keep wiping it. Re-shifting only pays off once the tone
// has moved enough to matter against the 800 Hz window centre.
val keepPreviousShift = previousShift != 0f &&
analysis.needsShift &&
abs(analysis.shiftHz - previousShift) < SHIFT_HYSTERESIS_HZ
detectedToneHz = analysis.toneHz
if (!keepPreviousShift) activeShiftHz = analysis.shiftHz
when {
analysis.toneHz == null ->
Log.d(TAG, "toneShift: no tone in ${sample.size} samples, shift stays 0")
!analysis.needsShift -> Log.d(
// Silence is absence of evidence, not evidence of a 0 Hz shift. CW keying leaves
// gaps, and a detection window landing in one used to collapse an established
// shift: measured over 180 s of keyed audio at 1400 Hz, 11 of 90 detections saw
// no tone, each wiping the window and leaving the next ~2 s buffered unshifted -
// outside the model's range, so invisible to it.
val toneHz = analysis.toneHz
if (toneHz == null) {
Log.d(
TAG,
"toneShift: tone=${analysis.toneHz}Hz inside " +
"toneShift: no tone in ${sample.size} samples, keeping shift=${previousShift}Hz"
)
return
}
// Hysteresis in tone space, anchored on the pitch that produced the active shift.
// Comparing shifts instead let the guard lapse exactly where the jump is largest:
// at the window edge one 12.5 Hz estimate hop flips between "inside" (shift 0) and
// "outside" (a large shift), and a shift of 0 is a real state rather than no state.
// Measured before this change: a 1205 Hz tone dropped the window 35 times in 60
// detections. The tone must now clear the edge by the margin before the decoder
// changes its mind.
val anchorTone = shiftAnchorToneHz
if (anchorTone != null && abs(toneHz - anchorTone) < SHIFT_HYSTERESIS_HZ) {
// Say so explicitly: a log showing a drifting tone against an unchanged shift
// otherwise looks like the detector is being ignored.
Log.d(
TAG,
"toneShift: tone=${toneHz}Hz within ${SHIFT_HYSTERESIS_HZ}Hz of " +
"${anchorTone}Hz, keeping shift=${previousShift}Hz"
)
return
}
activeShiftHz = analysis.shiftHz
shiftAnchorToneHz = toneHz
if (!analysis.needsShift) {
Log.d(
TAG,
"toneShift: tone=${toneHz}Hz inside " +
"${CwDeepSpectrogram.MIN_FREQ_HZ}-${CwDeepSpectrogram.MAX_FREQ_HZ}Hz, no shift"
)
else -> Log.i(
} else {
Log.i(
TAG,
"toneShift: tone=${analysis.toneHz}Hz outside window, " +
"toneShift: tone=${toneHz}Hz outside window, " +
"shifting ${analysis.shiftHz}Hz to ${CwToneShifter.TARGET_HZ}Hz"
)
}
@@ -545,10 +535,9 @@ class CwDeepDecoder(
_lastInferenceMs.value = 0
// Re-detect from scratch: the operator may have retuned before resetting.
activeShiftHz = 0f
detectedToneHz = null
shiftAnchorToneHz = null
lastDetectAtMs = 0L
detectFill = 0
detectWriteIndex = 0
detectionPool.clear()
streamingShifter.reset()
// Leave toneShiftWasEnabled unset so the next chunk re-seeds it from the
// current setting instead of reporting a spurious change.
core/data/src/test/java/com/rtbishop/look4sat/core/data/cw/CwToneShiftGateTest.kt
-64
View File
@@ -162,68 +162,4 @@ class CwToneShiftGateTest {
)
}
/**
* The detection pool must hand the analyser the most recent audio in chronological
* order. The decoder implements this as a ring buffer because detection is throttled
* to 2 s while the pool fills in 400 ms, so most chunks arrive at a full buffer.
*
* This pins the contract with the same ring semantics the decoder uses: getting the
* order wrong would feed the detector a spliced waveform and corrupt every estimate,
* silently, which no other assertion here would notice.
*/
@Test
fun `detection pool yields the most recent samples in order`() {
val capacity = 1280 // CwDeepDecoder.DETECT_MIN_SAMPLES
val pool = FloatArray(capacity)
var fill = 0
var writeIndex = 0
fun accumulate(chunk: FloatArray) {
val start = maxOf(0, chunk.size - pool.size)
for (i in start until chunk.size) {
pool[writeIndex] = chunk[i]
writeIndex = (writeIndex + 1) % pool.size
if (fill < pool.size) fill++
}
}
fun drain(): FloatArray {
val out = FloatArray(fill)
val oldest = if (fill == pool.size) writeIndex else 0
for (i in 0 until fill) out[i] = pool[(oldest + i) % pool.size]
fill = 0
writeIndex = 0
return out
}
// Feed a monotonically increasing ramp in 320-sample chunks, past capacity.
var next = 0f
repeat(10) {
accumulate(FloatArray(320) { next++ })
}
val drained = drain()
assertEquals("a full pool must hand over exactly its capacity", capacity, drained.size)
assertEquals(
"the pool must end on the newest sample fed",
next - 1f, drained.last(), 0f
)
assertEquals(
"the pool must start capacity-1 samples before the newest",
next - capacity, drained.first(), 0f
)
for (i in 1 until drained.size) {
assertEquals(
"sample $i is out of order, so the ring wrap is wrong",
drained[i - 1] + 1f, drained[i], 0f
)
}
// A short run must come back whole, without stale slots from the previous pass.
accumulate(FloatArray(320) { 9000f + it })
val partial = drain()
assertEquals("a partial pool must not report capacity", 320, partial.size)
assertEquals(9000f, partial.first(), 0f)
assertEquals(9319f, partial.last(), 0f)
}
}