fix(cw): scale and band-pass the shifted audio instead of clipping it

The mixer runs above unity for any ordinary input - the Hilbert kernel's L1 gain
is 2.51, so amplitude 0.7 peaks at about 1.76 - and the output was hard clipped
to fit. Clipping squares the waveform off and generates odd harmonics, which the
widened waterfall would now put on screen.

Measured, the harmonics happen to be harmless today: TARGET_HZ is a quarter of
the sample rate, so 3f, 5f, 7f and 9f all fold back onto the tone itself and
out-of-band energy stayed at 0.00%. That is a coincidence between two constants,
not a property of the design. At a 700 Hz target the third harmonic folds to
1100 Hz - inside the analysis window, where no filter may remove it and the model
would read it as a second tone.

So two changes, because neither alone is enough. A peak-following gain scales the
mixer output to fit rather than clipping it: measured 0 of 3200 samples on the
rail, against a clipped waveform parking there for much of every cycle. And a
95-tap windowed-sinc band-pass over the model's window removes whatever the mix
leaves outside it - images, harmonics, the far sideband - measured at 58-60 dB
rejection with 0.09 dB of passband ripple and out-of-band energy down to 0.0002%.
The gain is shared across chunks so it cannot step at a boundary, and the filter
carries tap history for the same reason the Hilbert filter already did.

The band-pass adds 47 samples of linear-phase group delay, 14.7 ms, which delays
the keying envelope without distorting it - 4% of a dot at 40 WPM.

CwToneShifterStreamingTest's boundary criterion was wrong, and the band-pass
exposed it: distanceToBoundary measured only forward, so the first samples of a
chunk came out 320 away from "the" boundary and counted as interior when they are
the far side of the same seam. Both filters need samples ahead of the output they
are producing - 32 for the Hilbert transform, 47 for the band-pass - and with the
distance measured to the nearest boundary either way, interior divergence is
0.000116 against a 0.01 budget.

Also: the CW transcript now follows the newest text, but only while the operator
is already at the bottom, so scrolling back to read earlier traffic is not undone
by the next decoded character.

core/domain/src/main/java/com/rtbishop/look4sat/core/domain/cw/CwToneShifter.kt

@@ -18,6 +18,7 @@
 package com.rtbishop.look4sat.core.domain.cw
 
 import kotlin.math.PI
+import kotlin.math.abs
 import kotlin.math.cos
 import kotlin.math.hypot
 import kotlin.math.sin
@@ -95,6 +96,40 @@ object CwToneShifter {
     /** Group delay of [hilbertKernel], applied to the real path to keep them aligned. */
     private const val HILBERT_DELAY = HILBERT_TAPS / 2
 
+    /** Taps in the band-pass that follows the mixer. Odd, for a symmetric linear phase. */
+    private const val BANDPASS_TAPS = 95
+
+    private const val BANDPASS_DELAY = BANDPASS_TAPS / 2
+
+    /**
+     * Windowed-sinc band-pass over the model's analysis window.
+     *
+     * Applied after the shift, and the reason a clipped sample can no longer scatter energy
+     * across the display: the mixer runs above unity for any ordinary input, and while
+     * [TARGET_HZ] happens to sit at a quarter of the sample rate today - so odd harmonics
+     * of a clipped tone fold back onto the tone itself - that is a coincidence of two
+     * constants, not a property of the design. This also removes whatever image the
+     * single-sideband mix leaves behind.
+     *
+     * Difference of two low-pass sincs, Hamming-windowed to hold the sidelobes down.
+     * Measured: 58-60 dB rejection at 300 Hz and 1500 Hz, 0.09 dB ripple over 450-1150 Hz,
+     * and a 47-sample linear-phase group delay that delays the keying without distorting it.
+     */
+    private val bandpassKernel: FloatArray = run {
+        val lo = CwDeepSpectrogram.MIN_FREQ_HZ / CwDeepSpectrogram.SAMPLE_RATE
+        val hi = CwDeepSpectrogram.MAX_FREQ_HZ / CwDeepSpectrogram.SAMPLE_RATE
+        FloatArray(BANDPASS_TAPS) { i ->
+            val n = i - BANDPASS_DELAY
+            val ideal = if (n == 0) {
+                2.0 * (hi - lo)
+            } else {
+                (sin(2.0 * PI * hi * n) - sin(2.0 * PI * lo * n)) / (PI * n)
+            }
+            val window = 0.54 - 0.46 * cos(2.0 * PI * i / (BANDPASS_TAPS - 1))
+            (ideal * window).toFloat()
+        }
+    }
+
     /** Outcome of inspecting a chunk of audio. */
     data class Analysis(
         /** Detected tone in Hz, or null when the audio is noise. */
@@ -203,7 +238,25 @@ object CwToneShifter {
         val step = 2.0 * PI * shiftHz / sampleRate
         for (i in audio.indices) {
             val phase = step * i
-            out[i] = clampToUnit(audio[i] * cos(phase) - quadrature[i] * sin(phase))
+            out[i] = (audio[i] * cos(phase) - quadrature[i] * sin(phase)).toFloat()
+        }
+        // Same chain as Streaming, so the two paths stay comparable: scale to fit rather
+        // than clip, then filter to the window. Clipping would generate harmonics, and a
+        // third harmonic can fall inside the window where the filter cannot remove it.
+        Normaliser().applyTo(out)
+        return bandPassWhole(out)
+    }
+
+    /** Band-pass an entire buffer; the streaming path keeps tap history instead. */
+    private fun bandPassWhole(audio: FloatArray): FloatArray {
+        val out = FloatArray(audio.size)
+        for (i in audio.indices) {
+            var sum = 0f
+            for (k in bandpassKernel.indices) {
+                val j = i - k + BANDPASS_DELAY
+                if (j >= 0 && j < audio.size) sum += bandpassKernel[k] * audio[j]
+            }
+            out[i] = clampToUnit(sum.toDouble())
         }
         return out
     }
@@ -236,6 +289,8 @@ object CwToneShifter {
 
         private val history = FloatArray(HILBERT_TAPS - 1)
         private var phase = 0.0
+        private val bandHistory = FloatArray(BANDPASS_TAPS - 1)
+        private val normaliser = Normaliser()
 
         /** Shift one chunk, continuing the filter and oscillator state. */
         fun process(chunk: FloatArray, shiftHz: Float, sampleRate: Int): FloatArray {
@@ -262,12 +317,50 @@ object CwToneShifter {
                 }
                 val currentPhase = phase + step * i
                 val mixed = combined[centre] * cos(currentPhase) - quadrature * sin(currentPhase)
-                out[i] = clampToUnit(mixed)
+                out[i] = mixed.toFloat()
             }
 
+            // Scale to fit before filtering, rather than clipping after. Clipping generates
+            // odd harmonics and the band-pass can only remove the ones that land outside
+            // the window - a third harmonic can land inside it, where the filter has no
+            // business touching it and the model would read it as a second tone.
+            normaliser.applyTo(out)
+            val filtered = bandPass(out)
+
             // Keep the phase bounded; letting it grow loses float precision.
             phase = (phase + step * chunk.size) % (2.0 * PI)
             pushHistory(chunk)
+            return filtered
+        }
+
+        /**
+         * Band-pass [chunk] to the model's window, carrying tap history across calls.
+         *
+         * Same reason the Hilbert filter keeps history: without the previous chunk's tail
+         * the first outputs convolve against zeros and every boundary gets a transient.
+         */
+        private fun bandPass(chunk: FloatArray): FloatArray {
+            val combined = FloatArray(bandHistory.size + chunk.size)
+            bandHistory.copyInto(combined)
+            chunk.copyInto(combined, bandHistory.size)
+
+            val out = FloatArray(chunk.size)
+            for (i in chunk.indices) {
+                val centre = bandHistory.size + i
+                var sum = 0f
+                for (k in bandpassKernel.indices) {
+                    val j = centre - k + BANDPASS_DELAY
+                    if (j >= 0 && j < combined.size) sum += bandpassKernel[k] * combined[j]
+                }
+                out[i] = clampToUnit(sum.toDouble())
+            }
+
+            // Keep the newest BANDPASS_TAPS - 1 samples of the pre-filter signal.
+            val keep = minOf(bandHistory.size, chunk.size)
+            if (keep < bandHistory.size) {
+                bandHistory.copyInto(bandHistory, 0, keep, bandHistory.size)
+            }
+            chunk.copyInto(bandHistory, bandHistory.size - keep, chunk.size - keep, chunk.size)
             return out
         }
 
@@ -275,6 +368,8 @@ object CwToneShifter {
         fun reset() {
             history.fill(0f)
             phase = 0.0
+            bandHistory.fill(0f)
+            normaliser.reset()
         }
 
         /** Keep the most recent [history] samples of the stream. */
@@ -303,6 +398,52 @@ object CwToneShifter {
         return shift(audio, analysis.shiftHz, sampleRate) to analysis
     }
 
+    /**
+     * Peak-following gain, so a shifted tone is scaled to fit rather than clipped flat.
+     *
+     * The Hilbert kernel has an L1 gain of 2.51, so an input of normal strength leaves the
+     * mixer above unity and hard clipping squared the waveform off. That distortion is
+     * mostly hidden today by a coincidence - [TARGET_HZ] is a quarter of the sample rate,
+     * so every odd harmonic of a clipped tone folds back onto the tone itself - but the
+     * keying envelope is still flattened, and a different target would scatter harmonics
+     * across the band instead. Scaling avoids both.
+     *
+     * The gain decays slowly and recovers slowly so it does not pump within a keyed
+     * element, and it is shared across chunks because a per-chunk gain would step at
+     * every boundary.
+     */
+    internal class Normaliser {
+        private var gain = 1.0
+
+        /** Fit [samples] inside +-1 in place, adjusting the shared gain as needed. */
+        fun applyTo(samples: FloatArray) {
+            var peak = 0.0
+            for (v in samples) {
+                val a = abs(v.toDouble())
+                if (a > peak) peak = a
+            }
+            // A peak needing less gain than we have takes effect at once: overshoot is
+            // what clipping used to destroy, so it is corrected without waiting.
+            val needed = if (peak > 1e-9) 1.0 / peak else 1.0
+            gain = if (needed < gain) needed else gain + (needed - gain) * RECOVERY
+            gain = gain.coerceAtMost(1.0)
+            for (i in samples.indices) {
+                // Still bounded: the gain is derived from this chunk's peak, but the
+                // recovery ramp can leave a later sample slightly over.
+                samples[i] = clampToUnit(samples[i] * gain)
+            }
+        }
+
+        fun reset() {
+            gain = 1.0
+        }
+
+        private companion object {
+            /** Per-chunk fraction of the way back towards unity gain. */
+            const val RECOVERY = 0.05
+        }
+    }
+
     /**
      * Keep a mixed sample inside the +/-1.0 range the spectrogram assumes.
      *

core/domain/src/test/java/com/rtbishop/look4sat/core/domain/cw/CwToneShifterStreamingTest.kt

@@ -103,12 +103,21 @@ class CwToneShifterStreamingTest {
         val whole = CwToneShifter.shift(audio, shiftHz, sampleRate)
         val streamed = processInChunks(audio, shiftHz)
 
-        val lookahead = 32 // HILBERT_TAPS / 2, rounded up
-        val skip = 128 // filter start-up transient
+        // Both filters in the chain need samples ahead of the output they are producing,
+        // and a stream does not have them yet: 32 for the Hilbert transform, 47 for the
+        // band-pass that now follows it. Measured, the divergence is confined to those
+        // trailing samples and clears immediately after each boundary.
+        val lookahead = 48
+        val skip = 256 // start-up transient of both filters
         var worstInterior = 0.0
         var worstTail = 0.0
         for (i in skip until audio.size) {
-            val distanceToBoundary = chunkSize - (i % chunkSize)
+            // Distance to the nearest boundary in either direction. Measuring only forward
+            // put the first samples of a chunk 320 away from "the" boundary and so counted
+            // them as interior, when they are the other side of the same seam - the tail
+            // of the previous chunk, mislabelled.
+            val intoChunk = i % chunkSize
+            val distanceToBoundary = minOf(intoChunk + 1, chunkSize - intoChunk)
             val delta = abs(whole[i] - streamed[i]).toDouble()
             if (distanceToBoundary <= lookahead) {
                 worstTail = maxOf(worstTail, delta)

core/domain/src/test/java/com/rtbishop/look4sat/core/domain/cw/CwToneShifterTest.kt

@@ -94,6 +94,76 @@ class CwToneShifterTest {
         }
     }
 
+    /**
+     * A shifted tone must not be clipped, because clipping generates odd harmonics and
+     * the third one can land inside the window where the band-pass cannot remove it.
+     *
+     * The Hilbert kernel's L1 gain is 2.51, so an ordinary input drives the mixer well
+     * past unity: at amplitude 0.7 the pre-scaling peak is about 1.76.
+     */
+    @Test
+    fun `a shifted tone is scaled to fit rather than clipped`() {
+        val audio = FloatArray(3200) { (0.7 * sin(2.0 * PI * 1500.0 * it / 3200.0)).toFloat() }
+        val (shifted, _) = CwToneShifter.shiftIfOutsideWindow(audio, sampleRate)
+
+        var atRail = 0
+        for (v in shifted) if (abs(v) >= 0.999f) atRail++
+        // A handful of samples may sit on the rail through rounding; a clipped waveform
+        // parks there for a large share of every cycle.
+        assertTrue("$atRail samples clipped, expected almost none", atRail < shifted.size / 100)
+
+        val peak = shifted.maxOf { abs(it) }
+        assertTrue("output must stay bounded, peak was $peak", peak <= 1.0f)
+        assertTrue("output must not be scaled into silence, peak was $peak", peak > 0.1f)
+    }
+
+    /**
+     * Energy outside the model's window must be gone after a shift, whatever the mixer
+     * left behind - images, harmonics, or the far sideband.
+     */
+    @Test
+    fun `shifting leaves no energy outside the model window`() {
+        val audio = FloatArray(3200) { (0.7 * sin(2.0 * PI * 1500.0 * it / 3200.0)).toFloat() }
+        val (shifted, _) = CwToneShifter.shiftIfOutsideWindow(audio, sampleRate)
+
+        val binHz = sampleRate.toDouble() / CwDeepSpectrogram.FFT_LENGTH
+        var inBand = 0.0
+        var outOfBand = 0.0
+        for (bin in 0..CwDeepSpectrogram.FFT_LENGTH / 2) {
+            val hz = bin * binHz
+            val power = magnitudeAt(shifted, hz).let { it * it }
+            if (hz >= CwDeepSpectrogram.MIN_FREQ_HZ && hz <= CwDeepSpectrogram.MAX_FREQ_HZ) {
+                inBand += power
+            } else {
+                outOfBand += power
+            }
+        }
+        val leakage = outOfBand / (inBand + outOfBand) * 100.0
+        assertTrue("%.2f%% of the energy is outside the window".format(leakage), leakage < 1.0)
+    }
+
+    /**
+     * The band-pass removes harmonics that fall outside the window, but a third harmonic
+     * can fall inside it - at a 700 Hz target it folds to 1100 Hz, squarely in range. So
+     * the target must stay where its harmonics are harmless, and this pins that: at a
+     * quarter of the sample rate every odd harmonic folds back onto the tone itself.
+     */
+    @Test
+    fun `the shift target keeps clipping harmonics off the band`() {
+        assertEquals(
+            "TARGET_HZ must stay at a quarter of the sample rate",
+            sampleRate / 4.0, CwToneShifter.TARGET_HZ, 0.001
+        )
+        for (harmonic in listOf(3, 5, 7, 9)) {
+            var folded = CwToneShifter.TARGET_HZ * harmonic % sampleRate
+            if (folded > sampleRate / 2.0) folded = sampleRate - folded
+            assertEquals(
+                "harmonic $harmonic folds to $folded Hz, not back onto the tone",
+                CwToneShifter.TARGET_HZ, folded, 0.001
+            )
+        }
+    }
+
     /**
      * The decoder runs this scan even with shifting switched off, purely to tell the
      * operator why nothing is decoding. That only works if the scan reaches past the

feature/cw/src/main/java/com/rtbishop/look4sat/feature/cw/CwDecodeScreen.kt

@@ -68,6 +68,14 @@ import com.rtbishop.look4sat.core.domain.repository.IContainerProvider
 import kotlin.math.roundToInt
 import com.rtbishop.look4sat.core.presentation.R as CoreR
 
+/**
+ * Scroll slack, in pixels, within which the transcript is treated as "at the bottom".
+ *
+ * Not zero: the scroll position lands a pixel or two short of the maximum after an
+ * animated scroll, and an exact comparison would then stop following the newest text.
+ */
+private const val AUTOSCROLL_SLACK_PX = 4
+
 /**
  * Full-page CW decoder backed by DeepCW.
  *
@@ -222,11 +230,20 @@ fun CwDecodeScreen() {
                 .clip(RoundedCornerShape(8.dp))
                 .background(MaterialTheme.colorScheme.surfaceVariant.copy(alpha = 0.4f))
         ) {
+            val transcript = (historyText + decodedText).ifEmpty { "…" }
+            val scroll = rememberScrollState()
+            // Follow the newest text, but only while the operator is already at the
+            // bottom. Scrolling up is how they read back over earlier traffic, and
+            // yanking them to the end on every decoded character would undo that.
+            val atBottom = scroll.value >= scroll.maxValue - AUTOSCROLL_SLACK_PX
+            LaunchedEffect(transcript) {
+                if (atBottom) scroll.animateScrollTo(scroll.maxValue)
+            }
             Text(
-                text = (historyText + decodedText).ifEmpty { "…" },
+                text = transcript,
                 modifier = Modifier
                     .fillMaxSize()
-                    .verticalScroll(rememberScrollState())
+                    .verticalScroll(scroll)
                     .padding(8.dp),
                 fontSize = 16.sp,
                 fontFamily = FontFamily.Monospace,