feat(qrz): tell an expired cookie apart from a station with no grid
The grid lookup returned String? and swallowed everything with catch { null }, so a
timeout, an expired cookie, a QRZ layout change and a station that simply has not
published a locator were one indistinguishable blank. The operator saw an empty grid
with no way to know that re-pasting their cookie would fix it. There was also no
retry at all, on a phone, mid-pass, on mobile data.
QrzGrid names the four outcomes and QrzGridParser holds the parsing, which is pure
string work and now testable without a network. The fetch moves to core:data as
QrzGridSource, using the project's own OkHttp client with three attempts and 700ms
then 2000ms of backoff. Only transport failures and 5xx are retried; a 4xx would
repeat identically. This also gets java.net.URL I/O out of core:domain, which that
module is meant to stay clear of for the KMP move.
Classifying signed-out took two goes. Keying on the detail table being absent held
for an expired cookie - QRZ genuinely serves no detail rows to an anonymous visitor,
verified against a live response - but an audit found that a callsign QRZ has never
heard of returns HTTP 200 with no detail rows either, because QRZ serves its search
form instead. That would have reported a mistyped callsign as an expired cookie and
sent the operator into settings mid-pass to re-paste one that was never broken. It
now keys on QRZ's own "Login is required for additional detail" notice, so an absent
locator degrades to the harmless outcome and only QRZ actually asking for a login
triggers the cookie prompt. All three cases are measured against live responses.
Not yet wired in: LogTab and SettingsScreen still call the old QrzGridClient, so
nothing changes for the operator yet. Cutting over needs an interface in core:domain
and a MainContainer provider, because feature modules cannot reach core:data
directly - and the cookie itself belongs in SettingsRepo rather than the separate
prefs file a composable currently reads through LocalContext.
This commit is contained in:
1 parent
38f939bd49
commit
ce68f48765
3 files changed
+340
No files matched your search
@@ -0,0 +1,100 @@
|
||||
/*
|
||||
* Look4Sat. Amateur radio satellite tracker and pass predictor.
|
||||
* Copyright (C) 2019-2026 Arty Bishop and contributors.
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
package com.rtbishop.look4sat.core.data.qrz
|
||||
|
||||
import com.rtbishop.look4sat.core.domain.qrz.QrzGrid
|
||||
import com.rtbishop.look4sat.core.domain.qrz.QrzGridParser
|
||||
import kotlinx.coroutines.CancellationException
|
||||
import kotlinx.coroutines.CoroutineDispatcher
|
||||
import kotlinx.coroutines.delay
|
||||
import kotlinx.coroutines.withContext
|
||||
import okhttp3.OkHttpClient
|
||||
import okhttp3.Request
|
||||
|
||||
/**
|
||||
* Reads a station's Maidenhead locator off its QRZ.com page.
|
||||
*
|
||||
* QRZ has no free lookup API for this, so the page is fetched with the operator's own session
|
||||
* cookie and parsed. The cookie is pasted by the operator in settings and never built into the
|
||||
* app. Parsing lives in [QrzGridParser] so it can be tested without a network; this class only
|
||||
* fetches and retries.
|
||||
*/
|
||||
class QrzGridSource(
|
||||
private val httpClient: OkHttpClient,
|
||||
private val dispatcher: CoroutineDispatcher
|
||||
) {
|
||||
|
||||
/**
|
||||
* Look up [callsign]'s locator.
|
||||
*
|
||||
* Retried because this runs on a phone, mid-pass, often on mobile data - a single timeout
|
||||
* used to mean the QSO was logged without a grid and the operator was never told. Retries
|
||||
* are bounded and backed off so a genuinely unreachable QRZ costs at most a few seconds:
|
||||
* only transport failures are retried, since a page that loaded and parsed will not parse
|
||||
* differently on a second attempt.
|
||||
*/
|
||||
suspend fun lookupGrid(callsign: String, cookieHeader: String): QrzGrid =
|
||||
withContext(dispatcher) {
|
||||
if (callsign.isBlank() || cookieHeader.isBlank()) return@withContext QrzGrid.SignedOut
|
||||
val url = "$DB_URL${callsign.trim().uppercase()}"
|
||||
fetchWithRetry(url, cookieHeader)?.let(QrzGridParser::parseGrid)
|
||||
?: QrzGrid.Unreachable(MAX_ATTEMPTS)
|
||||
}
|
||||
|
||||
/**
|
||||
* The callsign the pasted cookie is signed in as, so settings can show the operator whose
|
||||
* account it belongs to rather than just claiming success.
|
||||
*/
|
||||
suspend fun lookupOwnCallsign(cookieHeader: String): String? = withContext(dispatcher) {
|
||||
if (cookieHeader.isBlank()) return@withContext null
|
||||
fetchWithRetry(DB_URL, cookieHeader)?.let(QrzGridParser::parseOwnCallsign)
|
||||
}
|
||||
|
||||
/** Fetch [url], retrying transport failures with backoff. Null when every attempt failed. */
|
||||
private suspend fun fetchWithRetry(url: String, cookieHeader: String): String? {
|
||||
repeat(MAX_ATTEMPTS) { attempt ->
|
||||
try {
|
||||
val request = Request.Builder().url(url)
|
||||
.header("User-Agent", USER_AGENT)
|
||||
.header("Cookie", cookieHeader)
|
||||
.build()
|
||||
httpClient.newCall(request).execute().use { response ->
|
||||
if (response.isSuccessful) return response.body.string()
|
||||
// A 4xx will repeat identically, so only server-side faults are worth retrying.
|
||||
if (response.code < 500) return null
|
||||
}
|
||||
} catch (exception: CancellationException) {
|
||||
throw exception
|
||||
} catch (exception: Exception) {
|
||||
println("QrzGridSource attempt ${attempt + 1} failed: $exception")
|
||||
}
|
||||
if (attempt < MAX_ATTEMPTS - 1) delay(BACKOFF_MS[attempt])
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
private companion object {
|
||||
/** Bare form is the signed-in home page; a callsign appended is that station's page. */
|
||||
const val DB_URL = "https://www.qrz.com/db/"
|
||||
const val USER_AGENT = "Mozilla/5.0 (Linux; Android 13) Look4Sat"
|
||||
const val MAX_ATTEMPTS = 3
|
||||
|
||||
/** Waits before the second and third attempt. Short enough to finish inside a pass. */
|
||||
val BACKOFF_MS = longArrayOf(700L, 2_000L)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,107 @@
|
||||
/*
|
||||
* Look4Sat. Amateur radio satellite tracker and pass predictor.
|
||||
* Copyright (C) 2019-2026 Arty Bishop and contributors.
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU General Public License as published by
|
||||
* the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
package com.rtbishop.look4sat.core.domain.qrz
|
||||
|
||||
/**
|
||||
* Outcome of a QRZ grid lookup.
|
||||
*
|
||||
* Four outcomes rather than a nullable string, because the previous null meant any of "the
|
||||
* station has no grid on file", "the cookie expired", "the request timed out" and "QRZ changed
|
||||
* its markup" - and the operator saw the same blank either way, with no way to tell that
|
||||
* re-pasting the cookie would fix it.
|
||||
*/
|
||||
sealed interface QrzGrid {
|
||||
|
||||
/** The station's Maidenhead locator, as QRZ has it. */
|
||||
data class Found(val locator: String) : QrzGrid
|
||||
|
||||
/** The page was read and the station has no locator published. Not an error. */
|
||||
data object NotOnFile : QrzGrid
|
||||
|
||||
/** The detail table was absent, which is what QRZ serves when the cookie is not valid. */
|
||||
data object SignedOut : QrzGrid
|
||||
|
||||
/** The request never completed. [attempts] is how many tries were made before giving up. */
|
||||
data class Unreachable(val attempts: Int) : QrzGrid
|
||||
}
|
||||
|
||||
/**
|
||||
* Parsing of QRZ's callsign page, separate from the fetch so it can be tested without a
|
||||
* network. Pure string work over already-downloaded markup.
|
||||
*/
|
||||
object QrzGridParser {
|
||||
|
||||
/** The detail row QRZ renders for a station that published a locator. */
|
||||
private val gridRow = Regex("""<td class="dh">Grid Square</td>\s*<td class="di">([^<]+)</td>""")
|
||||
|
||||
/**
|
||||
* QRZ's own words on a callsign page served to a visitor who is not signed in.
|
||||
*
|
||||
* Classified on this positive notice rather than on the detail table being absent: measured
|
||||
* against live responses, a callsign QRZ has never heard of also returns HTTP 200 with zero
|
||||
* detail rows, because QRZ serves its search form instead of a callsign page. Keying on
|
||||
* absence therefore reported a mistyped callsign as an expired cookie, and would have sent
|
||||
* the operator off to re-paste a cookie that was never broken.
|
||||
*/
|
||||
private val signedOutNotice = Regex("""Login is required for additional detail""")
|
||||
|
||||
/** The account menu on a signed-in page, used to read back whose cookie this is. */
|
||||
private val accountCallsign = Regex("""<li class="leaf last"[^>]*>\s*([A-Z0-9/]+)\s*<ul""")
|
||||
|
||||
/** A cookie name=value pair inside a browser extension's JSON export. */
|
||||
private val jsonCookie = Regex(""""name"\s*:\s*"([^"]+)"\s*,\s*"value"\s*:\s*"([^"]*)"""")
|
||||
|
||||
/**
|
||||
* Interpret a callsign page.
|
||||
*
|
||||
* Only QRZ explicitly saying that a login is required counts as signed out, so an absent
|
||||
* locator degrades to the harmless [QrzGrid.NotOnFile] and only a genuinely stale cookie
|
||||
* sends the operator back to settings. Getting this wrong in either direction misdirects
|
||||
* them: the old client returned null for everything, and keying on the detail table being
|
||||
* absent would have blamed the cookie for a mistyped callsign.
|
||||
*/
|
||||
fun parseGrid(html: String): QrzGrid {
|
||||
val locator = gridRow.find(html)?.groupValues?.get(1)?.trim()
|
||||
if (!locator.isNullOrBlank()) return QrzGrid.Found(locator)
|
||||
if (signedOutNotice.containsMatchIn(html)) return QrzGrid.SignedOut
|
||||
return QrzGrid.NotOnFile
|
||||
}
|
||||
|
||||
/** The callsign this cookie is signed in as, or null when it is not signed in. */
|
||||
fun parseOwnCallsign(html: String): String? =
|
||||
accountCallsign.find(html)?.groupValues?.get(1)?.trim()?.takeIf { it.isNotBlank() }
|
||||
|
||||
/**
|
||||
* Normalise the pasted cookie into a Cookie header value.
|
||||
*
|
||||
* Accepts a raw `k=v; k=v` header or the JSON array a cookie-export extension produces,
|
||||
* since the operator pastes whatever their browser handed them. Parsed by regex rather
|
||||
* than a JSON library because org.json is compileOnly here - it is supplied by Android at
|
||||
* runtime and absent from unit tests, so a JSON path could not be tested.
|
||||
*/
|
||||
fun cookieHeader(raw: String): String {
|
||||
val text = raw.trim()
|
||||
if (text.isEmpty()) return ""
|
||||
if (!text.startsWith("[")) return text
|
||||
val pairs = jsonCookie.findAll(text)
|
||||
.map { it.groupValues[1] to it.groupValues[2] }
|
||||
.filter { it.first.isNotBlank() }
|
||||
.toList()
|
||||
return if (pairs.isEmpty()) text else pairs.joinToString("; ") { "${it.first}=${it.second}" }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
package com.rtbishop.look4sat.core.domain.qrz
|
||||
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertNull
|
||||
import org.junit.Test
|
||||
|
||||
class QrzGridParserTest {
|
||||
|
||||
/** A signed-in page for a station that published a locator. */
|
||||
private val withGrid = """
|
||||
<table class="detail">
|
||||
<tr><td class="dh">Callsign</td><td class="di">BG7NTA</td></tr>
|
||||
<tr><td class="dh">Grid Square</td><td class="di">OL72</td></tr>
|
||||
<tr><td class="dh">Country</td><td class="di">China</td></tr>
|
||||
</table>
|
||||
""".trimIndent()
|
||||
|
||||
/** Signed in, but this station has no Grid Square row at all. */
|
||||
private val withoutGrid = """
|
||||
<table class="detail">
|
||||
<tr><td class="dh">Callsign</td><td class="di">W1AW</td></tr>
|
||||
<tr><td class="dh">Country</td><td class="di">United States</td></tr>
|
||||
</table>
|
||||
""".trimIndent()
|
||||
|
||||
/**
|
||||
* What QRZ actually serves for a real callsign when nobody is signed in: no detail table,
|
||||
* plus its own notice saying why. Transcribed from a live response rather than invented.
|
||||
*/
|
||||
private val signedOut = """
|
||||
<div class="csgn">W1AW</div>
|
||||
<div>Login is required for additional detail.</div>
|
||||
<div>Email: Login required to view</div>
|
||||
""".trimIndent()
|
||||
|
||||
/**
|
||||
* A callsign QRZ has never heard of. Also HTTP 200, also zero detail rows, but no login
|
||||
* notice - QRZ serves its search form instead of a callsign page.
|
||||
*/
|
||||
private val unknownCallsign = """
|
||||
<div class="search">The available search types are callsign, name, address.</div>
|
||||
""".trimIndent()
|
||||
|
||||
@Test
|
||||
fun `reads the locator when the station published one`() {
|
||||
assertEquals(QrzGrid.Found("OL72"), QrzGridParser.parseGrid(withGrid))
|
||||
}
|
||||
|
||||
/**
|
||||
* The distinction that matters: a station with no grid must not look like an expired
|
||||
* cookie, because the fix for one is nothing and the fix for the other is re-pasting it.
|
||||
*/
|
||||
@Test
|
||||
fun `a station with no grid is not confused with being signed out`() {
|
||||
assertEquals(QrzGrid.NotOnFile, QrzGridParser.parseGrid(withoutGrid))
|
||||
assertEquals(QrzGrid.SignedOut, QrzGridParser.parseGrid(signedOut))
|
||||
}
|
||||
|
||||
/**
|
||||
* The cookie prompt must fire only when the cookie is the problem. Measured against live
|
||||
* responses, a mistyped callsign returns a page with no detail rows either - so classifying
|
||||
* on their absence would blame the cookie and send the operator off to re-paste a working
|
||||
* one, mid-pass, over a satellite that is about to set.
|
||||
*/
|
||||
@Test
|
||||
fun `an unknown callsign does not read as an expired cookie`() {
|
||||
assertEquals(QrzGrid.NotOnFile, QrzGridParser.parseGrid(unknownCallsign))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `an empty grid cell counts as not on file`() {
|
||||
val blank = withGrid.replace(">OL72<", "><")
|
||||
assertEquals(QrzGrid.NotOnFile, QrzGridParser.parseGrid(blank))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `whitespace around the locator is trimmed`() {
|
||||
val padded = withGrid.replace(">OL72<", "> OL72 <")
|
||||
assertEquals(QrzGrid.Found("OL72"), QrzGridParser.parseGrid(padded))
|
||||
}
|
||||
|
||||
/** Six-character locators are as common as four; nothing may truncate them. */
|
||||
@Test
|
||||
fun `a six character locator survives intact`() {
|
||||
val six = withGrid.replace(">OL72<", ">OL72ab<")
|
||||
assertEquals(QrzGrid.Found("OL72ab"), QrzGridParser.parseGrid(six))
|
||||
}
|
||||
|
||||
/** QRZ puts the two cells adjacent, but whitespace between them must not break the match. */
|
||||
@Test
|
||||
fun `whitespace between the two table cells is tolerated`() {
|
||||
val spaced = withGrid.replace(
|
||||
"<td class=\"dh\">Grid Square</td><td class=\"di\">OL72</td>",
|
||||
"<td class=\"dh\">Grid Square</td>\n <td class=\"di\">OL72</td>"
|
||||
)
|
||||
assertEquals(QrzGrid.Found("OL72"), QrzGridParser.parseGrid(spaced))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `reads back which callsign the cookie belongs to`() {
|
||||
val menu = """<li class="leaf last" onclick="return true">BG7NTA <ul class="sub">"""
|
||||
assertEquals("BG7NTA", QrzGridParser.parseOwnCallsign(menu))
|
||||
assertNull(QrzGridParser.parseOwnCallsign(signedOut))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `a raw cookie header is passed through unchanged`() {
|
||||
val raw = "qz_userid=1266043; qz_sess=abc123"
|
||||
assertEquals(raw, QrzGridParser.cookieHeader(raw))
|
||||
}
|
||||
|
||||
/** The shape a cookie-export extension produces. */
|
||||
@Test
|
||||
fun `a json cookie export is flattened into a header`() {
|
||||
val json = """
|
||||
[{"domain":".qrz.com","name":"qz_userid","value":"1266043"},
|
||||
{"domain":".qrz.com","name":"qz_sess","value":"abc123"}]
|
||||
""".trimIndent()
|
||||
assertEquals("qz_userid=1266043; qz_sess=abc123", QrzGridParser.cookieHeader(json))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `an empty cookie yields an empty header`() {
|
||||
assertEquals("", QrzGridParser.cookieHeader(" "))
|
||||
}
|
||||
|
||||
/** Malformed JSON is handed over as-is rather than silently becoming empty. */
|
||||
@Test
|
||||
fun `unparseable json is passed through rather than dropped`() {
|
||||
val broken = """[{"nope":"nothing here"}]"""
|
||||
assertEquals(broken, QrzGridParser.cookieHeader(broken))
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user