The first time I saw my family struggle to interpret the NHS and council letters, I decided to create an application that explains these letters in plain English. Government letters are unstructured data full of dates, instructions, codes, and jargon, but mostly people only need to know three things: what it’s about, when it’s happening, and what to do next. That became the starting point for LetterLens.
Purpose
I aimed to reduce the anxiety people feel when dealing with government paperwork. With LetterLens, the user simply scans the letter, and the system translates it into easy-to-understand English with clear next steps. It highlights the key actions the letter expects, so users know exactly what to do.
:::info
Disclaimer: LetterLens is an educational prototype, not an alternative for legal advice.
:::
Tech Stack (Decision-making)
- Jetpack Compose: Modern declarative UI, quick prototyping, and easier state handling.
- CameraX: Lifecycle-aware camera integration, seamless with compose.
- Ktor: Lightweight Kotlin-native backend to classify and elaborate the letter types.
- ML Kit: On-device OCR for privacy and low latency.
Under the Hood(Deep Dive)
Camerax: capture and preview
Camerax makes capturing and analyzing images. Here’s the setup to bind a preview and analyzer to the lifecycle.
val cameraProviderFuture = ProcessCameraProvider.getInstance(ctx)
cameraProviderFuture.addListener({
val provider = cameraProviderFuture.get()
val preview = Preview.Builder()
.setTargetAspectRatio(AspectRatio.RATIO_4_3) // keep 4:3
.build()
.also { it.setSurfaceProvider(view.surfaceProvider) }
provider.unbindAll()
provider.bindToLifecycle(
lifecycle,
CameraSelector.DEFAULT_BACK_CAMERA,
preview,
imageCapture
)
}, ContextCompat.getMainExecutor(ctx))
:::tip
Tip: Rotate based on EXIF and crop margins; sharper, upright images improve OCR markedly.
:::
ML Kit OCR: Extract raw text on-device
ML Kit processes the image and extracts raw text and confidence scores.
val img = InputImage.fromFilePath(context, Uri.fromFile(photo))
val rec = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)
rec.process(img)
.addOnSuccessListener { onText(it.text) }
.addOnFailureListener { e -> onError("OCR failed: ${e.message}") }
:::info
Note: I keep work entirely on-device; no letter images leave the phone.
:::
Ktor ‘explains’ endpoint: classify + extract
A small Ktor service classifies text and pulls deadlines/actions.
routing {
post("/explain") {
val req = call.receive<ExplainReq>()
val type = classify(req.text, req.hint)
val deadline = extractDeadline(req.text, type)
val (summary, actions, citations) = explainForType(type, req.text)
call.respond(ExplainRes(type, deadline, summary, actions, citations))
}
}
Keyword heuristics (examples)
-
NHS Appointment: “appointment”, “please attend”, “clinic”, “vaccination”, “CHI number”.
-
Electoral Register / Annual Canvass: “Electoral Registration Office”, “annual canvass”, “unique security code”, “register to vote”.
Data Parsing & Classification
Beyond the /explain endpoint, the core of LetterLens is its classifier. Government letters are messy—mixed fonts, spacing, codes, dates, so I added helpers for normalization, fuzzy matching, and deadline detection.
Normalization helpers:
private fun norm(s: String) = s
.lowercase()
.replace('’', ''')
.replace('–', '-')
.replace(Regex("\s+"), " ")
.trim()
Fuzzy matching (so “N H-S” still matches “NHS”):
private fun fuzzyRegex(token: String): Regex {
val letters = token.lowercase().filter { it.isLetterOrDigit() }
val pattern = letters.joinToString("\W*")
return Regex(pattern, RegexOption.IGNORE_CASE)
}
Classify by domain:
private fun classify(textRaw: String, hint: String?): String {
val n = norm("${hint ?: ""} $textRaw")
if (hasAny(n, "nhs", "appointment", "vaccination")) return "NHS Appointment"
if (hasAny(n, "electoral register", "unique security code")) return "Electoral Register"
if (hasAny(n, "council tax", "arrears")) return "Council Tax"
if (hasAny(n, "hmrc", "self assessment")) return "HMRC"
if (hasAny(n, "dvla", "vehicle tax")) return "DVLA"
if (hasAny(n, "ukvi", "visa", "biometric residence")) return "UKVI"
return "Unknown"
}
Deadline extraction (supports both 12 Sept 2025 and 12/09/2025 formats):
private fun extractDeadline(raw: String, type: String? = null): String? {
val n = norm(raw)
return DATE_DMY_SLASH.find(n)?.value ?: DATE_DMY_TEXT.find(n)?.value
}
Explain response with summary + actions + citations:
private fun explainForType(type: String, text: String): Triple<String, List<String>, List<String>> {
return when (type) {
"Electoral Register" -> Triple(
"Looks like an Electoral Register annual canvass letter.",
listOf("Go to website", "Enter unique security code", "Confirm/update household"),
listOf("https://www.gov.uk/register-to-vote")
)
"NHS Appointment" -> Triple(
"An NHS clinic invite (likely vaccination).",
listOf("Add to calendar", "Bring Red Book", "Call if reschedule needed"),
listOf("https://www.nhs.uk/nhs-services/appointments-and-bookings/")
)
else -> Triple("Generic gov letter", listOf("Read carefully", "Follow instructions"), listOf("https://www.gov.uk"))
}
}
Show an example API call/output:
Sample request:
POST /explain
{
"text": "Your NHS vaccination appointment is on 25 Sept at Glasgow Clinic. Please bring your Red Book."
}
Sample response:
{
"type": "NHS Appointment",
"deadline": "25 Sept 2025",
"summary": "This looks like an NHS appointment invite (e.g., vaccination). When: 25 Sept. Location: Glasgow Clinic.",
"actions": [
"Add the appointment date/time to your calendar.",
"Bring any requested documents (e.g., child Red Book).",
"If you need to reschedule, call the number on the letter."
],
"citations": ["https://www.nhs.uk/nhs-services/appointments-and-bookings/"]
}
Compose UI:
A simple card shows type, deadline, summary, and actions.
ElevatedCard(Modifier.fillMaxWidth()) {
Column(
Modifier.padding(16.dp),
verticalArrangement = Arrangement.spacedBy(8.dp)
) {
Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) {
Text("Type:", fontWeight = FontWeight.SemiBold)
Text(r.type)
}
Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) {
Text("Deadline:", fontWeight = FontWeight.SemiBold)
Text(r.deadline ?: "—")
}
Divider()
Text("Summary", style = MaterialTheme.typography.titleMedium)
Text(r.summary)
Divider()
Text("Actions", style = MaterialTheme.typography.titleMedium)
r.actions.forEach { a -> Text("• $a") }
Divider()
Text("Citations", style = MaterialTheme.typography.titleMedium)
r.citations.forEach { c -> Text(c) }
}
}
Results
- A letter that takes ~5 minutes to interpret can be summarized in ~10 seconds.
- In my tests on NHS/council letters, the app reliably pulled dates, locations, and required items.
- Clear, low-friction UX reduced the cognitive load for non-technical users.
Lessons Learned
- ML Kit OCR is surprisingly easy to set up in < 20 lines of Kotlin.
- On-device AI ensures privacy (no letter leaves the phone).
- Compose + CameraX makes UI binding smooth.
What’s next
- IOS version with KMP/Swift.
- Multi-lingual support.
- More letter types.
Screenshots
Conclusion
LettersLens demonstrates how small, focused AI tools can make everyday tasks, such as opening a letter, less stressful and more actionable.
Try it
-
Code: LetterLens GitHub Repo
-
Demo video: YouTube Dem
